parallel languages

• A parallel system contains multiple processors. In a multiprocessor, the CPUs are connected to a shared memory. In a multicomputer, they are connected through a network.
• Parallelism in a programming language is useful, because (1) some problems are easier to solve in parallel; and (2) many applications can exploit the physical parallelism of parallel machines. Parallel programming means writing a program that uses parallelism for either of these reasons. Distributed programming is writing a program that runs on a collection of machines connected by a network.
• Most parallel imperative languages use processes for expressing parallelism. Alternative constructs are co-routines and parfor and parbegin statements.
• Communication between processes is usually based on shared variables or message passing.
• With shared variables, additional primitives for mutual exclusion and condition synchronization are needed, such as locks, semaphores or monitors.
• Issues in message passing are: how to address the sender and the receiver; whether or not to block the sender; accepting messages explicitly or implicitly; controlling the order in which messages are accepted; and one-way versus two-way communication.
• Rendezvous and Remote Procedure Call are two frequently used high-level language primitives.
• Alternative communication primitives for imperative languages include Tuple Space and shared data-objects.
• To express parallelism and communication, non-imperative languages often use constructs that better fit in with their paradigm. Examples are parallel objects, parallel expressions and AND/OR parallelism.

  Overview of language primitives for parallel and distributed programming.