Lecture Parallel Programming (2011)

Lecturers prof dr ir H.E. Bal and dr. Rob van Nieuwpoort

Period Fall

Credits 6 ECTS

Time/location Mondays 15.30-17.15 and Thursdays 11.00-12.45 (HG-04A05, main building of VU)

Content This lecture discusses how programs can be written that run in parallel on a large number of processors, with the goal of reducing execution time. The class has a brief introduction into parallel computing systems (architectures). The focus of the class, however, is on programming methods, languages, and applications. Both traditional techniques (like message passing) and more advanced techniques (like parallel object-oriented languages) will be discussed. Several parallel applications are discussed, including N-body simulations and search algorithms. About 4-5 lectures are devoted to an important new development: programming many-core machines such as Graphical Processing Units (GPUs). The class fits well with existing research projects within the department of Computing Systems. It is a good basis for M.Sc. projects in the area of parallel programming, which use the parallel computing systems of the department.

Form Lectures

Assessment Written exam (no intermediate exams)

NB: There is a separate Practical course Parallel Programming of 6 ECTS that will start after the lectures. Details will eventually be made available on this page .

Literature

The material for the examination exists of:

A Reader with papers. Click here for a preliminary description of the reader. The reader is available only in electronic form, from the blackboard site of the lecture (search for "Parallel" under the course catalog of "Faculteit der Exacte Wetenschappen").
Everything that has been discussed during the lectures. The overhead sheets used for the lectures can be found below.

Example exams are here (old one, April 2002) and here (April 2008). Example questions for GPUs are here (Jan 2011).

Overhead sheets

The overhead sheets will be put online here before each lecture, together with a reference to the paper(s) from the reader that will be covered.

1. Background

reader: Introduction to Parallel Computing (Grama)

2. Introduction parallel programming

3a. Parallel machines - Introduction

reader: Parallel Programming (2nd edition); part of Chapter 1 (Wilkinson & Allen)

3b. Parallel machines - Cluster computers

reader: Recent trends in the marketplace of high performance computing (Strohmaier) 3c. Parallel machines - Performance metrics

reader: Parallel Computing - Theory and Practice; part of Chapter 3 (Quinn) 3c. Parallel machines - Roofline model

reader: Roofline: an insightful visual performance model for multicore architectures

4. Message passing

reader: Interprocess Communication and Synchronization based on Message Passing (Bal)

5. MPI

reader: A Message Passing Standard for MPP and Workstations

6. Java

7. HPF

reader: Designing and Building Parallel Programs (Chapter 7, High Performance Fortran)

8. N-body methods

reader: Load Balancing and Data Locality in Adaptive Hierarchical N-body Methods

9. Transposition-Driven Scheduling

reader: Transposition Table Driven Work Scheduling in Distributed Search

10. Many-core computing: hardware (2/5)

11. Many-core computing: CUDA introduction (3/5)

reader: NVIDIA's Next Generation CUDA Compute Architecture: Fermi.

12. Many-core computing: Advanced CUDA (4/5)

13. Many-core computing: The LOFAR software telescope (5/5)

reader: Correlating Radio Astronomy Signals with Many-Core Hardware

14. Summary