Lecture Parallel Programming (2012)




Lecturers prof dr ir H.E. Bal and dr. Rob van Nieuwpoort

Period Fall

Credits 6 ECTS

Time/location Mondays 13.30-15.15 (HG-04A05, main building) and Thursdays 9.00-10.45 (M143)

Content This lecture discusses how programs can be written that run in parallel on a large number of processors, with the goal of reducing execution time. The class has a brief introduction into parallel computing systems (architectures). The focus of the class, however, is on programming methods, languages, and applications. Both traditional techniques (like message passing) and more advanced techniques (like parallel object-oriented languages) will be discussed. Several parallel applications are discussed, including N-body simulations and search algorithms. About 4-5 lectures are devoted to an important new development: programming many-core machines such as Graphical Processing Units (GPUs). The class fits well with existing research projects within the department of Computing Systems. It is a good basis for M.Sc. projects in the area of parallel programming, which use the parallel computing systems of the department.

Form Lectures

Assessment Written exam (no intermediate exams)


NB: There is a separate Practical course Parallel Programming of 6 ECTS that will start after the lectures. Details will eventually be made available on this page .

Literature

The material for the examination exists of:

An example exam is here (April 2008). Example questions for GPUs are here (Jan 2011).

Overhead sheets

The overhead sheets will be put online here before each lecture, together with a reference to the paper(s) from the reader that will be covered.

1. Background

2. Introduction parallel programming

3a. Parallel machines - Introduction

3b. Parallel machines - Cluster computers

3c. Parallel machines - Performance metrics

4. Message passing

5. MPI

6. Java

7. HPF

8. N-body methods

9. Parallel machines - Roofline model

10. Many-core computing: hardware (2/5)

11. Many-core computing: CUDA introduction (3/5)

12. Many-core computing: Advanced CUDA (4/5)

13. Many-core computing: The LOFAR software telescope (5/5)

14. Summary