6–9 Nov 2023
TU Wien, Austria & ONLINE
Europe/Vienna timezone

Course Material

SLIDES

The links will always point to the newest version of the slides.  
The internal cross references in the PDFs work best if the file is downloaded and opened in Acrobat or Preview.

MPI course: mpi_3.1_rab.pdf (PDF)     or     MPI course: mpi_3.1_rab-animated.pdf (PDF)

STANDARD DOCUMENT

MPI: A Message-Passing Interface Standard Version 4.1 (PDF)

MPI4PY DOCUMENTATION

MPI4PY: https://mpi4py.readthedocs.io

EXERCISES

Hands-on labs are prepared for the course participants on the vsc4 & vsc5 clusters (see below the agenda).

Outside of the course the exercises and solutions can be downloaded from: (You'll need either Python + numpy + mpi4py or a C, C++, or Fortran compiler and an up-to-date MPI library, e.g., open-mpi or mpich.) (To verify your MPI library/installation please download either TEST.tar.gz or TEST.zip and do the tests described in the upper part at "A) Testing MPI" in the file TEST/README.txt within the archive.)

MPI31single.tar.gz     or     MPI31single.zip

 


 

AGENDA + SLIDE NUMBERS + HANDS-ON LABS on vsc4 and vsc5 (see below)

 

1st day – 6 November 2023 – Monday

08:45  Join in
09:00Welcome
09:05 
 
MPI overview (slides 9-51) 
--> Excercise 1 (slide 18, demo), Exercise 2 (slide 37, demo), Quiz 1 (slide 51, see Sol.)
10:30Break / breakout rooms (meet your exercise group) + break
10:45 


 
Process model and language bindings (slides 53-74) 
--> Exercise 1 (slide 63, 5 min+demos), Excercise 2 (slide 70, 5 min+demos) 
--> if you are fast, think about Exercise 3 (slide 72, discussion) 
--> if there is still some time left, try the version tests provided in Exercise 4 (slides 73-74)
12:00Break
12:10 

 
Messages and point-to-point communication (slides 77-102) 
--> BR (35 min): Exercise 1+2 (slides 86-88) + (Advanced Exercise 2b (slide 90, only if time)) 
--> BR (10 min): Exercise 3+4+(5 advanced) (slides 98-101), Quiz 3 (slide 102, see Sol.)
13:30End of first day

 

2nd day – 7 November 2023 – Tuesday

08:45  Join in
09:00Ping pong benchmark – solution and results (discussion)

09:10 
 

 

Nonblocking communication (slides 105-137) 
--> Exercise 1 (slides 114-117, demo) 
--> BR (30 min): Exercise 2+(3 advanced) (slides 131-134), 
     Quiz 4 A+B (slides 135-136, see Sol.) + (Quiz 4 C (slide 137, see Sol.))
10:30Break
10:45 


 
Collective communication (slides 161-184 + 186-188) 
--> Exercise 1 (slide 169, demo) + (Advanced Exercise 1b (slide 171, optional homework)) 
--> BR (20 min): Exercise 2 (slide 181) + (Advanced Exercises 1. only (slide 183)), 
     Quiz 6(1) (slide 184, see Sol.)
12:00Break
12:15Optimizing MPI communication – a real world example (discussion)
12:30Short tour: other MPI topics
13:00Fortran and MPI – only for Fortran participants (slides 141-157, +129-130, +56)
13:30End of second day

 

3rd day – 8 November 2023 – Wednesday

08:45  Join in
09:00 

 
Groups & communicators (slides 209-221) 
--> BR (30 min): Exercise 1 (slide 218) + (Exercise 2 advanced (slide 220, only if time)), 
     Quiz 8(1) (slide 221), see Sol.)
10:00Break
10:15 


 
Virtual topologies (slides 241-269 + 274, 279-281 + 286) 
--> BR (20 min): Exercise 1 (slides 252-253) + Advanced Exercise 1 b (slide 255, if time/later) 
--> BR (30 min): Exercise 2 (slides 264-265) + Exercises 3+4 advanced (slides 267-268), 
     Quiz 9(1) (slide 269, see Sol.)                    + Exercises 5+6 advanced (slides 280-281)
12:00Break
12:15 



 
Derived datatypes (slides 425-449) 
--> Exercise 1 (slides 432-433, demo) 
--> BR (30 min): Exercise 2 (slides 445-446) 
                          (Advanced: Exercise 1 b (slide 435), Exercises 3+4 (slide 448) 
     Quiz 12(1) (slide 449, see Sol.)
13:30End of third day

 

4th day – 9 November 2023 – Thursday

08:45  Join in
09:00 


 
One-sided communication (slides 321-360) 
--> BR (20 min): Exercise 1 (slides 337-339) 
--> BR (30 min): Exercises 2 (slides 355-357) + Advanced Exercise 3 (slide 359) 
     Quiz 10 (slide 360, see Sol.)
10:30Break
10:45 


 
Shared memory one-sided communication (slides 361-381 + 400) 
--> BR (20 min): Exercise 1 (slides 368-370) 
--> BR (15 min): Exercise 2 (slides 380-381) 
     Quiz 11(1) (slide 400, see Sol.)
12:00Break
12:15Short tour: MPI I/O (link to pdf)
12:45Best practice (link to pdf), Summary, Q&A
13:30End of fourth day (course)

 

Please provide your feedback (at the end of the course) here:

https://events.vsc.ac.at/event/107/surveys/102

Students, in addition, please provide your evaluation for the lecture VSC-School I:

https://tiss.tuwien.ac.at/survey/surveyForm.xhtml?courseNumber=057020&semesterCode=2023W

 


 

HANDS-ON LABS on vsc4 and vsc5

 

Please connect to the VSC Jupyterhub.

Alternatively, open a terminal and connect to vsc4 (or vsc5) (before you join Zoom for the online course!). If your terminal becomes unresponsive, please hit the Enter key, then type ~. (if it does not help, open a new terminal and connect again)!

 

Editors available on vsc4 and vsc5:

In the VSC Jupyterhub you can find the file in the left menu and edit it quite easily.

vi, vim, nano, emacs [ nedit – only at login nodes ] – editors cheat sheets: Nano, Vi(m)

 

Copy the exercises on vsc4 or vsc5 (just once) --> trainee## users don't need to do this

cd ~ ; cp -a ~training/MPI .

 

On the VSC Jupyterhub NO modules are needed :)

 

When connected via ssh, please load these modules (Intel compiler for C/C++/Fortran OR Python):

vsc4 - C/C++/Fortran:                    module load intel intel-mpi

vsc4 - Python+numpy+mpi4py: 
  
                                                           module load --auto python/3.10.8-gcc-12.2.0-apbi5uz

                                                           module load --auto py-numpy/1.23.4-gcc-12.2.0-zrzyojb

                                                           module load --auto py-mpi4py/3.1.4-gcc-12.2.0-itt5rum   
 

vsc4 - needed on (some) login nodes:   export I_MPI_FABRICS=shm 

 

Go to your exercise directory:

cd ~/MPI/tasks/C/Ch#   or   cd ~/MPI/tasks/F_30/Ch#   or   cd ~/MPI/tasks/PY/Ch#   [# = Chapter number]

 

C/C++/Fortran - Compilation:

                      VSC Jupyterhub (gcc)              Login node (double ii with Intel compiler)

C:                  mpicc my_program.c                 mpiicc my_program.c

C++:             mpic++ my_program.c++           mpiicpc my_program.c++

Fortran:       mpif90 my_program.f90            mpiifort my_program.f90

C/C++/Fortran - Execution:

mpirun -n # ./a.out    [# = number of MPI processes]

 

Python - Execution:

mpirun -n # python3 mpitest.py    [# = number of MPI processes]

 

Execution via SLURM --> this would be the standard way to execute MPI jobs

cp -a ~training/job-mpi.sh .         ! copy a sample job script and edit it accordingly --> C/C++/Fortran

cp -a ~training/job-mpi4py.sh .   ! copy a sample job script and edit it accordingly --> Python

sbatch job-.sh                               ! mpirun -n # ... [# = number of MPI processes] --> devel queue

sq                                                    ! alias for: squeue -u $USER

scancel                                          ! cancel a job

slurm-.out                                      ! output

 

Solutions:

Make sure you have fully completed the exercises before checking the solution, 
otherwise you will loose out on 90% of the learning benefits! 
(Solutions for the quizes and for all exercises can also be found in the slides/appendix.)

 see [# = Chapter number]: 
            ~/MPI/tasks/C/Ch#/solutions/ 
            ~/MPI/tasks/F_30/Ch#/solutions/ 
            ~/MPI/tasks/PY/Ch#/solutions/