SLIDES
The links will always point to the newest version of the slides.
The internal cross references in the PDFs work best if the file is downloaded and opened in Acrobat or Preview.
MPI course: mpi_3.1_rab.pdf (PDF) or MPI course: mpi_3.1_rab-animated.pdf (PDF)
STANDARD DOCUMENT
MPI: A Message-Passing Interface Standard Version 4.1 (PDF)
MPI4PY DOCUMENTATION
MPI4PY: https://mpi4py.readthedocs.io
EXERCISES
Hands-on labs are prepared for the course participants on the vsc4 & vsc5 clusters (see below the agenda).
Outside of the course the exercises and solutions can be downloaded from: (You'll need either Python + numpy + mpi4py or a C, C++, or Fortran compiler and an up-to-date MPI library, e.g., open-mpi or mpich.) (To verify your MPI library/installation please download either TEST.tar.gz or TEST.zip and do the tests described in the upper part at "A) Testing MPI" in the file TEST/README.txt within the archive.)
MPI31single.tar.gz or MPI31single.zip
AGENDA + SLIDE NUMBERS + HANDS-ON LABS on vsc4 and vsc5 (see below)
1st day – 6 November 2023 – Monday
| 08:45 | Join in |
| 09:00 | Welcome |
| 09:05 | MPI overview (slides 9-51) --> Excercise 1 (slide 18, demo), Exercise 2 (slide 37, demo), Quiz 1 (slide 51, see Sol.) |
| 10:30 | Break / breakout rooms (meet your exercise group) + break |
| 10:45 | Process model and language bindings (slides 53-74) --> Exercise 1 (slide 63, 5 min+demos), Excercise 2 (slide 70, 5 min+demos) --> if you are fast, think about Exercise 3 (slide 72, discussion) --> if there is still some time left, try the version tests provided in Exercise 4 (slides 73-74) |
| 12:00 | Break |
| 12:10 | Messages and point-to-point communication (slides 77-102) --> BR (35 min): Exercise 1+2 (slides 86-88) + (Advanced Exercise 2b (slide 90, only if time)) --> BR (10 min): Exercise 3+4+(5 advanced) (slides 98-101), Quiz 3 (slide 102, see Sol.) |
| 13:30 | End of first day |
2nd day – 7 November 2023 – Tuesday
| 08:45 | Join in |
| 09:00 | Ping pong benchmark – solution and results (discussion) |
09:10
| Nonblocking communication (slides 105-137) --> Exercise 1 (slides 114-117, demo) --> BR (30 min): Exercise 2+(3 advanced) (slides 131-134), Quiz 4 A+B (slides 135-136, see Sol.) + (Quiz 4 C (slide 137, see Sol.)) |
| 10:30 | Break |
| 10:45 | Collective communication (slides 161-184 + 186-188) --> Exercise 1 (slide 169, demo) + (Advanced Exercise 1b (slide 171, optional homework)) --> BR (20 min): Exercise 2 (slide 181) + (Advanced Exercises 1. only (slide 183)), Quiz 6(1) (slide 184, see Sol.) |
| 12:00 | Break |
| 12:15 | Optimizing MPI communication – a real world example (discussion) |
| 12:30 | Short tour: other MPI topics |
| 13:00 | Fortran and MPI – only for Fortran participants (slides 141-157, +129-130, +56) |
| 13:30 | End of second day |
3rd day – 8 November 2023 – Wednesday
| 08:45 | Join in |
| 09:00 | Groups & communicators (slides 209-221) --> BR (30 min): Exercise 1 (slide 218) + (Exercise 2 advanced (slide 220, only if time)), Quiz 8(1) (slide 221), see Sol.) |
| 10:00 | Break |
| 10:15 | Virtual topologies (slides 241-269 + 274, 279-281 + 286) --> BR (20 min): Exercise 1 (slides 252-253) + Advanced Exercise 1 b (slide 255, if time/later) --> BR (30 min): Exercise 2 (slides 264-265) + Exercises 3+4 advanced (slides 267-268), Quiz 9(1) (slide 269, see Sol.) + Exercises 5+6 advanced (slides 280-281) |
| 12:00 | Break |
| 12:15 | Derived datatypes (slides 425-449) --> Exercise 1 (slides 432-433, demo) --> BR (30 min): Exercise 2 (slides 445-446) (Advanced: Exercise 1 b (slide 435), Exercises 3+4 (slide 448) Quiz 12(1) (slide 449, see Sol.) |
| 13:30 | End of third day |
4th day – 9 November 2023 – Thursday
| 08:45 | Join in |
| 09:00 | One-sided communication (slides 321-360) --> BR (20 min): Exercise 1 (slides 337-339) --> BR (30 min): Exercises 2 (slides 355-357) + Advanced Exercise 3 (slide 359) Quiz 10 (slide 360, see Sol.) |
| 10:30 | Break |
| 10:45 | Shared memory one-sided communication (slides 361-381 + 400) --> BR (20 min): Exercise 1 (slides 368-370) --> BR (15 min): Exercise 2 (slides 380-381) Quiz 11(1) (slide 400, see Sol.) |
| 12:00 | Break |
| 12:15 | Short tour: MPI I/O (link to pdf) |
| 12:45 | Best practice (link to pdf), Summary, Q&A |
| 13:30 | End of fourth day (course) |
Please provide your feedback (at the end of the course) here:
https://events.vsc.ac.at/event/107/surveys/102
Students, in addition, please provide your evaluation for the lecture VSC-School I:
https://tiss.tuwien.ac.at/survey/surveyForm.xhtml?courseNumber=057020&semesterCode=2023W
HANDS-ON LABS on vsc4 and vsc5
Please connect to the VSC Jupyterhub.
Alternatively, open a terminal and connect to vsc4 (or vsc5) (before you join Zoom for the online course!). If your terminal becomes unresponsive, please hit the Enter key, then type ~. (if it does not help, open a new terminal and connect again)!
Editors available on vsc4 and vsc5:
In the VSC Jupyterhub you can find the file in the left menu and edit it quite easily.
vi, vim, nano, emacs [ nedit – only at login nodes ] – editors cheat sheets: Nano, Vi(m)
Copy the exercises on vsc4 or vsc5 (just once) --> trainee## users don't need to do this
cd ~ ; cp -a ~training/MPI .
On the VSC Jupyterhub NO modules are needed :)
When connected via ssh, please load these modules (Intel compiler for C/C++/Fortran OR Python):
vsc4 - C/C++/Fortran: module load intel intel-mpi
vsc4 - Python+numpy+mpi4py:
module load --auto python/3.10.8-gcc-12.2.0-apbi5uz
module load --auto py-numpy/1.23.4-gcc-12.2.0-zrzyojb
module load --auto py-mpi4py/3.1.4-gcc-12.2.0-itt5rum
vsc4 - needed on (some) login nodes: export I_MPI_FABRICS=shm
Go to your exercise directory:
cd ~/MPI/tasks/C/Ch# or cd ~/MPI/tasks/F_30/Ch# or cd ~/MPI/tasks/PY/Ch# [# = Chapter number]
C/C++/Fortran - Compilation:
VSC Jupyterhub (gcc) Login node (double ii with Intel compiler)
C: mpicc my_program.c mpiicc my_program.c
C++: mpic++ my_program.c++ mpiicpc my_program.c++
Fortran: mpif90 my_program.f90 mpiifort my_program.f90
C/C++/Fortran - Execution:
mpirun -n # ./a.out [# = number of MPI processes]
Python - Execution:
mpirun -n # python3 mpitest.py [# = number of MPI processes]
Execution via SLURM --> this would be the standard way to execute MPI jobs
cp -a ~training/job-mpi.sh . ! copy a sample job script and edit it accordingly --> C/C++/Fortran
cp -a ~training/job-mpi4py.sh . ! copy a sample job script and edit it accordingly --> Python
sbatch job-.sh ! mpirun -n # ... [# = number of MPI processes] --> devel queue
sq ! alias for: squeue -u $USER
scancel ! cancel a job
slurm-.out ! output
Solutions:
Make sure you have fully completed the exercises before checking the solution,
otherwise you will loose out on 90% of the learning benefits!
(Solutions for the quizes and for all exercises can also be found in the slides/appendix.)
see [# = Chapter number]:
~/MPI/tasks/C/Ch#/solutions/
~/MPI/tasks/F_30/Ch#/solutions/
~/MPI/tasks/PY/Ch#/solutions/
