Hybrid Programming in HPC – MPI+X

Most HPC systems are clusters of shared memory nodes. To use such systems efficiently both memory consumption and communication time has to be optimized. Therefore, hybrid programming may combine the distributed memory parallelization on the node interconnect (e.g., with MPI) with the shared memory parallelization inside of each node (e.g., with OpenMP or MPI-3.0 shared memory).

This course analyzes the strengths and weaknesses of several parallel programming models on clusters of SMP nodes. Multi-socket-multi-core systems in highly parallel environments are given special consideration. MPI-3.0 has introduced a new shared memory programming interface, which can be combined with inter-node MPI communication. It can be used for direct neighbor accesses similar to OpenMP or for direct halo copies, and enables new hybrid programming models. These models are compared with various hybrid MPI+OpenMP approaches and pure MPI.

Numerous case studies and micro-benchmarks demonstrate the performance-related aspects of hybrid programming. Hands-on sessions are included on all days. Tools for hybrid programming such as thread/process placement support and performance analysis are presented in a "how-to" section.

This course provides scientific training in Computational Science and, in addition, the scientific exchange of the participants among themselves.

This course is a joint training event of EuroCC@GCS and EuroCC Austria, the German and Austrian National Competence Centres for High-Performance Computing. It is organized by the HLRS in cooperation with the VSC Research Center, TU Wien and NHR@FAU.

Agenda & Content

See Agenda & Content in the left menu for a detailed timetable and course content.

Content level

Content level: Basic = 0:00h (0%) + Intermediate = 1:30h (10%) + Advanced = 13:30h (90%)

Entry level

Advanced

Prerequisites

Basic MPI and OpenMP knowledge as presented in our VSC or HLRS courses on MPI and OpenMP.   
For the hands-on sessions you should know how to work on the Linux command line and be able to program in either C/C++ or Fortran.

Target audience

Course for academia, industry, and public administration.

Course format

This course will be delivered in HYBRID format, i.e., you can choose either in-person attendance at HLRS, Stuttgart, Germany or online participation via Zoom.

However, in-person participation is recommended for a better learning experience as it will allow you to get the most of the course, to interact closely with the instructors and work together in groups of two during the hands-on labs – a very effective way that allows you to simply learn more!

Hands-on labs

The hands-on labs will be done on the HRLS Hawk supercomputer, all participants will get a training user account on Hawk for the duration of the course.

Lecturers

Claudia Blaas-Schenner (VSC Research Center, TU Wien and EuroCC Austria)

Georg Hager (NHR@FAU, University of Erlangen)

Rolf Rabenseifner (HLRS, University of Stuttgart)

Language

English

Date, time, and location

21.–23.01.2025, 09:00 – 16:30 CET, HLRS, Stuttgart, Germany & ONLINE (Hybrid format)

In-person: Rühle Saal, HLRS, University of Stuttgart, Nobelstraße 19, 70569 Stuttgart, Germany  
Online: Zoom

In-person participation is strongly encouraged for a better learning experience.

Prices and eligibility

This course is partly funded by the EuroCC 2 project. Therefore, the course is open and free of charge for participants from academia, industry, and public administration from the Member States (MS) of the European Union (EU) and Associated/Other Countries to the Horizon 2020 programme.

Registration

Registration for this training event is handled by HLRS through the HLRS course page.

All further details

Please see the HLRS course page for all further details (directions, travel, accommodation).

Organizers

This course is a joint training event of EuroCC@GCS and EuroCC Austria, the German and Austrian National Competence Centres for High-Performance Computing. It is organized by the HLRS in cooperation with the VSC Research Center, TU Wien and NHR@FAU.

Acknowledgements

This course is partially funded by the EuroCC 2 project.

The project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the Digital Europe Programme and Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Montenegro, Netherlands, North Macedonia, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Slovenia, Spain, Sweden, and Türkiye.

Additional funding for the project comes from the Austrian federal ministries BMBWF and BMK.