High Performance and Distributed Computing (HPDC)

Course Overview

On the application side, HPC includes, among others, the computation, modeling, and simulation of complex systems from the areas of astrophysics, particle physics, biology, genetics, quantum chemistry, computational fluids, and weather and climate research. Such workloads have huge requirements with regard to computing power, which (still) cannot be satisfied completely. In this course, we will look at the question of how computer architectures can support such challenging applications in the best way possible. Key aspects are highly parallel hardware architectures like clusters, messaging-based communication, and software environments. Furthermore, we will shortly review scalable training of machine learning models in the second part of the course. The exercises will mainly contain practical work, including the development of own applications.

Contents

• Challenges of HPC
• Message passing
• Parallel programming
• Optimizations
• Collectives
• Interconnection networks
• Scalable training for DNNs

Requirements

Recommended is solid knowledge of C/C++ and the basics of computer architecture.

Notes

• Frequency: summer term
• Among others, course qualifies for the following programs (please double-check listing within heiCO and possible specialization constraints)
  • MSc Data and Computer Science
  • MSc Computer Engineering
  • MSc Scientific Computing
  • MSc Physics

Next/current edition

• Next edition of this course is scheduled for summer 2025
• Course start: April 17, 2025, 14:00 c.t.
• Room is OMZ/INF350 basement, U014. Enter the building from the east. If you don’t see a ZITI sign when entering, you might be at the wrong entrance.
• Moodle has unrestricted enrollment. Course participation is determined by heiCO.