High Performance and Distributed Computing (HPDC)
Course Overview
On the application side, HPC includes, among others, the computation, modeling, and simulation of complex systems from the areas of astrophysics, particle physics, biology, genetics, quantum chemistry, computational fluids, and weather and climate research. Such workloads have huge requirements with regard to computing power, which (still) cannot be satisfied completely. In this course, we will look at the question of how computer architectures can support such challenging applications in the best way possible. Key aspects are highly parallel hardware architectures like clusters, messaging-based communication, and software environments. Furthermore, we will shortly review scalable training of machine learning models in the second part of the course. The exercises will mainly contain practical work, including the development of own applications.
Contents
- Challenges of HPC
- Message passing
- Parallel programming
- Optimizations
- Collectives
- Interconnection networks
- Scalable training for DNNs
Requirements
Recommended is solid knowledge of C/C++ and the basics of computer architecture.
Notes
- Next edition of this course is currently not scheduled. Most likely, it will be offered again in the 2025 summer term.