Anfängerpraktikum (BSc Informatik) - Neural Networks From Scratch

Course overview

Discover the fascination of machine learning by really programming neural networks from scratch . We start with a simple polynomial curve fitting problem and then step by step implement a neural network (Multi-Layer-Perceptron/MLP) as well as its forward and backward pass in “plain Python”. Subsequently, we accelerate our self-written algorithms with CuPy or JAX on the GPUs of our cluster, enabling us to use larger architectures and datasets.

The internship is conducted in groups of up to three students. In the introductory phase , all groups jointly work on the basics: algorithmic foundations, network structure, and GPU acceleration. In the project phase , each group selects its own “from-scratch” topic, such as a CNN architecture, a Transformer, or training an MLP for a specific application using advanced training techniques. Regular feedback sessions accompany you throughout the process.

Throughout the course, we also learn tools for monitoring, management, and visualization of your experiments, enabling you to intuitively track the learning progress and parameters of neural networks.

Lecturers

  • Wang Xiao
  • Hendrik Borras

Contents

  • Polynomial Curve Fitting : Understanding overfitting and underfitting of a complex function to data
  • Multi-Layer-Perceptron : Architecture, forward and backward pass in Python, training on the MNIST dataset
  • CuPy Acceleration : GPU-based training of self-written models
  • Experiment Management & Visualization : Tracking, logging, and plotting of training metrics
  • Project Phase : Own “from-scratch” topic, e.g., CNN, Transformer, training strategies

Requirements

  • Good knowledge of Python and NumPy
  • Basic knowledge of Machine Learning (ML)
    • Expected: General understanding of the structure of a neural network, specifically Multi-Layer Perceptrons. Knowledge of the general training process of a deep neural network (DNN) via backpropagation and stochastic gradient descent.
    • These skills can be acquired through self-study (e.g., Deep Learning, Goodfellow et al.) or through one of the following courses: Learning (IML), Fundamentals of Machine Learning (IFML), Advanced Machine Learning (IAML), Embedded Machine Learning (MScTI_EML), Scalable and Robust Embedded Machine Learning (MScTI_SREML), or similar.
  • Prerequisite Courses
    • Introduction to Practical Computer Science (IPI) or Programming Course (IPK)
    • Linear Algebra 1 (MA4) or Mathematics for Computer Science 1 (IMI1)

Notes

The internship generally follows the procedures for the beginner’s internship/advanced internship B.Sc. Computer Science (see module handbook):

  • Credit points: 6 or 8 ECTS credits
  • Workload: 180 or 240 hours; of which at least 15 hours of presence
  • The examination consists of a presentation in poster session format as well as a report plus documented Python implementation of the project - Schedule and room
    • Wed, 13:30 - 15:00
    • Course start: April 15, 2026
    • Room is OMZ/INF350 basement, U014. Enter the building from the east. If you don’t see a ZITI sign when entering, you might be at the wrong entrance.