AI Programming WS25 - TU Wien

Team

Professor
Jürgen Cito
University Assistant
Markus Böck
For Bachelor / Master thesis topics see the project homepage.

Registration

Important! Please register on TISS until 09.10. 12:00 (strict deadline!) to be able to participate in this course.

All students registered in TISS until the deadline, including those on the waiting list, will have access to the TUWEL course.
To officially register, you have to complete Assignment 1 (A1), which will be made available on TUWEL.
You lose your spot in the course if you do not submit A1.
You will be able to deregister until 20.10. 23:55, which is also the deadline for A1.
If you are on the waiting list, you still may want to complete A1. Typically, some students will drop out before A1 and you may get their spot.
If you have submitted A1 and you are not on the waiting list after the registration period (after students who did not submit A1 have been deregistered from the course), you will receive a certificate (Zeugnis).

Prerequisities

We expect that you have working knowledge of Python and are familiar with Jupyter Notebooks.
We also require basic knowledge of probability theory and statistics. This course will feature a lot of mathematics for expository reasons, altough we do not require you to do any mathematical manipulations in the exercises.

Timetable/Lectures

The following timetable lists all important dates for the course (lectures, assignment discussion sessions, deadlines, office hours) together with accompanying material (recommended reading, slides as PDFs).

If a deadline is listed on a certain date, assume it due at 23:55 that day unless specified otherwise.
All lecture dates are cum tempore (c.t.) - they will begin at quarter past.

Date Content Recommended Reading
02.10.
11:00-12:00
FAV Hörsaal 1		 Kick-Off
09.10.
10:00-12:00
FAV Hörsaal 2		 Lecture 1
  • Introduction to Probabilistic Programming
  • Probability and Bayesian Statistics Primer
  • Slides
09.10. TISS Registration Deadline
09.10. TUWEL Course available
09.10. Release Assignment 1 (A1)
20.10. A1 Deadline
20.10. TISS Deregistration Deadline
30.10.
10:00-12:00
FAV Hörsaal 2		 Lecture 2:
  • Bayesian Inference and Generative Modelling
  • Probabilistic Programming Languages
  • Implementation Designs
  • Minimal PPL Implementation
  • Independent Sampling
30.10. Release Assignment 2 (A2) Inspiration for models:
06.11.
10:00-12:00
FAV Hörsaal 2		 Lecture 3:
  • Dependent Sampling
  • Markov Chain Monte Carlo
  • Metropolis Hastings Algorithm
  • Hamiltonian Monte Carlo
06.11. Release Assignment 3 (A3)
13.11.
10:00-12:00
FAV Hörsaal 2		 Lecture 4:
  • Variational Inference
  • Automatic Differentiation VI
  • Stochastic VI
13.11. A2 Deadline
13.11. Release Assignment 4 (A4)
20.11.
10:00-12:00
Zoom		 Assignment Discussion Session A1 & A2
Online, Zoom link in TUWEL
Attendance is mandatory!
27.11.
10:00-12:00
FAV Hörsaal 2		 Lecture 5:
  • Hands-On Probabilistic Programming
  • Facebook Prophet Paper
    		•
    27.11. A3 Deadline
    04.12.
    10:00-12:00
    FAV Hörsaal 2    		 Lecture 6:
    • Custom Inference
    • Data-Driven Inference
    • Probabilistic Programs as Proposals
    • Deep Propbabilistic Programming
  • Paper: On Bayesian Analysis of Mixtures with an Unknown Number of Components
  • RJMCMC / Involutive MCMC in Gen Tutorial
  • Paper: Transforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
  • Data-Driven Proposals in Gen Tutorial
  • Paper: Using probabilistic programs as proposals
  • Paper: Pyro: Deep Universal Probabilistic Programming
  • An Introduction to Probabilistic Programming: Chapter 8 Deep Probabilistic Programming
  • Pyro ELBO Gradients Estimators
  • Paper: Auto-Encoding Variational Bayes
  • Pyro Semi-Supervised Variational Auto-Encoder
    		•
    11.12. A4 Deadline
    11.12.
    10:00-12:00
    Zoom    		 Assignment Discussion Session A3 & A4
    Online, Zoom link in TUWEL
    Attendance is mandatory!
    09.01. Project Report Deadline
    26.01. - 30.1. Oral Exams

    Lecture Mode

    The six lectures will be held in-person in FAV Hörsaal 2 (attendance not mandatory).
    There are four individual assignments which you have to complete on JupyterHub.
    The assignment discussion session are online via Zoom and attendance is required. You will be asked to share your solution to an assignment problem.
    With a project partner you complete a small project probabilistic programming and submit a written report.
    You finish the course with an oral exam at the end of January.

    Assignments

    In addition to the lecture, you will implement a minimal probabilistic programming language in individual assignments (A1 to A4):
    • Assignment 1 (A1) - Introduction to PPL (10 points): This will be your first hands-on experience with probabilistic programming.
    • Assignment 2 (A2) - Minimal PPL + Likelihood Weighting (10 points): You learn how to implement the core of a PPL in Python and write your first inference algorithm.
    • Assignment 3 (A3) - Metropolis Hastings (10 points): You implement the general-purpose Metropolis Hastings inference algorithm
    • Assignment 4 (A4) - HMC and ADVI (10 points): You implement the state-of-the-art inference algorithms HMC and ADVI
    In some assignments you may be asked to test an inference algorithm on a model of your choice. If you do not know what to implement, checkout these resources for inspiration:

    Group Project

    In pairs of 2 students, you will work on a small project.
    In this project, you can either:
    • Apply probabilistic programming to a real-world data set.
    • Implement an inference algorithm in our minimal PPL.
    In January, you submit a three page written report summarising your project.
    More details follow.

    Oral Exam

    At the end of the semester, you take an oral exam with your project partner, where we ask:
    • Questions about your project
    • Questions from a catalog, which we will share with you after the lectures

    Grading

    Your grade will be a combination of assignments scores and final project score.
    • 40% Assignments
    • 60% Project and Exam

    Grading Scale

    The points of the theoretical and practical part sum to exactly 100 points. The points map to grades as follows:
    • S1: 88-100
    • U2: 75-87.99
    • B3: 63-74.99
    • G4: 50-62.99
    • N5: 0-49.99

    Academic Honesty

    We expect all students to work on their own (both for the practical assignments and the group project). Any kind of plagiarism will result in expulsion of the course with a grade N5 (Nicht Genügend).