Probabilistic Programming and AI WS23 - TU Wien


Jürgen Cito
University Assistant
Markus Böck


Important! Please register on TISS until Wednesday, 02.10.2023 23:55 (strict deadline!) to be able to participate in this course.

All students registered in TISS until the deadline will have access to the course.
To officially register, you have to complete Assignment 1 (A1), which will be available on TUWEL.
You will be able to deregister until 17.10.2023 23:55, which is also the deadline for A1. As soon as you submit A1, you will receive a certificate (Zeugnis).


We expect that you have working knowledge of Python and are familiar with Jupyter Notebooks.
We also require basic knowledge of probability theory and statistics. This course will feature a lot of mathematics for expository reasons, altough we do not require you to do any mathematical manipulations in the exercises.


The following timetable lists all important dates for the course (lectures, assignment discussion sessions, deadlines, office hours) together with accompanying material (recommended reading, slides as PDFs).

If a deadline is listed on a certain date, assume it due at 23:55 that day unless specified otherwise.

Date Content Recommended Reading
02.10. TISS Registration Deadline
Kick-Off & Lecture 1: FAV Hörsaal 2
04.10. Release Assignment 1 (A1)
Lecture 2:
FAV Hörsaal 3 Zemanek
11.10. Release Assignment 2 (A2)
12.10. Office Hours 13:15-15:00 (online)
17.10. A1 Deadline
Lecture 3:
Lecture 4:
27.10. Office Hours 14:00-15:00 (online)
29.10. A2 Deadline Inspiration for models:
Assignment Discussion Session A1 & A2
Online, Zoom link in TUWEL
Attendance is mandatory!
30.10. Release Assignment 3 (A3)
09.11. Office Hours 13:15-15:00 (online)
If you have any problems, you can contact me via mail or use the TUWEL forum.
15.11. A3 Deadline
15.11. Release Assignment 4 (A4)
Lecture 5:
  • Factorisation of Density and Independence
  • Custom Inference
  • Data-Driven Inference
  • Probabilistic Programs as Proposals
  • Deep Propbabilistic Programming
  • Lecture 5 Slides: Advanced Topics
07.12. Office Hours 14:00-15:00 (online)
13.12. A4 Deadline
13.12. Project Proposal Deadline
Assignment Discussion Session A3 & A4
Online, Zoom link in TUWEL
Attendance is mandatory!
Lecture 6:
  • Guest Lecture: Alex Lew (MIT)
  • Generalizing Automatic Differentiation for Probabilistic Programs
Final Project Presentations
Attendance is mandatory!
Attention: Date changed!
Seminarraum FAV 01 A (Seminarraum 183/2)

Lecture Mode

The six lectures will be held in-person in Seminarraum FAV 01 A (Seminarraum 183/2) (attendance not mandatory).
The assignment discussion session are online via Zoom and attendance is required. We will compare solutions and you can earn bonus points.
Office hours are online via Zoom as well, where students can ask questions regarding the assignments or course material (optional).
The final project presentations will be mainly in-person. Exceptions can be made for those who cannot attend in-person and want to present via Zoom.


In addition to the lecture, you will implement a minimal probabilistic programming language in individual assignments (A1 to A4):
  • Assignment 1 (A1) - Introduction to PPL (10 points): This will be your first hands-on experience with probabilistic programming.
  • Assignment 2 (A2) - Minimal PPL + Likelihood Weighting (10 points): You learn how to implement the core of a PPL in Python and write your first inference algorithm.
  • Assignment 3 (A3) - Metropolis Hastings (10 points): You implement the general-purpose Metropolis Hastings inference algorithm
  • Assignment 4 (A4) - HMC and ADVI (10 points): You implement the state-of-the-art inference algorithms HMC and ADVI
In some assignments you may be asked to test an inference algorithm on a model of your choice. If you do not know what to implement, checkout these resources for inspiration:

Group Project

In groups of 3-4 students, you will propose and work on a final project. There will be a presentation session at the end of the semester, where each group is expected to present for 20 minutes and answer questions.
Projects may be anything related to probabilistic programming.
For instance,
  • Applying probabilistic programming to a non-trivial problem.
  • Implementing an advanced inference algorithm in our minimal PPL.
  • Implementing a PPL following a different design principle.
We will give more project ideas during the lectures.

Example Projects.

Reproducing research papers in a simplified form.

Take inspiration from the Bayesian inference examples given in the first lecture or look for publications that you find interesting. For instance, we don't expect you to do object tracking based on videos, but you can think of implementing a simplified simulator in which object tracking is more feasable. Or, try to break captchas with less noise, fewer letters, etc. For such problems getting the inference to work may be the most challenging part.

Answering questions for real-world data sets with Bayesian Inference.

You may find a real-world data set and pose interesting questions about it. For such problems the modelling aspect will be the most important. Try out several models, argue about your modelling decisions and see how they affect the result. Carefully interpret the results.


If you pick a hard problem, then all of your effort will probably go towards finding one model and implementing one sophistaced inference algorithm that solves it. For such problems, document and share the process of finding a model / inference algorithm combination that made it work.
If you have a "data-science" type of project, then inference maybe won't be too hard. Focus on the modelling aspect and how it affects the result. Document and share the process of modelling the data to answer your questions. Discuss your modelling decisions, carefully interpret the results and present your insights in a convincing manner.

Project Proposals

If your project is about applying probabilistic programming, then the proposal should include at least one paragraph for each of the following questions:
  • What is the objective of your project?
  • How does your data look like? What dataset do you use?
  • How do you intend to model the data? What could be the latent variables? What are the observed variables? ...
  • How do you solve the problem? Which inference algorithm? ...
  • How do you plan to split up the work?
If you want to do a different type of project, then answer following questions in the proposal:
  • What is the objective of your project?
  • How do you achieve your objective? Explain your methodology and be as specific as possible. What do you implement? Which technologies do you use? ...
  • When do you consider your project a success? How do you test your implementation?
  • How do you plan to split up the work?


Your grade will be a combination of assignments scores and final project score.
  • 40% Assignments
  • 60% Project

Grading Scale

The points of the theoretical and practical part sum to exactly 100 points. The points map to grades as follows:
  • S1: 88-100
  • U2: 75-87.99
  • B3: 63-74.99
  • G4: 50-62.99
  • N5: 0-49.99

Academic Honesty

We expect all students to work on their own (both for the practical assignments and the group project). Any kind of plagiarism will result in expulsion of the course with a grade N5 (Nicht Genügend).