Literature and slides
Literature (mandatory)
Studymanual Probabilistic Reasoning [pdf], author: S. Renooij.
The studymanual describes in detail how the course is assessed, and the level of knowledge and understanding expected. The studymanual also contains questions to aid in your selfstudy.Syllabus Probabilistic Reasoning with Bayesian networks [pdf], by L.C. van der Gaag, S. Renooij. (Possibly also available in paper from study association A-Eskwadraat.)Paper RR04: General state space Markov chains and MCMC algorithms by Gareth O. Roberts, Jeffrey S. Rosenthal (2004): read up to and including 3.2 thoroughly, and skim the rest.
Literature (optional)
For the Probabilistic Programming part:Book BDA3: Bayesian Data Analysis (3rd edition; 2021) by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, Donald Rubin: Ch 1, up to and including 1.7.
- Notes on Probabilistic Models: nice introduction to Bayesian parameter estimation, implicitly showing that inference (be it exact or approximate) and parameter learning are very similar tasks. Probabilistic models can be learned from data; in the syllabus we focus on structure learning for PGMs; in probabilistic programming you specify the 'structure' and learn only the parameters from data.
- Introduction to Bayesian Inference for Psychology by Alexander Etz and Joachim Vandekerckhove (2018): provides a nice introduction to Bayes rule, its use in Bayesian inference for learning model parameters and for hypothesis testing, both for discrete and continuous variables. Interesting if you like a perspective that is different from the mathematical or computer science / AI one.
Course slides (mandatory)
The lectures cover most of the above-mentioned literature, but some subjects will be discussed in more detail. You should be familiar with the additional details covered in the course slides:- these slides can be found on Blackboard (in Course Content)
Video Lectures (optional)
🎦 For the 2020 online edition of the course the story that goes with the slides was recorded in videoclips. Each clip has its own subject and lasts about 10-30 minutes. These 'V-lectures' can be found on the UU video platform. Note that course slides have changed since 2020 and that the topic of probabilistic programming is not covered at all in the videos.
For a better understanding of the link between the PGM/Bayesian network part and the probabilistic programming part: see the material for the Introduction to Probabilistic Models (Nordic ProbAI 2024, Day 1)(when last checked, the videos were not yet uploaded, but slides are).Further viewing: optional slides, videos etc
- online courses, tutorials and background info:
- Buffalo Probabilistic Graphical Models course;
- The online (Coursera) Stanford Probabilistic Graphical Models courses;
- The Helsinki B-Course;
- video lectures, search on e.g. videolectures.net, youtube.com, see e.g. various videos on Graphical models and d-separation.
Further reading: optional textbooks
- textbooks on Bayesian networks (with solutions to lots of exercises on books' own websites):
- Bayesian Networks and Decision Graphs, by Finn V. Jensen and Thomas D. Nielsen (2007)
- Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis, by Uffe B. Kjærulff and Anders L. Madsen (2013; second edition)
- Bayesian Networks in R, by Radhakrishnan Nagarajan, Marco Scutari and
Sophie Lébre (2013)
- on probability theory and statistics (includes exercises, applets and data sets): Random (formerly: the Virtual
Laboratories in Probability and Statistics)
- on d-separation (history and explanation) and Bayes-Ball: D-separation; Bayes-Ball (slides 36-40);
- on independence relations and Pearl's algorithm:
J. Pearl
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Morgan Kaufmann Publishers, 1988.
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Morgan Kaufmann Publishers, 1988.
- a lot more on independence relations and (semi-)graphoids:
M. Studený
On Probabilistic Conditional Independence Structures
Information Science and Statistics, 2005. (requires UU IP-address)
On Probabilistic Conditional Independence Structures
Information Science and Statistics, 2005. (requires UU IP-address)