Probabilistic Graphical Models for Image Analysis

Fall Semester 2014

This course will focus on inference with statistical models for image analysis. We use a framework called probabilistic graphical models which include Bayesian Networks and Markov Random Fields. We apply the approach to traditional vision problems such as image denoising, as well as recent problems such as object recognition. The course covers amongst others the following topics:

Traditional Supervised Learning: Logistic and Linear Regression, Naive Bayes
Directed and undirected Graphical Models: Markov Random Fields and Bayesian networks
Inference in Graphical Models: Sum-Product Algorithm, Graph-Cut, Belief Propagation, Variational Methods and Sampling
Applications of Graphical Models in vision: Conditional Random Fields and Structured Output Learning


12/18/2014 Added solutions for SSVM exercises.
12/17/2014 Added solutions for LBP exercises.
12/11/2104 Added homework for SSVM lecture.
12/10/2014 Added SSVM slides and solutions for SVM and CRF lectures.
12/04/2014 Added LBP exercises and solutions for factored Gaussians example.
12/03/2014 Added SVM slides and homework for SVM lecture.
11/26/2014 Added CRF slides and homework for CRF lecture.
11/17/2014 Added Sampling slides.
11/05/2014 Added solutions for belief nets and belief prop and reading for Loopy BP.
10/27/2014 Added homework for belief prop and Variational slides.
10/22/2014 Added solutions for Holmes/Watson network and another inference exercise.
10/22/2014 Added solutions for homework 5 and 6.
10/13/2014 Added solutions for homework 4.
10/02/2014 Added lecture 3 slides and additional exercises for lecture 1.
09/23/2014 Added more reading for lecture 1.


Lecturer Dr. Brian McWilliams, Dr. Aurelien Lucchi

Time and Place


Monday 15:00 - 16:00 CAB G 51
Thursday 10:00 - 12:00 CLA E 4


30 Minute oral exam in English.


DateTopicSlidesAdditional ExercisesReadingBackground Material
09/18/14 Introduction/Learning from Data Lecture hw solutions 1
hw solutions 2
hw solutions 3
hw solutions 4
Barber Ch. 1 notes on machine learning
probability background
09/22/14 Introduction/Learning from Data (cont.) Learning from data basics
Barber Ch. 1 , 8, 13.2 , 17.1, 18.1.1
09/25/14 Probabilistic models Lecture hw solutions 1
hw solutions 2
Barber Ch. 8, 10 Ghahramani on Bayesian modeling
Nice example of a generative model
09/29/14 Probabilistic models Barber Ch. 17.4, 29.3-5
10/02/14 Belief Networks Lecture worked example solutions
Inference in Belief nets (solutions)

Barber Ch. 2, 3
10/09/14 Markov Random Fields Lecture hw4 solutions
Barber Ch. 4
10/16/14 Learning as Inference Lecture hw5 solutions
Barber Ch. 9
10/16/14 MAP inference Lecture
hw6 solutions
Barber Ch. 9, 28.9 1. energy minimization via graph-cuts
2. texture synthesis
3. photomontage
10/23/14 Belief Propagation Lecture Barber Ch. 5
10/27/14 Belief Propagation (cont.) Belief-prop homework
10/27/14 Variational Approximation Lecture
Barber Ch. 18.2.2, 28
11/06/14 Variational Approximation (cont.) Lecture
Additional exercises
Solution to factored Gaussians
Barber Ch. 28
11/06/14 Loopy Belief Propagation Lecture
LBP exercises
Barber 28.7
Wainwright and Jordan 3-4.1.6
Challis and Barber. Gaussian Kullback-Leibler Approximate Inference
11/17/14 Sampling Lecture
Barber Ch. 27
11/27/14 Conditional Random Fields Lecture
hw11 solutions
Barber 9.6.5 and 23.4.3 Intro to CRFs
Application to image segmentation
Learning CRFs with graph cut
12/01/14 No class
12/04/14 SVMs Lecture
SVM tutorial
Learning the kernel
Discriminative MRFs
12/11/14 Structured SVMs Lecture
12/15/14 No class


Lecture Wednesday 9 - 11 h ML F 34
Exercise Wednesday 11 - 12 h ML F 34


D. Barber Bayesian Reasoning and Machine Learning Cambridge University Press 2012 The main course text. Brand new book which covers many topics in graphical models and machine learning.
M. Wainwright and M.I. Jordan Graphical models, exponential families and variational inference Foundations and Trends in Machine Learning 2008 Advanced treatment of graphical models and variational inference
David J.C. Mackay Information Theory, Inference and Learning Algorithms Cambridge University Press, 2003
C. Bishop Pattern Recognition and Machine Learning Springer 2007 This is an excellent introduction to machine learning that covers most topics which will be treated in the lecture. Contains lots of exercises, some with exemplary solutions.
D. Koller and N. Friedman Probabilistic Graphical Models: Principles and Techniques The MIT Press 2009 Covers Bayesian networks and undirected graphical models in great detail.

Frequently Asked Questions

What is a good reference for probability theory required for the course? See Barber Ch. 1. and MacKay: Ch. 2, 3. Make sure you are comfortable with the exercises in the first week's slides too.
What is the scope of the course? We cover material from Part I (all), II and III (some) and V (all) of Barber. We look briefly at the first four sections of Wainwright & Jordan.