Statistical Machine Learning

Course Homepage

Fall 2002/03

 

 

Jacob Goldberger

Office Hours: Thursday 10:00 – 11:00 (Ziskind 213)

T.A.: Eli Shechtman and Ira Kemelmacher

 

 

Various Messages:

 

·        Whoever submitted exercises, but does not wish to get credit for the course is kindly asked to notify us.

·        Checked exercises 6 are placed in Ira's mailbox.

·        In exercise 5 the measurement noise variance R should be 100 (instead of 10)

Below is the corrected version.

·        There was a small mistake in exercise 4 (question 3) and in lecture notes: 

The definition of alpha_1(j) should be p(j)p(y_1|x_1=j)

                        Below are the corrected versions.

·        Note that exercises are to be handed individually

·        Please check this page regularly for updates.

·        PDF files can be viewed using Acrobat Reader available free here.

 

Tentative Course Schedule:

 

Date:

Topic

Recommended Reading

 

 

 

31-10

Introduction

 

 

 

 

7-11

EM , MoG

Chapters 9,10

 

 

 

14-11

Factor Analysis

Chapters 12,13

 

 

 

21-11

New view on EM

- Chapter 10

- Neal and Hinton (1998), A view of the EM  http://www.cs.toronto.edu/~hinton/absps/emk.pdf

 

 

 

28-11

Hidden Markov Model

Chapter 11

 

 

 

5-12

Kalman filter and smoother

Chapter 14

 

 

 

12-12

System identification

Ghahramani and Hinton (1996)  http://gatsby.ucl.ac.uk/~zoubin/course02/tr-96-2.pdf

 

 

 

19-12

EKF, The unscented transform

 

 

 

 

26-12

Particle Filter

A Tutorial on Particle Filters for On-line Non-linear/Non-Gaussian Bayesian Tracking

 

 

 

2-01

Variational Approximation

An introduction to variational methods for graphical models. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul. In M. I. Jordan (Ed.), Learning in Graphical Models. Cambridge: MIT Press, 1999

 

 

 

9-01

Belief Propagation

Book of Judea Pearl "Probabilistic reasoning in intelligent system"

 

 

 

16-01

Markov Random Fields

Generalized belief propagation by Yedidia, freeman and Weiss, NIPS 2001

 

 

 

23-01

Loopy Belief Propagation

Generalized belief propagation by Yedidia, freeman and Weiss, NIPS 2001

 

 

 

30-01

Factor graphs and LDPC

Factor Graphs and the Sum Product Algorithm, by Kschichang, Frey and Loeliger. IEEE Trans on info. theory, 2001

 

 

 

 

Exercises:

·        Exercise 1, due date: November 19, 2002

·        Exercise 2, due date: November 21, 2002 

·        Exercise 3, due date: November 28, 2002 

·        Exercise 4, due date: December 12, 2002 

·        Exercise 5, due date: January  2, 2003

Data files: x.dat, y.dat or you can use this mat file. 

·        Exercise 6, due date: January 23, 2003 

 

 

 

Handouts:

·        Mixture of Gaussians

·        Factor Analysis

·        Hidden Markov Model

·        Linear Dynamical Systems

·        Non-Linear Dynamical Systems

 

Literature:

·        Introduction to graphical models, M. Jordan. Can be downloaded from here (password will be given in class).

 

Online Courses:

·        Zoubin Ghahramani, University College London: http://gatsby.ucl.ac.uk/~zoubin/course02/syllabus.html

·        Max Welling, University of Toronto: http://www.cs.toronto.edu/~welling/classnotes/classnotes.html

 

 

 

Last updated: January 30, 2003

Page maintained by Eli Shechtman and Ira Kemelmacher