Topics in Machine Learning
- Instructor: Ohad Shamir
- Teaching Assistant: Itay Safran (itay.safran at weizmann dot ac dot il)
Time and Location: Tuesday 1315-1600, Ziskind building, room 1
This course will provide a self-contained introduction to some of the actively-researched areas in machine learning today. It will cover theoretical principles and challenges as well as practical algorithms. The focus will be on supervised and discriminative learning, where the goal is to learn good predictors from data while making few or no probabilistic assumptions. Along the way, we will introduce and use tools from probability, game theory, convex analysis and optimization.
There are no formal prerequisites. However, the course requires mathematical maturity, and students are expected to be familiar with linear algebra and probability, as taught in undergraduate computer science or math programs.
- (1/3) The grades have been submitted to Feinberg. The course average is 91.8. Grades were calculated as ROUND(0.6*TEST + 0.4*EX), where EX is the average assignment grade. The exams are available in room 228 (the departmental secretariat). If you have any questions about the grading, you can contact Itay regarding part A questions 2-5 + part B question 1, and Ohad regarding part A question 1 and part B questions 2-3.
- (21/2) A clarification regarding the second exam linked below, part B question 3 (union of hypothesis classes): It was allowed to assume that the loss function is bounded (with values in [0,1]).
- (20/2) Here is a solution sketch of the exam from two years ago. Also, a general remark: These exams reflect the material and homework provided in those years, not necessarily those of this year's course.
- (13/2) To help you prepare for the exam, we uploaded slightly edited versions of the exams from the two previous years here and here.
- (24/1) Assignment 5 uploaded, due February 7th.
- (18/1) Clarification regarding assignment 4, question 3b: The difference of the risks should be in expectation over i1...ik. This has now been fixed in the assignment PDF.
- (17/1) The 13:15-15:00 lecture today is cancelled due to illness. The 15:15-16:00 tutorial will take place as usual.
- (10/1) Assignment 3 uploaded, due January 24th.
- (29/12) Assignment 3 uploaded, due January 8th.
- (11/12) Clarification regarding assignment 2, question 2b: You may assume the input dimension d is chosen after m is fixed.
- (29/11) Assignment 2 uploaded, due December 13th.
- (29/11) Clarification regarding the representer theorem as learned in class: For the result to be correct, one needs to assume that the function has a minimizer to begin with. In our case, where this is applied to regularized ERM, this is true since the function is strongly convex, and it can be shown that such a function must have a minimizer.
- (27/11) Clarification regarding assignment 1, question 3: x_j refers to the j-th decimal digit of an instance x, and not the j-th instance in the sample.
- (25/11) Clarification regarding assignment 1, question 2 part (b): When bounding this quantity, please also explain what is the probability under which this bound holds.
- (15/11) Assignment 1 uploaded, due November 15th.
The course does not follow any specific text, but much of the first two-thirds is covered by the following:
Additional Sources include:
- Kearns and Vazirani: An Introduction to Computational Learning Theory, MIT Press, 1994
- Cesa-Bianchi and Lugosi: Prediction, Learning and Games, MIT Press, 2006
- Hastie, Tibshirani and Friedman: The elements of statistical learning, Springer, 2009
- Foundations of Machine Learning, Rob Schapire, Princeton
- Introduction to Online Optimization, Sébastien Bubeck, Princeton
- Machine Learning Theory, Avrim Blum, Carnegie Mellon