
Topics in Machine Learning
(Fall 2016)

[Course description]
[Announcements]
[Lectures]
[Assignments]
[Reading material]
Course description
Course staff:
 Instructor: Ohad Shamir
 Teaching Assistant: Itay Safran (itay.safran at weizmann dot ac dot il)
Time and Location: Tuesday 13151600, Ziskind building, room 1
Syllabus:
This course will provide a selfcontained introduction to some of the activelyresearched areas in machine learning today. It will cover theoretical principles and challenges as well as practical algorithms. The focus will be on supervised and discriminative learning, where the goal is to learn good predictors from data while making few or no probabilistic assumptions. Along the way, we will introduce and use tools from probability, game theory, convex analysis and optimization.
Prerequisites:
There are no formal prerequisites. However, the course requires mathematical maturity, and students are expected to be familiar with linear algebra and probability, as taught in undergraduate computer science or math programs.
Announcements
 (1/3) The grades have been submitted to Feinberg. The course average is 91.8. Grades were calculated as ROUND(0.6*TEST + 0.4*EX), where EX is the average assignment grade. The exams are available in room 228 (the departmental secretariat). If you have any questions about the grading, you can contact Itay regarding part A questions 25 + part B question 1, and Ohad regarding part A question 1 and part B questions 23.
 (21/2) A clarification regarding the second exam linked below, part B question 3 (union of hypothesis classes): It was allowed to assume that the loss function is bounded (with values in [0,1]).
 (20/2) Here is a solution sketch of the exam from two years ago. Also, a general remark: These exams reflect the material and homework provided in those years, not necessarily those of this year's course.
 (13/2) To help you prepare for the exam, we uploaded slightly edited versions of the exams from the two previous years here and here.
 (24/1) Assignment 5 uploaded, due February 7th.
 (18/1) Clarification regarding assignment 4, question 3b: The difference of the risks should be in expectation over i1...ik. This has now been fixed in the assignment PDF.
 (17/1) The 13:1515:00 lecture today is cancelled due to illness. The 15:1516:00 tutorial will take place as usual.
 (10/1) Assignment 3 uploaded, due January 24th.
 (29/12) Assignment 3 uploaded, due January 8th.
 (11/12) Clarification regarding assignment 2, question 2b: You may assume the input dimension d is chosen after m is fixed.
 (29/11) Assignment 2 uploaded, due December 13th.
 (29/11) Clarification regarding the representer theorem as learned in class: For the result to be correct, one needs to assume that the function has a minimizer to begin with. In our case, where this is applied to regularized ERM, this is true since the function is strongly convex, and it can be shown that such a function must have a minimizer.
 (27/11) Clarification regarding assignment 1, question 3: x_j refers to the jth decimal digit of an instance x, and not the jth instance in the sample.
 (25/11) Clarification regarding assignment 1, question 2 part (b): When bounding this quantity, please also explain what is the probability under which this bound holds.
 (15/11) Assignment 1 uploaded, due November 15th.
Lectures
Assignments
Reading material
The course does not follow any specific text, but much of the first twothirds is covered by the following:
Additional Sources include:
 Kearns and Vazirani: An Introduction to Computational Learning Theory, MIT Press, 1994
 CesaBianchi and Lugosi: Prediction, Learning and Games, MIT Press, 2006
 Hastie, Tibshirani and Friedman: The elements of statistical learning, Springer, 2009
 Foundations of Machine Learning, Rob Schapire, Princeton
 Introduction to Online Optimization, Sébastien Bubeck, Princeton
 Machine Learning Theory, Avrim Blum, Carnegie Mellon