MVA Course - Introduction to Statistical Learning

2022/2023

***Instructors:
Prof. Nicolas VAYATIS
TA : Marie Garin
***Emails:
<firstname.name/-at-/ens-paris-saclay.fr>

***Course schedule and location:

On ENS Paris-Saclay campus --> how to get there
Date Time Room number Instructor Session Topics Material
Monday October 3
 11:30-13:30 1Z34 N. Vayatis Lecture #1 Chapter 1 - Optimality in binary classification
Data/Objectives/Optimal elements/ERM		 Slides
Tuesday October 11
 08:00-10:00 1Z34 N. Vayatis Lecture #2 Chapter 1 - Optimality in statistical learning
Other problems/Complexity of learning
 Slides
Monday October 17
 09:30-11:30 1Z56 M. Garin Exercise session #1 Optimal elements

 Set
Monday October 24
 12:00-14:00 1Z34 N. Vayatis Lecture #3 Chapter 2 - Mathematical foundations (I)
Probabilistic inequalities, complexity measures
 Slides
Friday November 4
 10:00-12:00 1Z34 M. Garin Exercise session #2 Inequalities, Rademacher complexity, VC dimension

 Set
Tuesday November 8
 09:00-11:00 1Z34
Partial exam - Mandatory
No documents

Monday November 21
 14:00-16:00 1Z34 N. Vayatis Lecture #4 Chapter 2 - Mathematical foundations (II)
Regularization and stability
 Slides
Link
Tuesday November 22
 09:00-11:00 1Z28 N. Vayatis Lecture #5 Chapter 3 - Consistency of Machine Learning methods (I)
Margin bounds and application to SVM
 Slides
Monday November 28 16:00-18:00 1Z34 M. Garin Exercise session #3 Consistency and convergence bounds

 Set
Monday December 5
 14:00-16:00 1Z34 N. Vayatis Lecture #6 Chapter 3 - Consistency of Machine Learning methods (II)
Neural networks, Bagging, Random Forests
 Slides
Monday December 12 14:00-16:00 1Z34 M. Garin Exercise session #4 Course wrap-up

 Set
Monday January 9
 10:00-12:00 1Z34
Final exam - Mandatory
Documents allowed



***Office hours (on demand):
Tuesdays 10am/11am

***Evaluation:
Partial exam on November 8 (9am-11am) - MANDATORY - No documents
Final exam on January 9 (10am-12am) - MANDATORY - With documents

***Grading:
Course grade = max(final ; (partial+final)/2)
 *************

Contents

Chapter 1 - Optimality in statistical learning:
    From information theory to statistical learning
    The probabilistic view on numerical data,
    Optimality and the bias-variance dilemma
Chapter 2 - The mathematical foundations of statistical learning:
    Risk minimization
    Concentration inequalities
    Complexity measures
    (Explicit) Regularization
    (Stability)
Chapter 3 - Theory: Consistency theorems and error bounds of learning algorithms
    Part 1 - SVM, Boosting
    Part 2 - Neural Networks, Bagging, Random Forests
(Chapter 4 - Advanced topics)

Some lecture notes

Unofficial (and confidential!) lecture notes will be distributed after Lecture #6
Disclaimer: this document was offered by a former student, some mistakes may be found in the document.


Past exams


/ Final exam 2020 / Partial exam 2020 / Final exam 2019 / Partial exam 2019 / Final exam 2018 / Partial exam 2018  / Final exam 2017 / Partial exam 2017


References