MVA Course - Introduction to Statistical Learning

2023/2024

***Instructors:
Prof.
Nicolas VAYATIS
TAs : Marie Garin - Gaëtan Serré

***Emails:
<firstname.name/-at-/ens-paris-saclay.fr>

***Course schedule and location:


Date Time Room number Instructor Session Topics Material
Tuesday October 3
08:30-10:30 1Z34 N. Vayatis Lecture #1 Chapter 1 - Optimality in binary classification
Data/Objectives/Optimal elements/ERM
Slides
LDA
Tuesday October 10
08:30-10:30 1Z34 N. Vayatis Lecture #2 Chapter 1 - Optimality in statistical learning
Other problems/Complexity of learning
Slides
Tuesday October 17
08:30-10:30 1Z34 G. Serré Exercise session #1 Optimal elements

Set
Tuesday October 24
08:30-10:30 1Z34 N. Vayatis Lecture #3 Chapter 2 - Mathematical foundations (I)
Probabilistic inequalities, complexity measures
Slides
Tuesday October 31
08:30-10:30 1Z34 M. Garin Exercise session #2 Inequalities, Rademacher complexity, VC dimension

Set
Tuesday November 7
08:30-10:30 1Z34
Partial exam - Mandatory
No documents


Tuesday November 14
08:30-10:30 1Z34 N. Vayatis Lecture #4 Chapter 2 - Mathematical foundations (II)
Regularization and stability
Slides
Link
Tuesday November 21
08:30-10:30 1Z34 N. Vayatis Lecture #5 Chapter 3 - Consistency of Machine Learning methods (I)
Margin bounds and application to SVM
Slides
Tuesday November 28 08:30-10:30 1Z34 G. Serré Exercise session #3 Consistency and convergence bounds

Set
Tuesday December 5
08:30-10:30 1Z34 N. Vayatis Lecture #6 Chapter 3 - Consistency of Machine Learning methods (II)
Ensemble methods: Bagging, Random Forests, Boosting
Slides
Tuesday December 12 08:30-10:30 1Z34 N. Vayatis Lecture #7 Chapter 3 - Consistency of Machine Learning methods (III)
Neural networks, Mirror Descent
Slides
Tuesday December 19 08:30-10:30 1Z34 M. Garin Exercise session #4 Course wrap-up

Set
Monday January 8
08:30-10:30 1Z34
Final exam - Mandatory
Documents allowed




***Office hours (on demand):
Tuesdays 10:30am/11:30am


***Evaluation:
Partial exam on November 7 (08:30am-10:30am) - MANDATORY - No documents
Final exam on January 8 (08:30am-10:30am) - MANDATORY - With documents

***Grading:
Course grade = max(final ; (partial+final)/2)

 *************

Contents

Chapter 1 - Optimality in statistical learning:
    From information theory to statistical learning
    The probabilistic view on numerical data,
   
Optimality and the bias-variance dilemma
Chapter 2 - The mathematical foundations of statistical learning:
    Risk minimization
    Concentration inequalities
    Complexity measures
    Regularization and Stability
Chapter 3 - Theory: Consistency theorems and error bounds of learning algorithms

    Part 1 - SVM
    Part 2 - Ensemble methods: Bagging, Random Forests, Boosting
    Part 3 - Neural Networks, Mirror Descent

Past exams


/ Final exam 2020 / Partial exam 2020 / Final exam 2019 / Partial exam 2019 / Final exam 2018 / Partial exam 2018  / Final exam 2017 / Partial exam 2017


References