Selected Topics on Machine and Deep Learning (STMDL), 2024

Meetings

This is the course schedule:

• Week 1 (03/06) - Least Squares
  • rec, notes, code-m, code-mlx
  • Supplementary videos: Linear Least Squares, Khan Academy
• Week 2: 10/06 - Model Evaluation
  • rec, notes, code-m
  • Homework data. Each student should use different dataset for the best \(N\) value evaluation.
• Week 3: 17/06 - Overfitting management: Regularization, standardization
  • rec
  • notes,
  • code-m
• Week 4: 24/06 - Kernel regression
  • rec
  • notes,
  • code-m, code-m
• Week 5: 01/07 - Regression metrics and losses
  • References:
    • A survey and taxonomy of loss functions in machine learning (arXiv)
    • A Comprehensive Overview of Regression Evaluation Metrics (Nvidia)
    • 5 Regression Loss Functions All Machine Learners Should Know
    • Mean Squared Logarithmic Error Loss
    • Huber and logcosh loss functions
    • Dealing with Outliers Using Three Robust Linear Regression Models (Nvidia)
  • notes
  • rec
  • code-m
• Week 6: 08/07 - Classification and logistic regression, k-NN
  • rec
  • notes
  • code-m
• Week 7: 15/07 - Classification metrics
  • Reference: Classification Performance by Google
  • rec
  • notes
  • updated logistic_regression_2d_example.m from logistic regression code-m
• Week 8: 22/07 - Signal classification: feature extraction and selection
  • rec
  • notes

• Week 9: 29/07 - lecture cancelled

• Week 10: 05/08 - Plotting, visualization and PCA
  • rec
  • notes
  • code-Ellipse
  • code-PCA-compression, code-PCA
  • code-PCA-Iris
  • code-Plots
  • Reference: Anscombe’s quartet
  • Reference: Colormap

• Week 11: 12/08 - No lecture

• Week 12: 19/08 - Introduction to neural networks
  • References:
    • A Quick Introduction to Vanilla Neural Networks (link)
    • On-line visualization of neural networks at TensorFlow Playground with some explanations from Google
    • Understanding Deep Learning book by Simon J.D. Prince: Chapters 3 and 4 (download page)
    • But what is a neural network? 3Blue1Brown video
    • Comprehensive synthesis of the main activation functions pros and cons (link)
    • Commonly used activation functions on stackexchange
    • Visualization of backpropagation and (Xavier) initialization by Katanforoosh & Kunin, “Initializing neural networks”, deeplearning.ai, 2019
    • GD modifications and stochastic GD by Katanforoosh, Kunin et al., “Parameter optimization in neural networks”, deeplearning.ai, 2019
    • Gradient descent, how neural networks learn 3Blue1Brown video
  • rec (including homework)
  • notes
  • code-homework