Class description:
Machine
learning is a fast-pacing and exciting field achieving human-level performance in
tasks such as image classification, speech recognition. machine translation,
precision medicine, and self-driving cars. Machine learning has already
impacted greatly our daily lives and has the potential to transform the world
even more in the near future. This course will provide a broad introduction to
machine learning and cover the fundamental algorithms for supervised and
unsupervised learning. We will cover topics related to regression,
classification, ensemble learning, neural networks, and deep learning. The
class will also provide an introduction into adversarial machine learning, an
emerging area that studies the fundamental security issues of machine learning,
Instructors:
Class Schedule:
· Monday and Wednesday, 2:50-4:30pm
· Location: WVH 108
Office Hours:
· Alina: Wednesday, 4:30-6:00pm, ISEC 625
· Ewen: Thursday, 5:00-6:00pm, ISEC 605
· Christopher: Monday, 5:00-6:00pm, ISEC 605
Class policies:
· We will use a Piazza forum and Gradescope for homework submission.
· Academic integrity policy is strictly enforced.
Pre-requisites:
· Probability
· Statistics
· Linear algebra
(Materials covered in class DS 5020)
· [ISL] Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. An Introduction to Statistical Learning with Applications in R. [PDF]
· [ESL] Trevor Hastie, Rob Tibshirani, and Jerry Friedman, Elements of Statistical Learning, Second Edition, Springer, 2009.
· [Bishop] Christopher Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
· [D2L] A. Zhang, Z. Lipton, and A. Smola. Dive into Deep Learning
The
grade will be based on:
- Assignments – 20%
- Final project report and presentation – 25%
- Midterm exam – 25%
- Final exam – 25%
- Class participation – 5%
|
Unit |
Week |
Date |
Topic |
Readings |
|
Introduction and review |
1 |
Wed 09/04 |
Course outline (syllabus, grading, policies) Introduction to supervised learning [PDF] |
[ISL] Chapters 1 and 2 |
|
2 |
Mon 09/09 |
Learning problems and challenges Linear algebra and probability review. [PDF] |
Linear algebra
review from Stanford Probability review
from Stanford [Bishop]
1.2 |
|
|
Linear regression |
Wed 09/11 |
Bivariate normal distribution. Simple linear regression. Maximum likelihood for linear
regression [PDF] |
[RW] Chapter 1 [ESL] Chapters 3.1-3.2 [RW] Chapter 5.1 |
|
|
3 |
Mon 09/16 |
Multiple linear regression. Gradient descent for linear regression [PDF] |
[RW] Chapter 5.2 [RW] Chapter 2 |
|
|
|
Wed 09/18 |
Linear classification Perceptron, LDA Guest lecture [PDF] |
[ESL] Chapter 4.3 |
|
|
4 |
Mon 09/23 |
LDA. Bias-variance decomposition [PDF] |
[RW] Chapter 5.3 [ESL] Chapter 4.3 |
|
|
Linear classification and evaluation Metrics |
Wed 09/25 |
Gradient descent Regularization Ridge regression [PDF] |
[RW] Chapter 2 [ESL] Chapter 3.4 |
|
|
|
5 |
Mon 09/30 |
k-Nearest Neighbors (kNN). Cross validation [PDF] Logistic regression |
|
|
|
Wed 10/02 |
CLASS CANCELLED Python Programming Session in ISEC 655, 5-6pm |
||
|
6 |
Mon 10/07 |
Gradient Descent for Logistic Regression. Metrics, ROC curves Evaluation of ML [PDF] |
[RW] Chapter 8.1 |
|
|
Decision trees and ensembles |
Wed 10/09 |
Density estimation. Naïve Bayes. KDE [PDF] |
[ESL] Chapter 6.6.3 [RW] Chapter 8.2 |
|
|
|
7 |
Mon 10/14 |
Columbus Day, no class |
|
|
Wed 10/16 |
Feature selection Information Gain Decision trees [PDF] |
[ESL] Chapter 9.2.3 Tree handout |
||
|
|
88 |
Mon 10/21 |
Decision trees, cont. [PDF] |
Project
proposal due |
|
|
Wed 10/23 |
Midterm prep [PDF] |
[ |
|
|
|
9 |
Mon 10/28 |
Midterm exam |
|
|
Non-linear classifiers |
Wed 10/30 |
Ensemble learning Bagging; Random forests Boosting; AdaBoost [PDF] |
[ESL] Chapter 10.1 [ESL] Chapter 15.1-15.3 |
|
|
|
10 |
Mon 11/04 |
AdaBoost Neural networks and deep learning. [PDF] |
http://cs229.stanford.edu/notes/cs229-notes-deep_learning.pdf |
|
Neural networks and deep learning |
Wed 11/06 |
Feed-Forward Networks. Forward Propagation. [PDF] |
Optional: [D2L] Chapter 4 |
|
|
|
11 |
Mon 11/11 |
Veterans Day, no class |
|
|
|
Wed 11/13 |
Multi-class classification Convolutional neural networks. [PDF] |
Optional: [D2L] Chapter 6 |
|
|
12 |
Mon 11/18 |
CNN Regularization [PDF] |
Project milestone due |
|
|
|
Wed 11/20 |
Backpropagation
[PDF] |
|
|
|
Security of ML |
13 |
Mon 11/25 |
SVM Maximum margin classifier. Kernels [PDF] |
[ESL] Chapter 4.5.2 [ESL] Chapter 12.1, 12.2, 12.3.1 |
|
|
Wed 11/27 |
Thanksgiving; no class |
|
|
|
Review Presentations Exam |
14 |
Mon 12/02 |
Review and exam preparation [PDF] |
|
|
|
Wed 12/04 |
Final exam |
|
|
|
|
|
12/09 |
Project presentations (tentative) |
|
|
|
|
12/10 |
Project report due |
|