- 9/7 - 9/11
- Week 1 / Module 1: Intro, Decision Tree
- Course Map
- Topics:
- Administrative
- Intro to ML, Matrix Data
- Rule-based Classifiers
- Decision/Regression Trees
- Linear Regression
- Background:
- Probabilities
- Convexity1
- Convexity2
- Linear algebra
- Programming: MATLAB, Java, Python, R
- DHS 8.1-8.5
- Other Decision Tree notes
- KMPP ch 16.2
- HW 1
- Due: Mon 9/18
- 9/11 - 9/18
- Week 2 / Module 1: Intro, Regression
- Course Map
- Topics:
- Setup, Cross Validation
- Error, Accuracy, ROC, AUC
Notes: Ridge Regression (normal equations)
- DHS ch 5 (up to 5.4.2)
- KMPP ch 7 (up to 7.5)
- 9/18 - 9/25
- Week 3 / Module 2: Gradient Descent
- Course Map
- Topics:
- Gradient Descent
- Linear Regression with GD
- Logistic Regression
- Newton Method
- DHS ch 5
- KMPP ch 7, 8
- HW2A
- Due: 9/29
- 9/25 - 10/2
- Week 4 / Module 2: Gradient Descent
- Course Map
- Topics:
- Perceptrons
- Neural Networks
- Notes: Perceptrons, Neural Networks
- Screencast: Perceptron
- Screencast: Neural networks
- Book: Intro to Neural Networks
- Visualizer : Neural Networks
- DHS ch 6
- HW2B
- Due: Fri 10/13
- 10/2 - 10/11
- Week 5 / Module 3: Generative
- Course Map
- Topics:
- Probabilities as data densities
- Maximum Likelihood, fit params to data
- Gaussian Discriminant Analysis
- Naive Bayes
- DHS ch 2, 3
- KMPP ch 2, 3, 4
- HW 3
- Due: 10/30
- 10/11 - 10/21
- Week 6 / Module 3: Generative
- Course Map
- Topics:
- EM algorithm for fitting mixtures
- Graphical Models
- KMPP ch 20, KMPP ch 11(EM)
- EM code example
- Wikipedia: EM notes
- Notes (Berkeley): EM
- Notes (other): EM
- Slides (set 1): Other EM
- Graphical Models - Bishop chapter
- Graphical Models Bishop video lecture: Part 1, Part 2, Part 3
- 10/21 - 11/3
- Week 7 / Module 4: Boosting
- Course Map
- Topics:
- Online Learning
- Adaboost Algorithm
- Bagging
- RankBoost, Gradient Boosting
- Optional: Notes: Gradient Boosting, Slides
- Optional: Rankboost for ranking
- Optional: Lambdamart for ranking
- Boosting Classifier Loss gradient
- DHS ch 9
- KMPP ch 16
- Adaboost testing error proof
- AdaBoost/Hedge Theory paper
Boosting preferences - rankboost- LambdaMart Slides, JForests Paper, implementation
- HW 4 (A+B)
- Due: 11/13
- 11/3 - 11/13
- Week 8 / Module 4: Boosting
- Course Map
- Topics:
- Active Learning and VC Dimension
- Multiclass ECOC
- Notes: Active Learning
- Slides: Active Learning, Screencast
- Wikipedia: VC dimension
- Slides: VC dimension
- Slides:Multiclass ECOC
- Paper: Multiclass ECOC, Screencast
- DHS ch 9.5
- KMPP ch 16.6
- KMPP ch 27.6.2
- 11/13 - 11/20
- Week 9 / Module 5: Features
- Course Map
- Topics:
- Margins, Boosting Feature Analysis
- PCA and LDA, Lagrangian Multipliers
- Missing Values
- Margins and Feature Analysis for Boosting (screencast)
PrincipalComponent Analysis (slides, sceencast)
Missing Values and Naive Bayes
optional: Fischer LDA
Slides: tSNE / paper / implementation
optional: tSNE gradient calculation
- PCA: DHS ch 10
- KMPP ch 25
- HW 5
- Due: 11/26
- 11/20 - 11/27
- Week 10 / Module 5: Features
- Course Map
- Topics:
- HAAR Features for Images
- Regularized Regression RIDGE and LASSO
- Text Features
ThanksGiving Break : no class/OH
Wed 11/22-Sun 11/26
- Feature selection. LASSO / RIDGE regularization for
Regression (slides,
screencast)
Haar Features for images - Wikipedia (slides, screencast)
Optional: Features for text: indexing, ngram matching
Optional: CRFs for text
Stanford slides
regularized Logistic Regression-Andrew Ng (paper, screencast)
Haar features integrated with boosting
- 11/27 - 12/2
-
- Week 11 / Module 6: SVM
- Course Map
- Topics:
- Support Vector Machines
- Duality with KKT conditions
- Maximizing Margins Constrained Optimization
- SMO Algorithm
- Notes:Support Vector Machines
- Book Ch: Support Vector Machines
- Convex Optimization
- Slides: Constrained Optimization
- DSH ch 5.11
- KMPP ch 14
- HW 6
- Due: 12/13
- 12/2 - 12/9
- Week 12 / Module 6: SVM, Kernels
- Course Map
- Topics:
- Kernels
- Kernels for SVM
paper: Kernel Methods in Machine Learning
- 12/2 - 12/9
- Week 13 / Module 7: Locality,KNN
- Course Map
- Topics:
- Catching up
- Recap
- Extra Demo Days
- 12/4 - 12/11
- Week 14 / Module 7: Locality and Similarity
- Course Map
- Topics:
- K-Nearest Neighbor
- Kernel Similarity and KNN
- Kernel Density Estimation
- Heat Kernels, Harmonic Equation
- Collaborative Filtering
- KMPP ch 14, ch 1
- DHS ch 4
- Clustering: DHS ch 3.8, KMPP ch 12.2
- Mon Dec 11, WVH rooms 210-212 Exam
2pm-6pm
Final Exam, you'll need a laptop. Expect programming problems similar to HW, about 1h to code and 15 min to run. You will have access to internet, but what you implement has to be your own.
Submit a copy of your code on gradescope together with running instructions. The TAs are encouraging you to use Jupyter Notebook, but this is not a requirement for the exam.