CS 6140: Machine Learning

Announcements

• Prerequisites: CS 5800 or CS 7800 with a minimum grade of C-. I will not enforce these pre-requisites this year. However, note that you are taking the course at your own risk. I will assume the knowledge of algorithms as well as intellectual capacity and work ethic of a student who passed such a course.  I cannot add you to the course directly. Please show this note to your graduate advisor to enroll you in the course if the system blocks you from doing it due to prerequisites. 
• How to prepare for the course? See here for some guidance.  
• First class: Friday, September 6, see 2023-2024 academic calendar 
• Thanksgiving break: November 28-29, no class on November 29
• Midterm exam: Week 8, Tuesday, in class.
• Final exam: Friday, December 6, in class
• Mini project report due: Monday, December 9
• This class will be held on ground.

———————————————————————

Last updated: December 3, 2024

Weekly Schedule

————————————— ——————————

Week 14, December 2 ☀️

Final exam: Friday in class!

Topics

• Kernel machines for graphs

• Review of topics for final exam

Reading materials

• Vishwanathan et al. Graph kernels. J Mach Learn Res, 2010. See here.

Handouts and code

• Kernel machines slides 

————————————— ——————————

Week 13, November 25 

Topics

• Principal component analysis
• Support vector machines

Reading materials

• Textbook #1 (Bishop): Sparse kernel machines (Chapter 7) 
• Section 7.1

Handouts and code

• Principal component analysis slides 
• Support vector machine slides 

Homework assignments

• Mini project instructions available here 

————————————— ——————————

Week 12, November 18 

Topics

• Empirical evaluation
• Committee machines 

Reading materials

• Bootstrapping tutorial: Efron & Tibshirani 1986 
• Textbook #1 (Bishop): Combining models (Chapter 14) 

Handouts and code

• Committee machines slides 
• Committee machines code 

————————————— ——————————

Week 11, November 11 

Topics

• Empirical evaluation
• Convolutional and deep neural networks

Reading materials

• Lecture notes (Radivojac & White): empirical evaluation 
• A nice online tutorial on convolutional networks available here 

Handouts and code

• Empirical evaluation slides 
• Convolutional and deep neural networks slides 

————————————— ——————————

Week 10, November 4 

Topics

• Neural networks

Reading materials

• Textbook #1 (Bishop): Neural networks (Chapter 5)
• Sections 5.1-5.3 (skip 5.3.4), 5.5 (skip 5.5.4 and after)
• The RPROP algorithm can be found here 

Handouts and code

• Neural netowork slides 

Homework assignments

• Assignment #4 available here 

————————————— ——————————

Week 9, October 28 

Topics

• Data preprocessing
• Classification and regression trees

Reading materials

• Tan et al. Introduction to Data Mining (Chapter 2: Data)
• Tan et al., Introduction to Data Mining (Chapter 4)
• Mitchell: Machine learning (Chapter 3)

Handouts and code

• Data preprocessing slides 
• Classification and regression trees slides 

————————————— ——————————

Week 8, October 21

Midterm exam, Tuesday in class.

Topics

• Generalized linear models
• Data preprocessing

Reading materials

• Lecture notes (Radivojac & White): generalized linear models 

Handouts and code

• Generalized linear models slides 

————————————— ——————————

Week 7, October 14 

Topics

• Perceptron
• Logistic regression

Reading materials

• Textbook #1 (Bishop): Linear models for classification (Chapter 4) 
• Sections 4.1 (4.1.1, 4.1.2, 4.1.3, 4.1.7), 4.3 (4.3.2, 4.3.3)

Handouts and code

• Perceptron code 
• Logistic regression code 
• Logistic regression slides 

Homework assignments

• Assignment #3 available here 

————————————— ——————————

Week 6, October 7

Topics

• Regularization
• Basic principles of optimization
• Perceptron
• Review for midterm exam

Reading materials

• Lecture notes (Radivojac & White): optimization 
• Lecture notes (Radivojac & White): linear classification 

Handouts and code

• Linear regression code 
• Optimization slides 
• Perceptron slides 
• Mini project slides 

———————————————————————

Week 5, September 30 

Topics

• Prediction
• Naive Bayes models
• Linear regression
• Linear regression for nonlinear problems

Reading materials

• Lecture notes (Radivojac & White): linear regression  
• Lecture notes (Radivojac & White): radial basis functions 
• Textbook #1 (Bishop): Linear models for regression (Chapter 3) 
• Sections 3.1, 3.2, 3.3 (light reading)

Handouts and code

• Linear regression slides 
• Linear regression for nonlinear problems slides 

Homework assignments

• Assignment #2 available here 

———————————————————————

Week 4, September 23

Topics

• K-means algorithm
• Prediction

Reading materials

• Lecture notes (Radivojac & White): prediction 
• K-means acceleration papers: Elkan 2003, Arthur 2006, Hamerly 2010, Hamerly 2014

Handouts and code

• K-means slides 
• Prediction slides 

———————————————————————

Week 3, September 16

Topics

• Bayesian estimation
• Expectation-maximization algorithm

Reading materials

• Textbook #1 (Bishop): Mixture Models and EM (Chapter 9)
• Sections 9.1, 9.2, 9.3, 9.4 (light reading)

Handouts and code

• EM algorithm code 

————————————— ——————————

Week 2, September 9 

Topics

• Random variables
• Basics of parameter estimation

Reading materials

• Textbook #1 (Bishop): Introduction (Chapter 1)
• Lecture notes (Radivojac & White): parameter estimation  

Handouts and code

• Random variables slides 
• Parameter estimation slides 

Homework assignments

• Assignment #1 available here 

———————————————————————

Week 1, September 2

Topics

• Class overview and logistics
• Short review of probability theory

Reading materials

• Textbook #1 (Bishop): Introduction (Chapter 1)
• Lecture notes (Radivojac & White): probability 

Handouts and code

• Class overview slides 
• Probability theory slides 

———————————————————————

———————————————————————