Schedule

Unless noted otherwise, all readings are from Reinforcement Learning: An Introduction, 2nd Ed., Sutton and Barto

Date Topic/Notes Reading
Assignment due
1/18 Introduction to RL SB 1.1--1.6
Self Assessment (Solutions); Intro assignment OUT
1/21 Bandit Problems SB 2.1--2.10
Bandits assignment OUT
1/25 Bandit Problems
Intro assignment DUE
1/28 MDPs SB 3.1--3.8
MDP assignment OUT 
2/1 MDPs

Bandits assignment DUE
2/4 Dynamic Programming SB 4.1--4.8
Dynamic Programming assignment OUT
2/8 Dynamic Programming

MDP assignment DUE
2/11 Monte Carlo Methods SB 5.1--5.7 (you can skip Example 5.5)
MC assignment OUT
2/15 Monte Carlo Methods
Project description OUT; DP assignment DUE
2/18 Temporal Difference Learning SB 6.1--6.8
TD Learning assignment OUT
2/22 Temporal Difference Learning
MC assignment DUE
2/25 Planning and Learning SB 8.1--8.6;8.9--8.12
Planning and Learning assignment OUT
3/1 Planning and Learning
TD Learning assignment DUE
3/4 Linear Function Approximation SB 9.1--9.5, 9.8, 10.1
Function approximation assingment OUT; Project proposal DUE
3/8 Deep Learning Overview/ DQN GBC, 6.1--6.4, 9.1--9.3, Mnih, 2014 (DQN)
Planning and Learning assignment DUE
3/11 DQN and extensions Hasselt, 2015 (Double DQN), Schaul, 2016 (Prioritized Replay), Wang, 2015 (Dueling) Mnih, 2016 (A3C)
DQN assignment OUT
3/15 No Class (Spring Break)

3/18 No Class (Spring Break)

3/22 Policy gradient and actor critic SB 13.1--13.7
Function assignment DUE; Policy Gradient assignment OUT
3/25 Deep policy gradient and actor critic Silver, 2014 (DPG), Lillicrap, 2016 (DDPG), Mnih, 2016 (A3C)
3/29 Advanced topics (TBD)  
 DQN assignment DUE
4/1 Advanced topics (TBD)
4/5 Advanced topics (TBD)
PG assignment DUE 
4/8 Advanced topics (TBD)
4/12 Partially observable RL


4/15 Multi-agent RL

4/19
 Representation learning in RL (Lawson Wong)


4/22
 RL for robotics (Rob Platt)


4/26
 Project Presentations


4/29
 Project Presentations


5/1
 Project Reports Due

Report due at 11:59 PM -- This is a hard deadline, no extensions


Important note: unless noted otherwise, all readings and assignments are due on the day that they appear in the schedule.

Unless noted otherwise, all readings are from Artificial Intelligence: A Modern Approach, 3rd Ed., Russell and Norvig.