Reinforcement Learning

Course Overview

Reinforcement learning studies how an agent can learn to make sequential decisions through interaction with an environment. This course develops the mathematical foundations of reinforcement learning and connects them to modern algorithms used in control, robotics, recommendation, games, and intelligent systems.

The course covers both classical methods and deep reinforcement learning, with an emphasis on understanding why algorithms work, when their assumptions are appropriate, and how to evaluate them reliably.

Learning Outcomes

By the end of the course, students should be able to:

1. Formulate sequential decision problems as Markov decision processes.
2. Derive and implement dynamic-programming, Monte Carlo, and temporal-difference methods.
3. Explain the differences between value-based, policy-based, and actor-critic algorithms.
4. Apply function approximation and deep neural networks to reinforcement-learning problems.
5. Design experiments that evaluate sample efficiency, stability, robustness, and generalization.
6. Read, analyze, and present current reinforcement-learning research.

Prerequisites

Students should be comfortable with:

• Probability and statistics
• Linear algebra and multivariable calculus
• Basic optimization and machine learning
• Python programming

Topics

The course is organized around the following topics:

1. Introduction to sequential decision-making
2. Multi-armed bandits and exploration
3. Markov decision processes
4. Bellman equations and dynamic programming
5. Monte Carlo methods
6. Temporal-difference learning
7. Value-function approximation
8. Policy-gradient methods
9. Actor-critic algorithms
10. Deep reinforcement learning
11. Model-based and offline reinforcement learning
12. Multi-agent, safe, and trustworthy reinforcement learning
13. Research presentations and final projects

The detailed schedule may be adjusted according to the pace of the class and recent developments in the field.

Teaching Material

• Lecture 1 (PDF)

Suggested References

• Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction
• Csaba Szepesvári, Algorithms for Reinforcement Learning
• Selected research papers and lecture notes provided during the course

Academic Integrity and Collaboration

Students must follow the university’s academic-integrity requirements. Discussion and peer learning are encouraged, but submitted work must follow the collaboration rules specified for each assignment. The use of external resources and AI tools must be disclosed when required.