EE290 starts Aug. 23rd.

EE290O | Deep multi-agent reinforcement learning with applications to autonomous traffic

Prerequisites for this class

Not required but helpful

Course Instructors

Alexandre Bayen

Eugene Vinitsky

Aboudy Kreidieh

Yashar Zeiynali Farid

Cathy Wu

Course Description

In this class, students will learn the fundamental techniques of machine learning (ML) / reinforcement learning (RL) required to train multi-agent systems to accomplish autonomous tasks in complex environments. Foundations include reinforcement learning, dynamical systems, control, neural networks, state estimation, and partially observed Markov decision processes (POMDPs). Core methods include Deep Q Networks (DQN), actor-critic methods, and derivative-free methods. Multi-agent reinforcement learning topics include independent learners, action-dependent baselines, MADDPG, QMIX, shared policies, multi-headed policies, feudal reinforcement learning, switching policies, and adversarial training. The students will have the opportunity to implement the techniques learned on a multi-agent simulation platform, called Flow, which integrates RL libraries and SUMO (a state-of-the-art microsimulation software) on AWS EC2. The students may alternatively implement the techniques learned on their own platforms or platforms of their choice (in which case they are responsible for implementation). The class will teach applications of the ML/RL methods in the context of urban mobility and mixed autonomy, i.e., insertion of self driving vehicles in human-driven traffic. Thus the class will also includes an introduction to traffic modeling to enable the students to perform meaningful simulations, on benchmark cases as well as concrete calibrated models with field data.

Learning Outcomes

By the end of the class students should be able to:

Class Time and Location

Fall Semester (August 23 - December ??, 2018)
Lecture: Tuesday, Thursday 3:30-5:00pm
Location: 531 Cory Hall

Course Schedule / Syllabus (Including Due Dates)

See the Course Schedule page.



There is no official textbook for the class but a number of the supporting readings will come from:

Some other additional references that may be useful are listed below:

Grade Breakdown

Late Day Policy

Homework submissions

Regrading Requests

Office Hours

All office hours will be held in McLaughlin 109 at TBD


Attendance is not required but is encouraged. Lectures are not recorded. Sometimes we may do in class exercises or discussions and these are harder to do and benefit from by yourself.


We believe students often learn an enormous amount from each other as well as from us, the course staff. Therefore to facilitate discussion and peer learning, we request that you please use Piazza for all questions related to lectures, homework and projects. When discussing solutions on Piazza, take care not to post words, code, or math that directly leads to solutions.

You will be awarded with up to 2% extra credit if you answer other students' questions in a substantial and helpful way on Piazza.


Announcements will be posted via email.