Traffic Signal Control Using Reinforcement Learning

Student(s): Ziv Zafir, Barak Farbman
Supervisor(s): Aviv Tamar
Abstract:

Most traffic lights in Israel operate according to a fixed and predetermined schedule. As vehicle sensing solutions (e.g., traffic cameras and road sensors) have become fairly common, it is now possible to design more intelligent and adaptive policies for traffic lights, with the potential of significantly reducing road congestion. The design of such policies however, requires the solution of difficult control problems with large state spaces. This project applies a Reinforcement Learning (RL) algorithm to the problem of traffic signal control. We used an open-source traffic simulator (GLD), and compared the Q-learning and SARSA RL algorithms to several hand-designed policies. Function approximation was used to overcome the problems of the large state space. This approach was proposed in a recent paper by Prashanth and Bhatnagar, and we relied on their results, and introduced some improvements in the features, dynamics, and performance criteria. We present simulations of a road system based on the Horev junction in Haifa, and show that the RL approach out performs the heuristic solutions.