Traffic Control using Deep Reinforcement Learning

Abstract

Traffic congestion is the condition that happens on traffic roads in which vehicles move at slower speeds than the normal road speed and the vehicle trips take longer time and queues of vehicles are formed. Traffic congestion has a massive cost. One cause of congestions is traffic intersections in which some of the vehicles have to wait for the others to pass to avoid accidents. Traffic lights are used to control traffic inside an intersection by deciding which lane should be moving and which lane should be waiting to avoid accidents. The way traffic lights normally work is that they allow one lane to move while the conflicting lane is waiting for a fixed period of time and then switch to the opposite for another fixed period passing by an intermediate state in which it signals cars in the moving lane that this lane will be closed soon. These research experiments show the possibility of improving the currently widely used traffic lights controllers without needing huge investments. Solving the traffic congestion problem has many benefits financially and environmentally. The application of artificial intelligence to solving the traffic congestion problem has been going on for a while. However, most of the current research in this area depends on knowing lots of information about all vehicles in the network. While it produces promising results, applying such techniques in the current world is not easy. In this research, the thesis shows the potential of applying reinforcement learning and deep reinforcement learning in the field of traffic control without needing much information. Two models are proposed for controlling the traffic light. Both models show high potential as they beat the currently deployed fixed-time traffic lights in the problems under test. One model outperformed another controller based on longest queue first algorithm. That algorithm has access to more data than the suggested model, but its performance was worse. Multiple configurations of the models were tested in a trial to understand the effect of different hyper parameters on the performance. These models can be deployed to the traffic lights without adding much cost like the approaches presented in most of the recent research in this field which depend on having lots of available data that are practically hard to acquire in today’s world.