Abstract:

Deep reinforcement learning (DRL) has been widely used in adaptive traffic signal control (ATSC), but the existing algorithms can't capture the overall intersection state well, and also lack the consideration of the influence of complex traffic flow composition on the signal control effect. In this paper, a deep learning algorithm based on KAHGN-Q is proposed, which can extract the traffic flow information of each entrance of the target intersection and the adjacent intersections, and obtain a complete and comprehensive intersection state representation. A new graph neural network input architecture is built, which divides traffic flow into three levels, divides nodes into direction level nodes, lane type nodes and vehicle type nodes, and couples macro traffic flow and micro vehicle composition characteristics. D3QN reinforcement learning with prioritized experience replay incorporating dynamic priorities (PERDP) is employed to continuously learn an optimal action-selection policy while ensuring full coverage of all action strategies. Different weights are used for different vehicle types in the reward setting, which can be used to realize bus priority. The experimental results show that KAHGN-Q algorithm has advantages in reducing the average waiting time and average delay of vehicles.

Key words: urban transportation, traffic signal control, deep reinforcement learning, heterogeneous graph neural network, knowledge distillation