Learning Cooperative Max-Pressure Control by Leveraging Downstream Intersections Information for Traffic Signal Control

Peng, Yuquan and Li, Lin and Xie, Qing and Tao, Xiaohui ORCID: https://orcid.org/0000-0002-0020-077X (2021) Learning Cooperative Max-Pressure Control by Leveraging Downstream Intersections Information for Traffic Signal Control. In: 5th International Joint Conference on Asia-Pacific Web and Web-Age Information Management, Part II (APWeb-WAIM 2021), 23 Aug - 25 Aug 2021, Guangzhou, China.


Abstract

Traffic signal control problems are critical in urban intersections. Recently, deep reinforcement learning demonstrates impressive performance in the control of traffic signals. The design of state and reward function is often heuristic, which leads to highly vulnerable performance. To solve this problem, some studies introduce transportation theory into deep reinforcement learning to support the design of reward function e.g., max-pressure control, which have yielded promising performance. We argue that the constant changes of intersections’ pressure can be better represented with the consideration of downstream neighboring intersections. In this paper, we propose CMPLight, a deep reinforcement learning traffic signal control approach with a novel cooperative max-pressure-based reward function to leverage the vehicle queue information of neighborhoods. The approach employs cooperative max-pressure to guide the design of reward function in deep reinforcement learning. We theoretically prove that it is stabilizing when the average traffic demand is admissible and traffic flow is stable in road network. The state of deep reinforcement learning is enhanced by neighboring information, which helps to learn a detailed representation of traffic environment. Extensive experiments are conducted on synthetic and real-world datasets. The experimental results demonstrate that our approach outperforms traditional heuristic transportation control approaches and the state-of-the-arts learning-based approaches in terms of average travel time of all vehicles in road network.


Statistics for USQ ePrint 46211
Statistics for this ePrint Item
Item Type: Conference or Workshop Item (Commonwealth Reporting Category E) (Paper)
Refereed: Yes
Item Status: Live Archive
Additional Information: Files associated with this item cannot be displayed due to copyright restrictions.
Faculty/School / Institute/Centre: Historic - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sep 2019 - 31 Dec 2021)
Faculty/School / Institute/Centre: Historic - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sep 2019 - 31 Dec 2021)
Date Deposited: 11 May 2022 23:23
Last Modified: 31 May 2022 00:03
Uncontrolled Keywords: Deep reinforcement learning; Traffic signal control; Cooperative max-pressure; Downstream information
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080105 Expert Systems
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining
Fields of Research (2020): 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460207 Modelling and simulation
46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460308 Pattern recognition
46 INFORMATION AND COMPUTING SCIENCES > 4605 Data management and data science > 460502 Data mining and knowledge discovery
Socio-Economic Objectives (2008): E Expanding Knowledge > 97 Expanding Knowledge > 970108 Expanding Knowledge in the Information and Computing Sciences
Socio-Economic Objectives (2020): 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280115 Expanding knowledge in the information and computing sciences
Identification Number or DOI: https://doi.org/10.1007/978-3-030-85899-5_29
URI: http://eprints.usq.edu.au/id/eprint/46211

Actions (login required)

View Item Archive Repository Staff Only