Development of an adaptive traffic light control system using Markov decision processes

The paper presents the results of development of the MARLIN24 software suite, designed to implement adaptive control of traffic light systems. The primary objective of the development is to optimize the operation of traffic signals in order to minimize the total time vehicles spend within the detection zones of optical sensors. The architecture of the software suite comprises three principal modules: the adaptive control module, the traffic flow simulation module, and the validation module, complemented by an additional visualization module. The adaptive control module integrates four control approaches: fixed-time planning, uncoordinated reinforcement learning, coordinated multi-agent reinforcement learning, and phase duration control based on a MISO fuzzy logic controller (Multiple Input Single Output, MISO). Traffic flow simulation for performance evaluation is conducted via a microsimulation module utilizing the "Intelligent Driver Model" (IDM). The validation module employs copula functions to generate realistic optical sensor data reflecting actual traffic conditions, with marginal distributions derived from historical traffic intensity data collected during 2019-2020. The MARLIN24 software suite facilitates the analysis and comparison of multiple traffic control models on real-world sections of the road network in Krasnoyarsk under various conditions.