Reinforcement learning for the control of traffic flow on highways
Thesis (PhD)--Stellenbosch University, 2018.
ENGLISH ABSTRACT: Traffc congestion has become a significant problem around the world, not only in first-world countries, but also in third-world countries such as South Africa. Due to spatial limitations, especially in well-developed metropolitan areas, which typically experience the worst congestion problems, capacity expansion is not always feasible for relieving the pressure on the transportation network. Furthermore, the theory of induced traffic demand suggests that increasing highway capacity is not a long-term solution to traffic congestion due to additional traffic demand on new or updated routes, induced by commuters' perception that new or upgraded routes should be congestion free. As a result, various approaches toward improving highway traffic flow without increasing infrastructure capacity have been proposed in the literature. Ramp metering and variable speed limits are the best-known control measures for effective traffic flow on highways. In most approaches towards solving the control problems presented by these control measures, optimal control techniques or online feedback control have been employed. Feedback control does not, however, guarantee optimality with respect to the on-ramp metering rate or the speed limit chosen, while optimal control techniques are limited to small networks due to their large computational burden. Reinforcement learning is a promising alternative, providing the means and framework required to achieve near-optimal control policies at a fraction of the computational burden associated with optimal control algorithms. In this dissertation, a decentralised reinforcement learning approach is adopted towards simultaneously solving both the ramp metering and variable speed limit control problems. The dawn of the autononomous vehicle promises further improvements in traffic flow which may be achieved over and above those of the aforementioned established highway traffic control measures, if their capabilities are harnessed effectively. A novel method of ramp metering by autonomous vehicles is introduced in this dissertation, based on the premise that specific instructions may be provided to autonomus vehicles travelling along an on-ramp. The control problem presented by this method of ramp metering via autonomous vehicles is also solved using a reinforcement learning approach. The above solution approaches are implemented as a concept demonstrator within a simple, benchmark microscopic highway traffic simulation model. The effectiveness of the decentralised reinforcement learning approach is evaluated by means of statistical comparisons within the context of this simple benchmark simulation model. These approaches are finally applied within the context of a real-world case study simulation model of a section of the N1 highway outbound out of Cape Town, South Africa in order to demonstrate the effectiveness of the approaches within the context of a realistic scenario based on a real highway network and real traffic flow data.
AFRIKAANSE OPSOMMING: Raadpleeg teks vir opsomming