TY - GEN
T1 - Minimizing energy consumption from connected signalized intersections by reinforcement learning
AU - Bin Al Islam, S. M.A.
AU - Aziz, H. M.Abdul
AU - Wang, Hong
AU - Young, Stanley E.
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/12/7
Y1 - 2018/12/7
N2 - Explicit energy minimization objectives are often discouraged in signal optimization algorithms due to its negative impact on mobility performance. One potential direction to solve this problem is to provide a balanced objective function to achieve desired mobility with minimized energy consumption. This research developed a reinforcement learning (RL) based control with reward functions considering energy and mobility in a joint manner-a penalty function is introduced for number of stops. Further, we proposed a clustering-based technique to make the state-space finite which is critical for a tractable implementation of the RL algorithm. We implemented the algorithm in a calibrated NG-SIM network within a traffic micro-simulator-PTV VISSIM. With sole focus on energy, we report 47% reduction in energy consumption when compared with existing signal control schemes, however causing a 65.6% increase in system travel time. In contrast, the control strategy focusing on energy minimization with penalty for stops yields 6.7% reduction in energy consumption with 27% increase in system travel time. The developed RL algorithm with a flexible penalty function in the reward will achieve desired energy goals for a network of signalized intersections without compromising on the mobility performance. Disclaimer: This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).
AB - Explicit energy minimization objectives are often discouraged in signal optimization algorithms due to its negative impact on mobility performance. One potential direction to solve this problem is to provide a balanced objective function to achieve desired mobility with minimized energy consumption. This research developed a reinforcement learning (RL) based control with reward functions considering energy and mobility in a joint manner-a penalty function is introduced for number of stops. Further, we proposed a clustering-based technique to make the state-space finite which is critical for a tractable implementation of the RL algorithm. We implemented the algorithm in a calibrated NG-SIM network within a traffic micro-simulator-PTV VISSIM. With sole focus on energy, we report 47% reduction in energy consumption when compared with existing signal control schemes, however causing a 65.6% increase in system travel time. In contrast, the control strategy focusing on energy minimization with penalty for stops yields 6.7% reduction in energy consumption with 27% increase in system travel time. The developed RL algorithm with a flexible penalty function in the reward will achieve desired energy goals for a network of signalized intersections without compromising on the mobility performance. Disclaimer: This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).
KW - Reinforcement learning
KW - connected vehicles
KW - energy minimization
KW - fuel consumption
KW - traffic state observability
UR - http://www.scopus.com/inward/record.url?scp=85060472446&partnerID=8YFLogxK
U2 - 10.1109/ITSC.2018.8569891
DO - 10.1109/ITSC.2018.8569891
M3 - Conference contribution
AN - SCOPUS:85060472446
T3 - IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC
SP - 1870
EP - 1875
BT - 2018 IEEE Intelligent Transportation Systems Conference, ITSC 2018
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 21st IEEE International Conference on Intelligent Transportation Systems, ITSC 2018
Y2 - 4 November 2018 through 7 November 2018
ER -