Spatiotemporal attention mechanism-based multistep traffic volume prediction model for highway toll stations

Zijing Huang; Peiqun Lin; Xukun Lin; Chuhao Zhou; Tongge Huang

doi:10.5604/01.3001.0015.8148

Authors

Zijing Huang School of Civil Engineering and Transportation, South China University of Technology, Guangzhou Author https://orcid.org/0000-0002-8437-2244
Peiqun Lin School of Civil Engineering and Transportation, South China University of Technology, Guangzhou Author https://orcid.org/0000-0002-1812-0620
Xukun Lin School of Civil Engineering and Transportation, South China University of Technology, Guangzhou, China Department of Transportation of Guangdong Guangzhou, Author
Chuhao Zhou School of Civil Engineering and Transportation, South China University of Technology, Guangzhou Author https://orcid.org/0000-0001-7406-4187
Tongge Huang Institute of Software, Chinese Academy of Sciences, Beijing Author https://orcid.org/0000-0002-5534-4623

DOI:

https://doi.org/10.5604/01.3001.0015.8148

Keywords:

traffic volume forecasting, attention mechanism, highway toll station, model interpretation, ITS

Abstract

As the fundamental part of other Intelligent Transportation Systems (ITS) applications, short-term traffic volume prediction plays an important role in various intelligent transportation tasks, such as traffic management, traffic signal control and route planning. Although Neural-network-based traffic prediction methods can produce good results, most of the models can’t be explained in an intuitive way. In this paper, we not only proposed a model that increase the short-term prediction accuracy of the traffic volume, but also improved the interpretability of the model by analyzing the internal attention score learnt by the model. we propose a spatiotemporal attention mechanism-based multistep traffic volume prediction model (SAMM). Inside the model, an LSTM-based Encoder-Decoder network with a hybrid attention mechanism is introduced, which consists of spatial attention and temporal attention. In the first level, the local and global spatial attention mechanisms considering the micro traffic evolution and macro pattern similarity, respectively, are applied to capture and amplify the features from the highly correlated entrance stations. In the second level, a temporal attention mechanism is employed to amplify the features from the time steps captured as contributing more to the future exit volume. Considering the time-dependent characteristics and the continuity of the recent evolutionary traffic volume trend, the timestamp features and historical exit volume series of target stations are included as the external inputs. An experiment is conducted using data from the highway toll collection system of Guangdong Province, China. By extracting and analyzing the weights of the spatial and temporal attention layers, the contributions of the intermediate parameters are revealed and explained with knowledge acquired by historical statistics. The results show that the proposed model outperforms the state-of-the-art model by 29.51% in terms of MSE, 13.93% in terms of MAE, and 5.69% in terms of MAPE. The effectiveness of the Encoder-Decoder framework and the attention mechanism are also verified.

References

Connor, J. T., Martin, R. D., & Atlas, L. E. (1994). Recurrent neural networks and robust time series prediction. IEEE Transactions on Neural Networks, 5(2), 240-253. DOI: 10.1109/72.279188.

Cui, Z., Henrickson, K., Ke, R., & Wang, Y. (2020). Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. IEEE Transactions on Intelligent Transportation Systems, 21(11), 4883-4894. DOI: 10.1109/TITS.2019.2950416.

Du, S., Li, T., Gong, X., & Horng, S.-J. (2020). A hybrid method for traffic flow forecasting using multimodal deep learning. International Journal of Computational Intelligence Systems, 13(1), 85-97. DOI: 10.2991/ijcis.d.200120.001.

Du, S., Li, T., Yang, Y., Gong, X., & Horng, S.-J. (2019). An LSTM based encoder-decoder model for multistep traffic flow prediction. In 2019 international joint conference on neural networks, IJCNN 2019, july 14 - july 19, 2019 (Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc. DOI: 10.1109/IJCNN.2019.8851928.

Feng, X., Ling, X., Zheng, H., Chen, Z., & Xu, Y. (2019). Adaptive multi-kernel SVN with spatial-temporal correlation for short-term traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 20(6), 2001-2013. DOI: 10.1109/TITS.2018.2854913.

Ge, W., Ding, Z., Cao, Y., & Guo, L. (2019). Forecasting model of traffic flow prediction model based on multiresolution SVR. In 3rd international conference on innovation in artificial intelligence, ICIAI 2019, march 15-18, 2019 (Vol. Part F148152, p. 1-5). Association for Computing Machinery. DOI: 10.1145/3319921.3319923.

Ghosh, B., Basu, B., & O’Mahony, M. (2007). Bayesian time-series model for short-term traffic flow forecasting. Journal of Transportation Engineering, 133(3), 180-189. DOI: 10.1061/(ASCE)0733947X(2007)133:3(180).

Giraka, O., & Selvaraj, V. K. (2020). Short-term prediction of intersection turning volume using seasonal arima model. Transportation Letters, 12(7), 483-490. DOI: 10.1080/19427867.2019.1645476.

Guo, J., Huang, W., & Williams, B. M. (2014). Adaptive Kalman filter approach for stochastic short-term traffic flow rate prediction and uncertainty quantification. Transportation Research Part C: Emerging Technologies, 43, 50-64. DOI: 10.1016/j.trc.2014.02.006.

Hong, W.-C. (2011). Traffic flow forecasting by seasonal SVR with chaotic simulated annealing algorithm. Neurocomputing, 74(12-13), 2096-2107. DOI: 10.1016/j.neucom.2010.12.032.

Hu, X., Wang, W., & Lu, J. (2010). Urban short-term traffic flow forecasting based on the semi-variable cell transmission model. In Traffic and transportation studies 2010 - proceedings of the 7th international conference on traffic and transportation studies (Vol. 383, p. 861-871). American Society of Civil Engineers (ASCE). DOI: 10.1061/41123(383)81.

Hu, X., Wang, W., & Sheng, H. (2010). Urban traffic flow prediction with variable cell transmission model. Journal of Transportation Systems Engineering and Information Technology, 10(4), 73-78. DOI: 10.1016/S1570-6672(09)60055-6.

Huang, W., Shen, F., & Yang, X. (2008). Research on the characteristic and applicability of traffic flow simulation based on CTM. In 8th international conference of Chinese logistics and transportation professionals - logistics: The emerging frontiers of transportation and development in China, july 31- august 3, 2008 (p. 1837-1842). ASCE - American Society of Civil Engineers. DOI: 10.1061/40996(330)269.

Huang, W., Song, G., Hong, H., & Xie, K. (2014). Deep architecture for traffic flow prediction: Deep belief networks with multitask learning. IEEE Transactions on Intelligent Transportation Systems, 15(5), 2191-2201. DOI: 10.1109/TITS.2014.2311123.

Ji, Y., Daamen, W., Zhang, X., & Sun, L. (2009). Traffic incident recovery time prediction model based on cell transmission model. In 2009 12th international IEEE conference on intelligent transportation systems, ITSC ’09, october 3-7, 2009 (p. 809-812). Institute of Electrical and Electronics Engineers Inc. DOI: 10.1109/ITSC.2009.5309829.

Karlaftis, M. G., & Vlahogianni, E. (2011). Statistical methods versus neural networks in transportation research: differences, similarities and some insights. Transportation Research Part C: Emerging Technologies, 19(3), 387-399. DOI:10.1016/j.trc.2010.10.004.

Kyriakou, K., Lakakis, K., Savvaidis, P., & Basbas, S. (2019). Analysis of spatiotemporal data to predict traffic conditions aiming at a smart navigation system for sustainable urban mobility. Archives of Transport, 52(4), 27-46. DOI: 10.5604/01.3001.0014.0206.

Li, S., Shen, Z., & Xiong, G. (2012). A k-nearest neighbor locally weighted regression method for short-term traffic flow forecasting. In 2012 15th international IEEE conference on intelligent transportation systems, ITSC 2012, septembers 16-19, 2012 (p. 1596-1601). Institute of Electrical and Electronics Engineers Inc. DOI: 10.1109/ITSC.2012.6338648.

Li, T., Yang, Y., Wang, Y., Chen, C., & Yao, J. (2016). Traffic fatalities prediction based on support vector machine. Archives of Transport, 39(3), 21-30. DOI: 10.5604/08669546.1225447.

Lin, F., Xu, Y., Yang, Y., & Ma, H. (2019). A spatial-temporal hybrid model for short-term traffic prediction. Mathematical Problems in Engineering, 2019, IV. DOI: 10.1155/2019/4858546.

Lu, Z., Lv, W., Cao, Y., Xie, Z., Peng, H., & Du, B. (2020). LSTM variants meet graph neural networks for road speed prediction. Neurocomputing, 400, 34-45. DOI: 10.1016/j.neu-com.2020.03.031.

Luo, X., Li, D., Yang, Y., & Zhang, S. (2019). Spatiotemporal traffic flow prediction with KNN and LSTM. Journal of Advanced Transportation, 2019. DOI: 10.1155/2019/4145353.

Lv, Y., Duan, Y., Kang, W., Li, Z., & Wang, F.-Y. (2015). Traffic flow prediction with big data: A deep learning approach. IEEE Transactions on Intelligent Transportation Systems, 16(2), 865-873. DOI: 10.1109/TITS.2014.2345663.

Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., & Wang, Y. (2017). Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors (Switzerland), 17(4). DOI: 10.3390/s17040818.

Ma, X., Tao, Z., Wang, Y., Yu, H., & Wang, Y. (2015). Long short-term memory neural net-work for traffic speed prediction using remote microwave sensor data. Transportation Research Part C: Emerging Technologies, 54, 187-197. DOI: 10.1016/j.trc.2015.03.014.

Newell, G. F. (2002). A simplified car-following theory: A lower order model. Transportation Research Part B: Methodological, 36(3), 195-205. DOI: 10.1016/S0191-2615(00)00044-8.

Park, H.-C., Kim, D.-K., & Kho, S.-Y. (2018). Bayesian network for freeway traffic state prediction. Transportation Research Record, 2672(45), 124-135. DOI: 10.1177/0361198118786824.

Qi-ming, W., Ai-wan, F., & He-sheng, S. (2017). Network traffic prediction based on improved support vector machine. International Journal of System Assurance Engineering and Management, 8(3), S1976-S1980. DOI: 10.1007/s13198-016-0412-8.

Qin, Y., Song, D., Cheng, H., Cheng, W., Jiang, G., & Cottrell, G. W. (2017). A dual-stage attention-based recurrent neural network for time series prediction. In 26th international joint conference on artificial intelligence, IJCAI 2017, august 19-25, 2017 (Vol. 0, p. 2627-2633). International Joint Conferences on Artificial Intelligence.

Sun, B., Cheng, W., Goswami, P., & Bai, G. (2018). Short-term traffic forecasting using self-adjusting k-nearest neighbours. IET Intelligent Transport Systems, 12(1), 41-48. DOI: 10.1049/iet-its.2016.0263.

Sun, S., Zhang, C., & Yu, G. (2006). A bayesian network approach to traffic flow forecasting. IEEE Transactions on Intelligent Transportation Systems, 7(1), 124-133. DOI: 10.1109/TITS.2006.869623.

Wang, J., Deng, W., & Guo, Y. (2014). New bayesian combination method for short-term traffic flow forecasting. Transportation Research Part C: Emerging Technologies, 43, 79-94. DOI: 10.1016/j.trc.2014.02.005.

Wang, Y., Xiao, Y., Lai, J., & Chen, Y. (2020). An adaptive k nearest neighbour method for imputation of missing traffic data based on two similarity metrics. Archives of Transport, 54(2), 59-73. DOI: 10.5604/01.3001.0014.2968.

Wang, Z., Ji, S., & Yu, B. (2019). Short-term traffic volume forecasting with asymmetric loss based on enhanced KNN method. Mathematical Problems in Engineering, 2019. DOI: 10.1155/2019/4589437.

Wang, Z.-W. (2019). The trip characteristic analysis of guangdong highway based on net-work toll collection. Journal of Guangdong Communication Polytechnic, 18(03), 20-25.

Williams, B. M., & Hoel, L. A. (2003). Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results. Journal of Transportation Engineering, 129(6), 664-672. DOI: 10.1061/(ASCE)0733947X(2003)129:6(664).

Xiao, X., Duan, H., & Wen, J. (2020). A novel car-following inertia gray model and its application in forecasting short-term traffic flow. Applied Mathematical Modelling, 87, 546-570. DOI: 10.1016/j.apm.2020.06.020.

Xie, B., Xu, M., Harri, J., & Chen, Y. (2013). A traffic light extension to cell transmission model for estimating urban traffic jam. In 2013 IEEE 24th annual international symposium on personal, indoor, and mobile radio communications, PIMRC 2013, september 8-11, 2013 (p. 2566-2570). Institute of Electrical and Electronics Engineers Inc. DOI: 10.1109/PIMRC.2013.6666579.

Xu, D., Wang, Y., Peng, P., Beilun, S., Deng, Z., & Guo, H. (2020). Real-time road traffic state prediction based on kernel-KNN. Transport-metrica A: Transport Science, 16(1), 104-118. DOI: 10.1080/23249935.2018.1491073.

Xu, J., Zhang, Y., Jia, Y., & Xing, C. (2018). An efficient traffic prediction model using deep spatial-temporal network. In 14th EAI international conference on collaborative computing: Networking, applications and worksharing, collaboratecom 2018, december 1-3, 2018 (Vol. 268, p. 386-399). Springer Verlag. DOI: 10.1007/978-3030-12981-127.

Yang, Q., & Koutsopoulos, H. N. (1996). Microscopic traffic simulator for evaluation of dynamic traffic management systems. Transportation Research Part C: Emerging Technologies, 4(3), [d]113-129. DOI: 10.1016/S0968-090X(96)00006-X.

Yu, Y. J., & Cho, M.-G. (2008). A short-term prediction model for forecasting traffic information using bayesian network. In 3rd international conference on convergence and hybrid information technology, ICC 2008, november 11-13, 2008 (Vol. 1, p. 242-247). IEEE Computer Society. DOI: 10.1109/ICCIT.2008.355.

Zhang, J., Zheng, Y., Qi, D., Li, R., Yi, X., & Li, T. (2018). Predicting citywide crowd flows using deep spatio-temporal residual networks. Artificial Intelligence, 259, 147-166. DOI: https://doi.org/10.1016/j.artint.2018.03.002.

Zhang, L., Sun, Y., & Ma, J. (2011). An adaptive Kalman filter for short-term traffic flow forecasting. In ICTE 2011 (p. 97-102). DOI: 10.1061/41184(419)17.

Zhao, X., & Gao, Z. (2005). A new car-following model: Full velocity and acceleration difference model. European Physical Journal B, 47(1), 145-150. DOI: 10.1140/epjb/e2005-00304-3.

Zhao, Z., Chen, W., Wu, X., Chen, P. C. Y., & Liu, J. (2017). LSTM network: A deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems, 11(2), 68-75. DOI: 10.1049/iet-its.2016.0208.

Zhou, T., Jiang, D., Lin, Z., Han, G., Xu, X., & Qin, J. (2019). Hybrid dual Kalman filtering model for short-term traffic flow forecasting. IET Intelligent Transport Systems, 13(6), 1023-1032. DOI: 10.1049/iet-its.2018.5385.