This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Implementation of Particle Swarm Optimization (PSO) to Improve Neural Network Performance in Univariate Time Series Prediction
Corresponding Author(s) : Fitri Ayuning Tyas
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 6, No. 4, November 2021
Abstract
One of the oldest known predictive analytics techniques is time series prediction. The target in time series prediction is use historical data about a specific quantity to predicts value of the same quantity in the future. Multivariate time series (MTS) data has been widely used in time series prediction research because it is considered better than univariate time series (UTS) data. However, in reality MTS data sets contain various types of information which makes it difficult to extract information to predict the situation. Therefore, UTS data still has a chance to be developed because it is actually simpler than MTS data. UTS prediction treats forecasts as a single variable problem, whereas MTS may employ a large number of time-concurred series to make predictions. Neural Network (NN) model could be built to predict the target variable given the other (predictor) variables. In this study, we used Particle Swarm Optimization (PSO) algorithm to optimize performance of NN on a UTS dataset. Our proposed model is validated using x-validation and and use RMSE to measure its performance. The experimental results show that NN performance after optimization using PSO produces good results compared to classical NN performance. This is evidenced by the value of RMSE = 0.410 which is the smallest RMSE value produced. The smaller the RMSE value, the better the model performance. It can be concluded that the proposed method can improve NN performance on UTS data.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- F. Arias et al., “Auto-adaptive multilayer perceptron for univariate time series classification,” Expert Systems with Applications, Vol. 181, No. April, 2021. https://doi.org/10.1016/j.eswa.2021.115147
- A. Lahreche and B. Boucheham, “A fast and accurate similarity measure for long time series classification based on local extrema and dynamic time warping,” Expert Systems with Applications, Vol. 168, No. May 2020, P. 114374, 2021. https://doi.org/10.1016/j.eswa.2020.114374
- N. Suhermi, D. D. Prastyo, and B. Ali, “Roll motion motion prediction prediction using a hybrid deep learning and ARIMA model,” Procedia Computer Science, Vol. 144, Pp. 251–258, 2018. https://doi.org/10.1016/j.procs.2018.10.526
- C. Koutlis, S. Papadopoulos, M. Schinas, and I. Kompatsiaris, “LAVARNET : Neural network modeling of causal variable relationships for multivariate time series forecasting,” Applied Soft Computing Journal, Vol. 96, P. 106685, 2020. https://doi.org/10.1016/j.asoc.2020.106685
- J. Sun, Y. Yang, Y. Liu, C. Chen, W. Rao, and Y. Bai, “Univariate time series classification using information geometry,” Pattern Recognition, Vol. 95, Pp. 24–35, Nov. 2019. https://doi.org/10.1016/j.patcog.2019.05.040
- V. Osipov, S. Kuleshov, D. Levonevskiy, and D. Miloserdov, “Neural network forecasting of news feeds,” Expert Systems With Applications, 2020. https://doi.org/10.1016/j.eswa.2020.114521
- A. R. S. Parmezan, V. M. A. Souza, and G. E. A. P. A. Batista, “Evaluation of statistical and machine learning models for time series prediction : Identifying the state-of-the-art and the best conditions for the use of each model,” Information Sciences, Vol. 484, Pp. 302–337, 2019. https://doi.org/10.1016/j.ins.2019.01.076
- G. Papacharalampous, H. Tyralis, and D. Koutsoyiannis, “Univariate Time Series Forecasting of Temperature and Precipitation with a Focus on Machine Learning Algorithms : a Multiple-Case Study from Greece,” Water Resources Management, 2018. https://doi.org/10.1007/s11269-018-2155-6
- B. Li, J. Ding, Z. Yin, K. Li, X. Zhao, and L. Zhang, “Optimized neural network combined model based on the induced ordered weighted averaging operator for vegetable price forecasting,” Expert Systems with Applications, Vol. 168, P. 114232, Apr. 2021. https://doi.org/10.1016/j.eswa.2020.114232aie
- H. Quan, D. Srinivasan, and A. Khosravi, “Particle swarm optimization for construction of neural network-based prediction intervals,” Neurocomputing, Vol. 127, Pp. 172–180, 2014. https://doi.org/10.1016/j.neucom.2013.08.020
- B. Jamali, M. Rasekh, F. Jamadi, R. Gandomkar, and F. Makiabadi, “Using PSO-GA algorithm for training artificial neural network to forecast solar space heating system parameters,” Applied Thermal Engineering, 2019. https://doi.org/10.1016/j.applthermaleng.2018.10.070
- N. Noviandi and A. Ilham, “Optimization fuzzy inference system based particle swarm optimization for onset prediction of the rainy season,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 4, No. 1, Pp. 61–70, 2020. https://doi.org/https://doi.org/10.22219/kinetik.v5i1.985
- X. Liu, Y. Gu, S. He, Z. Xu, and Z. Zhang, “A robust reliability prediction method using Weighted Least Square Support Vector Machine equipped with Chaos Modified Particle Swarm Optimization and Online Correcting Strategy,” Applied Soft Computing Journal, Vol. 85, P. 105873, 2019. https://doi.org/10.1016/j.asoc.2019.105873
- B. Bai, J. Zhang, X. Wu, G. wei Zhu, and X. Li, “Reliability prediction-based improved dynamic weight particle swarm optimization and back propagation neural network in engineering systems,” Expert Systems with Applications, Vol. 177, No. November 2020, P. 114952, 2021. https://doi.org/10.1016/j.eswa.2021.114952
- P. Singh, S. Chaudhury, and B. K. Panigrahi, “Hybrid MPSO-CNN: Multi-level Particle Swarm optimized hyperparameters of Convolutional Neural Network,” Swarm and Evolutionary Computation, Vol. 63, No. February, P. 100863, 2021. https://doi.org/10.1016/j.swevo.2021.100863
- A. P. Piotrowski, J. J. Napiorkowski, and A. E. Piotrowska, “Population size in Particle Swarm Optimization,” Swarm and Evolutionary Computation, Vol. 58, No. May, P. 100718, 2020. https://doi.org/10.1016/j.swevo.2020.100718
- R. S. Wahono, N. S. Herman, and S. Ahmad, “Neural network parameter optimization based on genetic algorithm for software defect prediction,” Advanced Science Letters, Vol. 20, No. 10–12, Pp. 1951–1955, 2014. https://doi.org/10.1166/asl.2014.5641
- Z. Alameer, M. Abd, A. A. Ewees, H. Ye, and Z. Jianhua, “Forecasting gold price fluctuations using improved multilayer perceptron neural network and whale optimization algorithm,” Resources Policy, Vol. 61, Pp. 250–260, 2019. https://doi.org/10.1016/j.resourpol.2019.02.014
- M. Chayama and Y. Hirata, “When univariate model-free time series prediction is better than multivariate,” Physics Letters A, Pp. 1–7, 2016. https://doi.org/10.1016/j.physleta.2016.05.027
- F. fMohammad and R. Gupta, “ARIMA and NAR based prediction model for time series analysis of COVID-19 cases in India,” Journal of Safety Science and Resilience, Vol. 1, No. June, Pp. 12–18, 2020. https://doi.org/10.1016/j.jnlssr.2020.06.007
References
F. Arias et al., “Auto-adaptive multilayer perceptron for univariate time series classification,” Expert Systems with Applications, Vol. 181, No. April, 2021. https://doi.org/10.1016/j.eswa.2021.115147
A. Lahreche and B. Boucheham, “A fast and accurate similarity measure for long time series classification based on local extrema and dynamic time warping,” Expert Systems with Applications, Vol. 168, No. May 2020, P. 114374, 2021. https://doi.org/10.1016/j.eswa.2020.114374
N. Suhermi, D. D. Prastyo, and B. Ali, “Roll motion motion prediction prediction using a hybrid deep learning and ARIMA model,” Procedia Computer Science, Vol. 144, Pp. 251–258, 2018. https://doi.org/10.1016/j.procs.2018.10.526
C. Koutlis, S. Papadopoulos, M. Schinas, and I. Kompatsiaris, “LAVARNET : Neural network modeling of causal variable relationships for multivariate time series forecasting,” Applied Soft Computing Journal, Vol. 96, P. 106685, 2020. https://doi.org/10.1016/j.asoc.2020.106685
J. Sun, Y. Yang, Y. Liu, C. Chen, W. Rao, and Y. Bai, “Univariate time series classification using information geometry,” Pattern Recognition, Vol. 95, Pp. 24–35, Nov. 2019. https://doi.org/10.1016/j.patcog.2019.05.040
V. Osipov, S. Kuleshov, D. Levonevskiy, and D. Miloserdov, “Neural network forecasting of news feeds,” Expert Systems With Applications, 2020. https://doi.org/10.1016/j.eswa.2020.114521
A. R. S. Parmezan, V. M. A. Souza, and G. E. A. P. A. Batista, “Evaluation of statistical and machine learning models for time series prediction : Identifying the state-of-the-art and the best conditions for the use of each model,” Information Sciences, Vol. 484, Pp. 302–337, 2019. https://doi.org/10.1016/j.ins.2019.01.076
G. Papacharalampous, H. Tyralis, and D. Koutsoyiannis, “Univariate Time Series Forecasting of Temperature and Precipitation with a Focus on Machine Learning Algorithms : a Multiple-Case Study from Greece,” Water Resources Management, 2018. https://doi.org/10.1007/s11269-018-2155-6
B. Li, J. Ding, Z. Yin, K. Li, X. Zhao, and L. Zhang, “Optimized neural network combined model based on the induced ordered weighted averaging operator for vegetable price forecasting,” Expert Systems with Applications, Vol. 168, P. 114232, Apr. 2021. https://doi.org/10.1016/j.eswa.2020.114232aie
H. Quan, D. Srinivasan, and A. Khosravi, “Particle swarm optimization for construction of neural network-based prediction intervals,” Neurocomputing, Vol. 127, Pp. 172–180, 2014. https://doi.org/10.1016/j.neucom.2013.08.020
B. Jamali, M. Rasekh, F. Jamadi, R. Gandomkar, and F. Makiabadi, “Using PSO-GA algorithm for training artificial neural network to forecast solar space heating system parameters,” Applied Thermal Engineering, 2019. https://doi.org/10.1016/j.applthermaleng.2018.10.070
N. Noviandi and A. Ilham, “Optimization fuzzy inference system based particle swarm optimization for onset prediction of the rainy season,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 4, No. 1, Pp. 61–70, 2020. https://doi.org/https://doi.org/10.22219/kinetik.v5i1.985
X. Liu, Y. Gu, S. He, Z. Xu, and Z. Zhang, “A robust reliability prediction method using Weighted Least Square Support Vector Machine equipped with Chaos Modified Particle Swarm Optimization and Online Correcting Strategy,” Applied Soft Computing Journal, Vol. 85, P. 105873, 2019. https://doi.org/10.1016/j.asoc.2019.105873
B. Bai, J. Zhang, X. Wu, G. wei Zhu, and X. Li, “Reliability prediction-based improved dynamic weight particle swarm optimization and back propagation neural network in engineering systems,” Expert Systems with Applications, Vol. 177, No. November 2020, P. 114952, 2021. https://doi.org/10.1016/j.eswa.2021.114952
P. Singh, S. Chaudhury, and B. K. Panigrahi, “Hybrid MPSO-CNN: Multi-level Particle Swarm optimized hyperparameters of Convolutional Neural Network,” Swarm and Evolutionary Computation, Vol. 63, No. February, P. 100863, 2021. https://doi.org/10.1016/j.swevo.2021.100863
A. P. Piotrowski, J. J. Napiorkowski, and A. E. Piotrowska, “Population size in Particle Swarm Optimization,” Swarm and Evolutionary Computation, Vol. 58, No. May, P. 100718, 2020. https://doi.org/10.1016/j.swevo.2020.100718
R. S. Wahono, N. S. Herman, and S. Ahmad, “Neural network parameter optimization based on genetic algorithm for software defect prediction,” Advanced Science Letters, Vol. 20, No. 10–12, Pp. 1951–1955, 2014. https://doi.org/10.1166/asl.2014.5641
Z. Alameer, M. Abd, A. A. Ewees, H. Ye, and Z. Jianhua, “Forecasting gold price fluctuations using improved multilayer perceptron neural network and whale optimization algorithm,” Resources Policy, Vol. 61, Pp. 250–260, 2019. https://doi.org/10.1016/j.resourpol.2019.02.014
M. Chayama and Y. Hirata, “When univariate model-free time series prediction is better than multivariate,” Physics Letters A, Pp. 1–7, 2016. https://doi.org/10.1016/j.physleta.2016.05.027
F. fMohammad and R. Gupta, “ARIMA and NAR based prediction model for time series analysis of COVID-19 cases in India,” Journal of Safety Science and Resilience, Vol. 1, No. June, Pp. 12–18, 2020. https://doi.org/10.1016/j.jnlssr.2020.06.007