Issue

Vol. 9, No. 4, November 2024

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Aspect-level Sentiment Analysis on GoPay App Reviews Using Multilayer Perceptron and Word Embeddings

https://doi.org/10.22219/kinetik.v9i4.2041
Henzi Juandri
Telkom University
Hasmawati
Telkom University
Bunyamin
Telkom University

Corresponding Author(s) : Henzi Juandri

henzijuandri.work@gmail.com

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 9, No. 4, November 2024

Abstract

The increasing use of smartphone in Indonesia has encouraged the development of digital wallet applications, one of which is GoPay. Nowadays, GoPay has gained significant popularity among the public in Indonesia. Therefore, this research conducts aspect-level sentiment analysis to analyze user reviews of the GoPay application in more detail and depth. The sentiment analysis process in this study utilizes the Multilayer Perceptron (MLP) with fastText and word2vec as word embeddings. The dataset used is GoPay application reviews, which consist of 15,000 reviews collected from Google Play Store. The dataset is categorized into three main aspects: Feature and functionality, App Interface, and User Satisfaction. The stages of the research include data preparation, data preprocessing, word embeddings, model training, and model testing and evaluation. This research explores the effect of fastText and word2vec as word embeddings on model performance. Furthermore, this research examines the application of oversampling techniques, such as SMOTE and Random Oversampling. Based on the experiments conducted, utilizing fastText as word embeddings in MLP with a balanced dataset resulted the best model performance, with an F1-Score of 97%, Recall of 96%, and Precision of 95% for category classification. Then, for sentiment classification, using fastText on MLP with a balanced dataset resulted in a value of 98% for each of the F1-score, Recall, and Precision metrics. This research validates that MLP is effective for aspect-level sentiment analysis, delivering strong evaluation results.

Keywords

Gopay Sentiment Analysis Aspect Based Sentiment Analysis Aspect Level Sentiment Analysis Word Embeddings Natural Language Processing
Juandri, H., Hasmawati, & Bunyamin. (2024). Aspect-level Sentiment Analysis on GoPay App Reviews Using Multilayer Perceptron and Word Embeddings. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 9(4). https://doi.org/10.22219/kinetik.v9i4.2041
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. Statista, “1 Number of smartphone users in Indonesia from 2018 to 2028 (in millions) [Graph].
  2. Rakuten Insight, “2 Major e-payment services used among respondents in Indonesia as of October 2022 [Graph].”
  3. GoPay, “3 GoPay: Transfer & Payment," Google Play Store.”
  4. K. L. Tan, C. P. Lee, and K. M. Lim, “4 A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research,” Apr. 01, 2023, MDPI. https://doi.org/10.3390/app13074550
  5. N. C. Dang, M. N. Moreno-García, and F. De la Prieta, “5 Sentiment analysis based on deep learning: A comparative study,” Electronics (Switzerland), vol. 9, no. 3, Mar. 2020. https://doi.org/10.3390/electronics9030483
  6. W. Zhang, X. Li, Y. Deng, L. Bing, and W. Lam, “6 A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges,” Mar. 2022. https://doi.org/10.48550/arXiv.2203.01054
  7. S. Behl, A. Rao, S. Aggarwal, S. Chadha, and H. S. Pannu, “7 Twitter for disaster relief through sentiment analysis for COVID-19 and natural hazard crises,” International Journal of Disaster Risk Reduction, vol. 55, Mar. 2021. https://doi.org/10.1016/j.ijdrr.2021.102101
  8. A. E. O. Carosia, G. P. Coelho, and A. E. A. Silva, “8 Analyzing the Brazilian Financial Market through Portuguese Sentiment Analysis in Social Media,” Applied Artificial Intelligence, vol. 34, no. 1, pp. 1–19, Jan. 2020. https://doi.org/10.1080/08839514.2019.1673037
  9. R. P. Aluna, I. N. Yulita, and R. Sudrajat, “9 Electronic News Sentiment Analysis Application to New Normal Policy During The Covid-19 Pandemic Using Fasttext And Machine Learning,” in 2021 International Conference on Artificial Intelligence and Big Data Analytics, 2021, pp. 236–241. https://doi.org/10.1109/ICAIBDA53487.2021.9689756
  10. I. Kaibi, E. H. Nfaoui, and H. Satori, “10 A Comparative Evaluation of Word Embeddings Techniques for Twitter Sentiment Analysis,” in 2019 International Conference on Wireless Technologies, Embedded and Intelligent Systems (WITS), 2019, pp. 1–4. https://doi.org/10.1109/WITS.2019.8723864
  11. S. F. Sabbeh and H. A. Fasihuddin, “11 A Comparative Analysis of Word Embedding and Deep Learning for Arabic Sentiment Classification,” Electronics (Basel), vol. 12, no. 6, 2023. https://doi.org/10.3390/electronics12061425
  12. C. Wang, P. Nulty, and D. Lillis, “12 A Comparative Study on Word Embeddings in Deep Learning for Text Classification,” in Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval, in NLPIR ’20. New York, NY, USA: Association for Computing Machinery, 2021, pp. 37–46. https://doi.org/10.1145/3443279.3443304
  13. D. S. Asudani, N. K. Nagwani, and P. Singh, “13 Impact of word embedding models on text analytics in deep learning environment: a review,” Artif Intell Rev, vol. 56, no. 9, pp. 10345–10425, 2023. https://doi.org/10.1007/s10462-023-10419-1
  14. S. H. Janjua, G. F. Siddiqui, M. A. Sindhu, and U. Rashid, “14 Multi-level aspect based sentiment classification of Twitter data: using hybrid approach in deep learning,” PeerJ Comput Sci, vol. 7, pp. 1–25, Apr. 2021. https://doi.org/10.7717/peerj-cs.433
  15. N. Alturaief, H. Aljamaan, and M. Baslyman, “15 AWARE: Aspect-Based Sentiment Analysis Dataset of Apps Reviews for Requirements Elicitation,” in Proceedings - 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops, ASEW 2021, Institute of Electrical and Electronics Engineers Inc., 2021, pp. 211–218. https://doi.org/10.1109/ASEW52652.2021.00049
  16. S. Gunathilaka and N. De Silva, “16 Aspect-based Sentiment Analysis on Mobile Application Reviews,” in 22nd International Conference on Advances in ICT for Emerging Regions, ICTer 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 183–188. https://doi.org/10.1109/ICTer58063.2022.10024070
  17. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “17 Efficient Estimation of Word Representations in Vector Space,” Jan. 2013. https://doi.org/10.48550/arXiv.1301.3781
  18. E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov, “18 Learning Word Vectors for 157 Languages,” CoRR, vol. abs/1802.06893, 2018. https://doi.org/10.48550/arXiv.1802.06893
  19. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “19 Efficient Estimation of Word Representations in Vector Space,” Jan. 2013. https://doi.org/10.48550/arXiv.1301.3781
  20. N. Dilawar et al., “20 Understanding citizen issues through reviews: A step towards data informed planning in Smart Cities,” Applied Sciences (Switzerland), vol. 8, no. 9, Sep. 2018. https://doi.org/10.3390/app8091589
  21. A. Hafeez et al., “21 Addressing Imbalance Problem for Multi Label Classification of Scholarly Articles,” IEEE Access, vol. PP, p. 1, Jun. 2023. https://doi.org/10.1109/ACCESS.2023.3293852
  22. R. Ali, J. Hussain, and S. W. Lee, “22 Multilayer perceptron-based self-care early prediction of children with disabilities,” Digit Health, vol. 9, Jan. 2023. https://doi.org/10.1177/20552076231184054
  23. S. R. Dubey, S. K. Singh, and B. B. Chaudhuri, “23 Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark,” Sep. 2021. https://doi.org/10.48550/arXiv.2109.14545
  24. . Usha Ruby Dr.A, “24 Binary cross entropy with deep learning technique for Image classification,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 4, pp. 5393–5397, Aug. 2020. https://doi.org/10.30534/ijatcse/2020/175942020
  25. S. Chatterjee and A. Keprate, “25 Predicting Remaining Fatigue Life of Topside Piping Using Deep Learning,” in 2021 International Conference on Applied Artificial Intelligence, ICAPAI 2021, Institute of Electrical and Electronics Engineers Inc., May 2021. https://doi.org/10.1109/ICAPAI49758.2021.9462055
  26. A. I. Ramadhan and E. B. Setiawan, “26 Aspect-based Sentiment Analysis on Social Media Using Convolutional Neural Network (CNN) Method,” Building of Informatics, Technology and Science (BITS), vol. 4, no. 4, Mar. 2023. https://doi.org/10.47065/bits.v4i4.3103
  27. S. Riyanto, I. S. Sitanggang, T. Djatna, and T. D. Atikah, “27 Comparative Analysis using Various Performance Metrics in Imbalanced Data for Multi-class Text Classification.”. https://dx.doi.org/10.14569/IJACSA.2023.01406116
  28. C. Padurariu and M. E. Breaban, “28 Dealing with data imbalance in text classification,” in Procedia Computer Science, Elsevier B.V., 2019, pp. 736–745. https://doi.org/10.1016/j.procs.2019.09.229
  29. M. Hayaty, S. Muthmainah, and S. M. Ghufran, “29 Random and Synthetic Over-Sampling Approach to Resolve Data Imbalance in Classification,” International Journal of Artificial Intelligence Research, vol. 4, no. 2, p. 86, Jan. 2021. https://doi.org/10.29099/ijair.v4i2.152
  30. M. R. Ilham and A. D. Laksito, “30 Comparative Analysis of Using Word Embedding in Deep Learning for Text Classification,” Jurnal Riset Informatika, vol. 5, no. 2, pp. 195–202, Mar. 2023.
  31. P. Mojumder, M. Hasan, M. F. Hossain, and K. M. A. Hasan, “31 A study of fasttext word embedding effects in document classification in bangla language,” in Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, Springer, 2020, pp. 441–453. https://doi.org/10.1007/978-3-030-52856-0_35
  32. C. Yang, E. A. Fridgeirsson, J. A. Kors, J. M. Reps, and P. R. Rijnbeek, “32 Impact of random oversampling and random undersampling on the performance of prediction models developed using observational health data,” J Big Data, vol. 11, no. 1, Dec. 2024. https://doi.org/10.1186/s40537-023-00857-7
Read More
Author biographies is not available.
Download this PDF file
PDF
Statistic
Read Counter : 0 Download : 0

Downloads

Download data is not yet available.