Issue

Vol. 10, No. 3, August 2025

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Classification of Livin' by Mandiri Customer Satisfaction Using MLP with BM25 and TF-IDF Feature Weighting

https://doi.org/10.22219/kinetik.v10i3.2248
Aina Mardiah
Universitas Negeri Makassar
Salsa Dillah
Universitas Negeri Makassar
Dewi Fatmarani Surianto
Universitas Negeri Makassar
Nur Fadilah
Universitas Megarezky Makassar
Satria Gunawan Zain
Universitas Negeri Makassar

Corresponding Author(s) : Dewi Fatmarani Surianto

dewifatmaranis@unm.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 10, No. 3, August 2025

Abstract

The increasing use of mobile banking applications such as Livin' by Mandiri requires an analysis of customer satisfaction based on user reviews. This study classifies customer satisfaction levels using the Multi-Layer Perceptron (MLP) algorithm with two feature extraction methods, namely BM25 and TF-IDF. A total of 1,143 reviews were collected from the Google Play Store and App Store. Three test scenarios were applied: (1) comparison of feature extraction methods, (2) application of Synthetic Minority Over-Sampling Technique (SMOTE), and (3) application of Synonym Replacement-based Easy Data Augmentation (EDA) technique. The evaluation results show that the combination of BM25 and data augmentation produces the highest performance, with 97% accuracy and 98% precision, recall, and F1-score, respectively. BM25 proved to be more effective in understanding the context of reviews, while data augmentation improved the quality of representation, especially for minority classes such as neutral sentiment. These findings make a significant contribution to the improvement of Livin' by Mandiri digital services and serve as a reference for the development of review-based satisfaction classification systems in the digital banking sector.

Keywords

BM25 TF-IDF Customer Satisfaction Multil-layer Perception Livin' Mandiri
Mardiah, A., Dillah, S. ., Surianto, D. F. ., Fadilah, N., & Zain, S. G. (2025). Classification of Livin’ by Mandiri Customer Satisfaction Using MLP with BM25 and TF-IDF Feature Weighting . Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 10(3), 355-366. https://doi.org/10.22219/kinetik.v10i3.2248
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. S. I. Murpratiwi, S. E. Anjarwani, I. G. P. S. Wijaya, and A. Aranta, “Sosialisasi Internet Sehat Dan Pelatihan Penggunaan Internet Sebagai Media Penunjang Pembelajaran Di SD Negeri Anggaraksa,” J. Begawe Teknol. Inf., vol. 4, no. 2, pp. 256–263, 2023. https://doi.org/10.29303/jbegati.v4i2.1111
  2. D. D. Audiansyah, D. E. Ratnawati, and B. T. Hanggara, “Analisis Sentimen Aplikasi MyXL menggunakan Metode Support Vector Machine berdasarkan Ulasan Pengguna di Google Play Store,” vol. 6, no. 8, pp. 3987–3994, 2022.
  3. A. Ahmad, W. Gata, and S. Panggabean, “Sentimen Analisis dengan Long Short-Term Memory dan Synthetic Minority Over Sampling Technic Pada Aplikasi Digital Perbankan,” J. Teknol. Inf. dan Komun., vol. 8, no. 4, pp. 973–984, 2024.
  4. T. Wahudi and Z. Hutabarat, “Faktor-Faktor Yang Mempengaruhi Niat Penggunaan Digital Banking: Livin ’ By Mandiri,” J. Tek. Inform. dan Sist. Inf., vol. 10, no. 1, pp. 509–525, 2023.
  5. S. H. Alviyanti, A. Purwandira, I. Febiyanti, E. Daniati, and A. Ristyawan, “Klasifikasi Sentimen Pengguna Aplikasi Livin By Mandiri Pada Playstore Menggunakan Algoritma Naive Bayes,” Agustus, vol. 8, pp. 2549–7952, 2024.
  6. S. L. Ranataru and N. Trianasari, “Analisis Sentimen Media Sosial Terhadap Aplikasi Perbankan untuk Mengetahui Kepuasan Pengguna Aplikasi: Studi Kasus pada Livin by Mandiri dan BCA Mobile,” Al-Kharaj J. Ekon. , Keuang. Bisnis Syariah, vol. 6, pp. 6818–6838, 2024. https://doi.org/10.47467/alkharaj.v6i9.3805
  7. I. D. Onantya and P. P. Adikara, “Analisis Sentimen Pada Ulasan Aplikasi BCA Mobile Menggunakan BM25 Dan Improved K-Nearest Neighbor,” J-Ptiik.Ub.Ac.Id, vol. 3, no. 3, pp. 2575–2580, 2019.
  8. M. Z. Hariansyah and Siswanto, “Implementasi Metode Multinomial Naive Bayes pada Analisis Sentimen Terhadap Layanan Aplikasi Livin by Mandiri,” Semin. Nas. Mhs. Fak. Teknol. Inf., vol. 1, no. 1, pp. 517–524, 2022.
  9. N. Nurfadila, M. Ariyanti, and N. Trianasari, “Analisis Kualitas Layanan Mobile Banking New Livin’ By Mandiri Menggunakan Sentiment Analysis,” JIBR J. Indones. Bus. Res., vol. 1, no. 1, pp. 77–82, 2023.
  10. C. A. Qurniaty and K. Kusnawi, “Ekspresi Emosi Berdasarkan Suara Menggunakan Algortima Multi Layer Perceptron dan Support Vector Machine,” Indones. J. Comput. Sci., vol. 12, no. 6, pp. 4014–4025, 2023. https://doi.org/10.33022/ijcs.v12i6.3567
  11. G. G. Warow and H. Pandia, “Analisis Sentimen Aplikasi Dana Menggunakan Naïve Bayes Classifier dan Support Vector Machine,” Jutisi J. Ilm. Tek. Inform. dan Sist. Inf., vol. 13, no. 1, p. 609, 2024. https://doi.org/10.35889/jutisi.v13i1.1893
  12. N. Meilani, Mhd. Furqan and Suhardi, “Analisis sentimen pengguna aplikasi BSI mobile akibat ransomware menggunakan algoritma support vector machine,” INFOTECH J. Inform. Teknol., vol. 5, no. 1, pp. 42–51, 2024. https://doi.org/10.37373/infotech.v5i1.1102
  13. R. A. Husen, R. Astuti, L. Marlia, R. Rahmaddeni, and L. Efrizoni, “Analisis Sentimen Opini Publik pada Twitter Terhadap Bank BSI Menggunakan Algoritma Machine Learning,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 3, no. 2, pp. 211–218, 2023. https://doi.org/10.57152/malcom.v3i2.901
  14. F. H. Pasaribu, N. Khairina, D. Noviandri, S. Susilawati, and R. Syah, “Analysis of The Multilayer Perceptron Algorithm on Twitter User’s Sentiment Towards The COVID-19 Vaccine,” J. Informatics Telecommun. Eng., vol. 7, no. 1, pp. 155–163, 2023. https://doi.org/10.31289/jite.v7i1.9664
  15. L. Efrizoni, S. Defit, M. Tajuddin, and A. Anggrawan, “Komparasi Ekstraksi Fitur dalam Klasifikasi Teks Multilabel Menggunakan Algoritma Machine Learning,” MATRIK J. Manajemen, Tek. Inform. dan Rekayasa Komput., vol. 21, no. 3, pp. 653–666, 2022. https://doi.org/10.30812/matrik.v21i3.1851
  16. I. Daniel, A. Fahmi Limas Ptr, and A. Ichsan, “Klasifikasi Risiko Penyakit Serangan Jantung Dengan Multi-Layer Perceptron,” Data Sci. Indones., vol. 14, no. 1, pp. 57–64, 2024. https://doi.org/10.47709/dsi.v4i1.4667
  17. A. Purnamawati, M. N. Winarto, and M. Mailasari, “Analisis Sentimen Aplikasi TikTok menggunakan Metode BM25 dan Improved K-NN Fitur Chi-Square,” J. Komtika (Komputasi dan Inform., vol. 7, no. 1, pp. 97–105, 2023. https://doi.org/10.31603/komtika.v7i1.8938
  18. R. Maulana, A. Voutama, and T. Ridwan, “Analisis Sentimen Ulasan Aplikasi MyPertamina pada Google Play Store menggunakan Algoritma NBC,” J. Teknol. Terpadu, vol. 9, no. 1, pp. 42–48, 2023. https://www.doi.org/10.54914/jtt.v9i1.609
  19. D. Duei Putri, G. F. Nama, and W. E. Sulistiono, “Analisis Sentimen Kinerja Dewan Perwakilan Rakyat (DPR) Pada Twitter Menggunakan Metode Naive Bayes Classifier,” J. Inform. dan Tek. Elektro Terap., vol. 10, no. 1, pp. 34–40, 2022. https://doi.org/10.23960/jitet.v10i1.2262
  20. N. Agustina, D. H. Citra, W. Purnama, C. Nisa, and A. R. Kurnia, “Implementasi Algoritma Naive Bayes untuk Analisis Sentimen Ulasan Shopee pada Google Play Store,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 2, no. 1, pp. 47–54, 2022. https://doi.org/10.57152/malcom.v2i1.195
  21. R. A. Nurdian, Mujib Ridwan, and Ahmad Yusuf, “Komparasi Metode SMOTE dan ADASYN dalam Meningkatkan Performa Klasifikasi Herregistrasi Mahasiswa Baru,” J. Tek. Inform. dan Sist. Inf., vol. 8, no. 1, pp. 24–32, 2022. https://doi.org/10.28932/jutisi.v8i1.4004
  22. S. Sasmita, R. N. Jariah S.Intam, D. F. Surianto, and M. F. B, “Analisis Sentimen Terhadap Kontroversi Putusan MK Mengenai Usia Capres-Cawapres Menggunakan Multi-Layer Perceptron Dengan Teknik SMOTE,” Fakt. Exacta, vol. 17, no. 2, p. 188, 2024. https://doi.org/10.30998/faktorexacta.v17i2.22442
  23. W. Wahyudi, R. Kurniawan, and Y. Arie Wijaya, “Analisis Sentimen Pengguna Terhadap Aplikasi Blu Bca Di Playstore Mengunakan Algoritma Naïve Bayes,” JATI (Jurnal Mhs. Tek. Inform., vol. 8, no. 3, pp. 2511–2517, 2024. https://doi.org/10.36040/jati.v8i3.9216
  24. R. N. Harahap and K. Muslim, “Peningkatan Akurasi pada Prediksi Kepribadian Mbti Pengguna Twitter Menggunakan Augmentasi Data,” J. Teknol. Inf. dan Ilmu Komput., vol. 7, no. 4, p. 815, 2020. https://doi.org/10.25126/jtiik.2020743622
  25. A. Nur Azizah, M. Falach Asy’ari, I. Wisma Dwi Prastya, and D. Purwitasari, “Easy Data Augmentation untuk Data yang Imbalance pada Konsultasi Kesehatan Daring,” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 5, pp. 1095–1104, 2023. https://doi.org/10.25126/jtiik.20231057082
  26. I. Athiyyah Rahma and L. Hulliyyatus Suadaa, “Penerapan Text Augmentation Untuk Mengatasi Data Yang Tidak Seimbang Pada Klasifikasi Teks Berbahasa Indonesia Studi Kasus: Deteksi Judul Clickbait Dan Komentar Hate Speech Pada Berita Online,” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 6, pp. 1329–1340, 2023. https://doi.org/10.25126/jtiik.2023107325
  27. A. I. Tanggraeni and M. N. N. Sitokdana, “Analisis Sentimen Aplikasi E-Government pada Google Play Menggunakan Algoritma Naïve Bayes,” JATISI (Jurnal Tek. Inform. dan Sist. Informasi), vol. 9, no. 2, pp. 785–795, 2022. https://doi.org/10.35957/jatisi.v9i2.1835
Read More
Author biographies is not available.
Download this PDF file
PDF
Statistic
Read Counter : 0 Download : 0

Downloads

Download data is not yet available.