This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Hate Speech Analysis of YouTube Comments on the 2024 Indonesian Presidential Debate Using IndoBERT
Corresponding Author(s) : Agus Sasmito Aribowo
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 11, No. 3, August 2026 (Article in Progress)
Abstract
The rapid digitization of political campaigns has intensified the spread of hate speech on social media, threatening democratic discourse and social cohesion. In Indonesia, YouTube comments on the 2024 presidential election debates have emerged as a critical yet underexplored source of polarizing content. However, existing detection systems struggle with Indonesian-specific linguistic features, code-mixing, and implicit political sarcasm, while high annotation costs and class imbalance further limit the scalability of supervised approaches. To address these challenges, this study introduces a large-scale dataset of 38,742 YouTube comments collected from the five official debate stages and labeled using a cost-effective semi-supervised framework (20% expert-annotated, 80% pseudo-labeled). We systematically evaluate four classification models —IndoBERT, mBERT, SVM, and Random Forest—under identical experimental conditions using evaluation metrics optimized for imbalanced data. Experimental results demonstrate that IndoBERT consistently outperforms all baseline models, achieving an average accuracy of 89.7% and a macro F1-score of 0.89 across all debate stages. Notably, IndoBERT maintains high recall for the minority hate speech class (0.81–0.90), confirming its superior ability to capture localized political rhetoric and contextual nuances that multilingual and classical models frequently miss. This study contributes a publicly available Indonesian political hate speech dataset, validates a scalable semi-supervised annotation pipeline, and provides empirical evidence that domain-specific Transformer models are essential for reliable content moderation in politically charged, low-resource environments.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- V. D. Lestari, A. Kumalasari, and S. Kasiami, “Media Sosial Sebagai Alat Kampanye Pemilu 2024_ Perspektif Pengguna Tiktok,” Jurnal Komunikasi Nusantara, pp. 30–37, 2024.
- L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 9, no. 3, pp. 746–757, 2023. https://doi.org/10.26555/jiteki.v9i3.26490
- V. A. Tricahyo and S. M. Isa, “Classification of indonesian presidential campaign on twitter using word2vec,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 4, pp. 5501–5508, 2020. https://doi.org/10.30534/ijatcse/2020/193942020
- C. D. Wulandari et al., “Fenomena Buzzer Di Media Sosial Jelang Pemilu 2024 Dalam Perspektif Komunikasi Politik,” Avant Garde, vol. 11, no. 01, pp. 134–146, 2024. https://doi.org/10.36080/ag.v11i1.2380
- P. N. M. Densa and G. K. Assidik, “Sentimen Ujaran Kebencian Pasca Pemilu 2024 di Media Sosial,” Dinamika : Jurnal Bahasa, Sastra, Pembelajarannya, vol. 8, no. 2, pp. 139–152, 2025. https://doi.org/10.35194/jd.v8i2.5079
- P. Sayarizki and H. Nurrahmi, “Implementation of IndoBERT for Sentiment Analysis of Indonesian Presidential Candidates,” Indonesian Journal On Computing, vol. 9, no. August, pp. 61–72, 2024. https://doi.org/10.34818/indojc.2024.9.2.934
- R. I. Yulfa, B. H. Setiawan, G. G. Lourensius, and K. Purwandari, “Enhancing Hate Speech Detection in Social Media Using IndoBERT Model : A Study of Sentiment Analysis during the 2024 Indonesia Presidential Election,” 2023. https://doi.org/10.1109/ICCA59364.2023.10401700
- F. V. P. Samosir and S. Riyaldi, “Sentiment Analysis of TikTok Comments on Indonesian Presidential Elections Using IndoBERT,” 2024. https://doi.org/10.1109/ICCIT62134.2024.10701256
- R. N. Tanaja, A. Widjaya, Johnny, A. A. S. Gunawan, and K. E. Setiawan, “Evaluating Public Opinion on the 2024 Indonesian Presidential Election Candidate : An IndoBERT Approach to Twitter Sentiment Analysis,” 2024. https://doi.org/10.1109/ICSCC62041.2024.10690796
- A. Jazuli and R. Kusumaningrum, “Aspect-based sentiment analysis on student reviews using the Indo-Bert base model .,” in ICENIS 2023, 2023, vol. 04, pp. 1–10. https://doi.org/10.1051/e3sconf/202344802004
- Enrico Fernandez, Anderies, M. G. Winata, F. H. Fasya, and A. A. S. Gunawan, “Improving IndoBERT for Sentiment Analysis on Indonesian Stock Trader Slang Language,” 2022. https://doi.org/10.1109/IoTaIS56727.2022.9975975
- M. A. K. Fata, S. Sumpeno, A. D. Wibawa, and D. A. Feryando, “Evaluating the Sentiment Analysis from Auto-Generated Summary Text Using IndoBERT Fine-Tuning Model in Indonesian News Text,” 2023. https://doi.org/10.1109/CICN59264.2023.10402345
- R. Husaini, N. H. Cahyana, W. Wisnalmawati, T. Mardiana, and Y. Fauziah, “Enhancing Sentiment and Emotion Classification with LSTM-Based Semi-Supervised Learning,” Compiler, vol. 14, no. 1, p. 1, 2025. https://doi.org/10.28989/compiler.v14i1.2965
- S. Fitrianie and L. J. M. Rothkrantz, “Constructing Knowledge for Automated Text-Based Emotion Expressions,” in International Conference on Computer Systems and Technologies - CompSysTech’06, 2006, pp. 1–6.
- M. I. Prasetiyowati, N. U. Maulidevi, and K. Surendro, “Feature selection to increase the random forest method performance on high dimensional data,” International Journal of Advances in Intelligent Informatics, vol. 6, no. 3, pp. 303–312, 2020. https://doi.org/10.26555/ijain.v6i3.471
- D. Y. Choi and B. C. Song, “Semi-Supervised Learning for Continuous Emotion Recognition Based on Metric Learning,” IEEE Access, vol. 8, pp. 113443–113455, 2020. https://doi.org/10.1109/ACCESS.2020.3003125
- Y. A. Singgalen, “Performance Analysis of IndoBERT for Sentiment Classification in Indonesian Hotel Review Data,” Journal of Information System Research, vol. 6, no. 2, pp. 978–988, 2025. https://doi.org/10.47065/josh.v6i2.6505
- N. H. Cahyana, S. Saifullah, Y. Fauziah, A. S. Aribowo, and R. Drezewski, “Semi-supervised Text Annotation for Hate Speech Detection using K-Nearest Neighbors and Term Frequency-Inverse Document Frequency,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 10, pp. 147–151, 2022. https://doi.org/10.14569/ijacsa.2022.0131020
- A. S. Aribowo, N. H. Cahyana, and Y. Fauziah, “Enhancing Semi-Supervised Sentiment Analysis Through Hyperparameter Tuning Within Iterations: A Comparative Study Using Grid Search and Random Search,” in Proceedings of the 2023 1st International Conference on Advanced Informatics and Intelligent Information Systems (ICAI3S 2023), 2024, no. Icai3s, pp. 248–260. https://doi.org/10.2991/978-94-6463-366-5_23
- S. Khomsah and A. S. Aribowo, “Model text-preprocessing komentar Youtube dalam bahasa Indonesia,” Rekayasa Sistem dan Teknologi Informasi, RESTI, vol. 4, no. 4, pp. 648–654, 2020. https://doi.org/10.13140/RG.2.2.32319.74403
- A. S. Aribowo, H. Basiron, N. S. Herman, and S. Khomsah, “An Evaluation of Preprocessing Steps and Tree-based Ensemble Machine Learning for Analysing Sentiment on Indonesian YouTube Comments,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 5, pp. 7078–7086, 2020. https://doi.org/10.30534/ijatcse/2020/29952020
- J. L. Cruz Paulino, L. C. Antoja Almirol, J. M. Cruz Favila, K. A. G. Loria Aquino, A. Hernandez De La Cruz, and R. E. Roxas, “Multilingual Sentiment Analysis on Short Text Document Using Semi-Supervised Machine Learning,” ACM International Conference Proceeding Series, pp. 164–170, 2021. https://doi.org/10.1145/3485768.3485775
- A. S. Aribowo, H. Basiron, and N. F. A. Yusof, “Semi-supervised learning for sentiment classification with ensemble multi-classifier approach,” International Journal of Advances in Intelligent Informatics, vol. 8, no. 3, pp. 349–361, 2022. https://doi.org/10.26555/ijain.v8i3.929
- S. Saifullah, R. Dreżewski, F. A. Dwiyanto, A. S. Aribowo, and Y. Fauziah, “Sentiment Analysis Using Machine Learning Approach Based on Feature Extraction for Anxiety Detection,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14074 LNCS, no. July, pp. 365–372, 2023. https://doi.org/10.1007/978-3-031-36021-3_38
- S. Khomsah, A. F. Hidayatullah, and A. S. Aribowo, “Comparison of the Effects of Feature Selection and Tree-Based Ensemble Machine Learning for Sentiment Analysis on Indonesian YouTube Comments,” in Lecture Notes in Electrical Engineering, vol. 746 LNEE, 2021, pp. 269–279. https://doi.org/10.1007/978-981-33-6926-9_15
- A. S. Aribowo, S. Khomsah, and S. Saifullah, “Semi-Supervised Sentiment Classification Using Self-Learning and Enhanced,” Infotel, vol. 17, no. 3, pp. 472–489, 2025. https://doi.org/10.20895/INFOTEL.v17i3.1344
- Y. A. Singgalen, “IndoBERT-Based Sentiment Analysis for Understanding Hotel Guests ’ Preferences,” Journal of Computer System and Informatics, vol. 6, no. 2, pp. 508–520, 2025. https://doi.org/10.47065/josyc.v6i2.6864
References
V. D. Lestari, A. Kumalasari, and S. Kasiami, “Media Sosial Sebagai Alat Kampanye Pemilu 2024_ Perspektif Pengguna Tiktok,” Jurnal Komunikasi Nusantara, pp. 30–37, 2024.
L. Geni, E. Yulianti, and D. I. Sensuse, “Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models,” Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 9, no. 3, pp. 746–757, 2023. https://doi.org/10.26555/jiteki.v9i3.26490
V. A. Tricahyo and S. M. Isa, “Classification of indonesian presidential campaign on twitter using word2vec,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 4, pp. 5501–5508, 2020. https://doi.org/10.30534/ijatcse/2020/193942020
C. D. Wulandari et al., “Fenomena Buzzer Di Media Sosial Jelang Pemilu 2024 Dalam Perspektif Komunikasi Politik,” Avant Garde, vol. 11, no. 01, pp. 134–146, 2024. https://doi.org/10.36080/ag.v11i1.2380
P. N. M. Densa and G. K. Assidik, “Sentimen Ujaran Kebencian Pasca Pemilu 2024 di Media Sosial,” Dinamika : Jurnal Bahasa, Sastra, Pembelajarannya, vol. 8, no. 2, pp. 139–152, 2025. https://doi.org/10.35194/jd.v8i2.5079
P. Sayarizki and H. Nurrahmi, “Implementation of IndoBERT for Sentiment Analysis of Indonesian Presidential Candidates,” Indonesian Journal On Computing, vol. 9, no. August, pp. 61–72, 2024. https://doi.org/10.34818/indojc.2024.9.2.934
R. I. Yulfa, B. H. Setiawan, G. G. Lourensius, and K. Purwandari, “Enhancing Hate Speech Detection in Social Media Using IndoBERT Model : A Study of Sentiment Analysis during the 2024 Indonesia Presidential Election,” 2023. https://doi.org/10.1109/ICCA59364.2023.10401700
F. V. P. Samosir and S. Riyaldi, “Sentiment Analysis of TikTok Comments on Indonesian Presidential Elections Using IndoBERT,” 2024. https://doi.org/10.1109/ICCIT62134.2024.10701256
R. N. Tanaja, A. Widjaya, Johnny, A. A. S. Gunawan, and K. E. Setiawan, “Evaluating Public Opinion on the 2024 Indonesian Presidential Election Candidate : An IndoBERT Approach to Twitter Sentiment Analysis,” 2024. https://doi.org/10.1109/ICSCC62041.2024.10690796
A. Jazuli and R. Kusumaningrum, “Aspect-based sentiment analysis on student reviews using the Indo-Bert base model .,” in ICENIS 2023, 2023, vol. 04, pp. 1–10. https://doi.org/10.1051/e3sconf/202344802004
Enrico Fernandez, Anderies, M. G. Winata, F. H. Fasya, and A. A. S. Gunawan, “Improving IndoBERT for Sentiment Analysis on Indonesian Stock Trader Slang Language,” 2022. https://doi.org/10.1109/IoTaIS56727.2022.9975975
M. A. K. Fata, S. Sumpeno, A. D. Wibawa, and D. A. Feryando, “Evaluating the Sentiment Analysis from Auto-Generated Summary Text Using IndoBERT Fine-Tuning Model in Indonesian News Text,” 2023. https://doi.org/10.1109/CICN59264.2023.10402345
R. Husaini, N. H. Cahyana, W. Wisnalmawati, T. Mardiana, and Y. Fauziah, “Enhancing Sentiment and Emotion Classification with LSTM-Based Semi-Supervised Learning,” Compiler, vol. 14, no. 1, p. 1, 2025. https://doi.org/10.28989/compiler.v14i1.2965
S. Fitrianie and L. J. M. Rothkrantz, “Constructing Knowledge for Automated Text-Based Emotion Expressions,” in International Conference on Computer Systems and Technologies - CompSysTech’06, 2006, pp. 1–6.
M. I. Prasetiyowati, N. U. Maulidevi, and K. Surendro, “Feature selection to increase the random forest method performance on high dimensional data,” International Journal of Advances in Intelligent Informatics, vol. 6, no. 3, pp. 303–312, 2020. https://doi.org/10.26555/ijain.v6i3.471
D. Y. Choi and B. C. Song, “Semi-Supervised Learning for Continuous Emotion Recognition Based on Metric Learning,” IEEE Access, vol. 8, pp. 113443–113455, 2020. https://doi.org/10.1109/ACCESS.2020.3003125
Y. A. Singgalen, “Performance Analysis of IndoBERT for Sentiment Classification in Indonesian Hotel Review Data,” Journal of Information System Research, vol. 6, no. 2, pp. 978–988, 2025. https://doi.org/10.47065/josh.v6i2.6505
N. H. Cahyana, S. Saifullah, Y. Fauziah, A. S. Aribowo, and R. Drezewski, “Semi-supervised Text Annotation for Hate Speech Detection using K-Nearest Neighbors and Term Frequency-Inverse Document Frequency,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 10, pp. 147–151, 2022. https://doi.org/10.14569/ijacsa.2022.0131020
A. S. Aribowo, N. H. Cahyana, and Y. Fauziah, “Enhancing Semi-Supervised Sentiment Analysis Through Hyperparameter Tuning Within Iterations: A Comparative Study Using Grid Search and Random Search,” in Proceedings of the 2023 1st International Conference on Advanced Informatics and Intelligent Information Systems (ICAI3S 2023), 2024, no. Icai3s, pp. 248–260. https://doi.org/10.2991/978-94-6463-366-5_23
S. Khomsah and A. S. Aribowo, “Model text-preprocessing komentar Youtube dalam bahasa Indonesia,” Rekayasa Sistem dan Teknologi Informasi, RESTI, vol. 4, no. 4, pp. 648–654, 2020. https://doi.org/10.13140/RG.2.2.32319.74403
A. S. Aribowo, H. Basiron, N. S. Herman, and S. Khomsah, “An Evaluation of Preprocessing Steps and Tree-based Ensemble Machine Learning for Analysing Sentiment on Indonesian YouTube Comments,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 5, pp. 7078–7086, 2020. https://doi.org/10.30534/ijatcse/2020/29952020
J. L. Cruz Paulino, L. C. Antoja Almirol, J. M. Cruz Favila, K. A. G. Loria Aquino, A. Hernandez De La Cruz, and R. E. Roxas, “Multilingual Sentiment Analysis on Short Text Document Using Semi-Supervised Machine Learning,” ACM International Conference Proceeding Series, pp. 164–170, 2021. https://doi.org/10.1145/3485768.3485775
A. S. Aribowo, H. Basiron, and N. F. A. Yusof, “Semi-supervised learning for sentiment classification with ensemble multi-classifier approach,” International Journal of Advances in Intelligent Informatics, vol. 8, no. 3, pp. 349–361, 2022. https://doi.org/10.26555/ijain.v8i3.929
S. Saifullah, R. Dreżewski, F. A. Dwiyanto, A. S. Aribowo, and Y. Fauziah, “Sentiment Analysis Using Machine Learning Approach Based on Feature Extraction for Anxiety Detection,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14074 LNCS, no. July, pp. 365–372, 2023. https://doi.org/10.1007/978-3-031-36021-3_38
S. Khomsah, A. F. Hidayatullah, and A. S. Aribowo, “Comparison of the Effects of Feature Selection and Tree-Based Ensemble Machine Learning for Sentiment Analysis on Indonesian YouTube Comments,” in Lecture Notes in Electrical Engineering, vol. 746 LNEE, 2021, pp. 269–279. https://doi.org/10.1007/978-981-33-6926-9_15
A. S. Aribowo, S. Khomsah, and S. Saifullah, “Semi-Supervised Sentiment Classification Using Self-Learning and Enhanced,” Infotel, vol. 17, no. 3, pp. 472–489, 2025. https://doi.org/10.20895/INFOTEL.v17i3.1344
Y. A. Singgalen, “IndoBERT-Based Sentiment Analysis for Understanding Hotel Guests ’ Preferences,” Journal of Computer System and Informatics, vol. 6, no. 2, pp. 508–520, 2025. https://doi.org/10.47065/josyc.v6i2.6864