Issue

Vol. 10, No. 3, August 2025

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Exploiting Vulnerabilities of Machine Learning Models on Medical Text via Generative Adversarial Attacks

https://doi.org/10.22219/kinetik.v10i3.2280
Maulana Akmal Shahib
Universitas Muhammadiyah Malang
Setio Basuki
Universitas Muhammadiyah Malang
Wardhana Aulia Arif
Wroclaw University of Science and Technology

Corresponding Author(s) : Setio Basuki

setio_basuki@umm.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 10, No. 3, August 2025

Abstract

Significant developments in artificial intelligence (AI) technology have fueled its adoption across a range of fields. The use of AI, particularly machine learning (ML), has expanded significantly in the medical field due to its high diagnostic precision. However, the AI model faces a serious challenge to handle the adversarial attacks. These attacks use perturbed data (modified data), which is unnoticeable to humans but can significantly alter prediction results. This paper uses a medical text dataset containing descriptions of patients with lung diseases classified into eight categories. This paper aims to implement the TextFooler technique to deceive predictive models on medical text against adversarial attacks. The experiment reveals that three ML models developed using popular approaches, i.e., transformer-based model based on Bidirectional Encoder Representations from Transformers (BERT), Stack Classifier that combines three traditional machine learning models, and individual traditional algorithms achieved the same classification accuracy of 99.98%.  The experiment reveals that BERT is the weakest model, with an attack success rate of 76.8%, followed by traditional machine learning methods and the stack classifier, with success rates of 28.73% and 5.21%, respectively. This implies that although BERT classification demonstrates good performance, it is highly vulnerable to adversarial attacks. Therefore, there is an urgency to develop predictive models that are robust and secure against potential attacks.

Keywords

Adversarial Attack Artificial Intelligence Medical Field Perturbed Data Textfooler
Akmal Shahib, M., Basuki, S., & Aulia Arif, W. (2025). Exploiting Vulnerabilities of Machine Learning Models on Medical Text via Generative Adversarial Attacks. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 10(3), 431-442. https://doi.org/10.22219/kinetik.v10i3.2280
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. R. Gubareva and R. Lopes, “Virtual Assistants for Learning: A Systematic Literature Review,” Oct. 2020, pp. 97–103. https://doi.org/10.5220/0009417600970103
  2. A. M. Nascimento et al., “A Systematic Literature Review About the Impact of Artificial Intelligence on Autonomous Vehicle Safety,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 12, pp. 4928–4946, 2020. https://doi.org/10.1109/TITS.2019.2949915
  3. M. Vázquez-Hernández, L. A. Morales-Rosales, I. Algredo-Badillo, S. I. Fernández-Gregorio, H. Rodr’iguez-Rangel, and M.-L. Córdoba-Tlaxcalteco, “A Survey of Adversarial Attacks: An Open Issue for Deep Learning Sentiment Analysis Models,” Applied Sciences, vol. 14, no. 11, p. 4614, 2024. https://doi.org/10.3390/app14114614
  4. M. Pejić Bach, Ž. Krstić, S. Seljan, and L. Turulja, “Text mining for big data analysis in financial sector: A literature review,” Sustainability, vol. 11, no. 5, p. 1277, 2019. https://doi.org/10.3390/su11051277
  5. M. Ahmed and M. N. Uddin, “Cyber attack detection method based on nlp and ensemble learning approach,” in 2020 23rd International Conference on Computer and Information Technology (ICCIT), 2020, pp. 1–6. https://doi.org/10.1109/ICCIT51783.2020.9392682
  6. T. Arjunan, “Detecting Anomalies and Intrusions in Unstructured Cybersecurity Data Using Natural Language Processing,” Int J Res Appl Sci Eng Technol, vol. 12, no. 9, pp. 10–22214, 2024. https://doi.org/10.22214/ijraset.2024.58497
  7. S. Huang, J. Yang, S. Fong, and Q. Zhao, “Artificial intelligence in the diagnosis of covid-19: Challenges and perspectives,” 2021, Ivyspring International Publisher. https://doi.org/10.7150/ijbs.58855
  8. L. Q. Zhou et al., “Artificial intelligence in medical imaging of the liver,” World J Gastroenterol, vol. 25, no. 6, pp. 672–682, 2019. https://doi.org/10.3748/wjg.v25.i6.672
  9. M. A. Al-Garadi et al., “Text classification models for the automatic detection of nonmedical prescription medication use from social media,” BMC Med Inform Decis Mak, vol. 21, pp. 1–13, 2021. https://doi.org/10.1186/s12911-021-01394-0
  10. X. Li, H. Wang, H. He, J. Du, J. Chen, and J. Wu, “Intelligent diagnosis with Chinese electronic medical records based on convolutional neural networks,” BMC Bioinformatics, vol. 20, pp. 1–12, 2019. https://doi.org/10.1186/s12859-019-2617-8
  11. H. Lu, L. Ehwerhemuepha, and C. Rakovski, “A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance,” BMC Med Res Methodol, vol. 22, no. 1, p. 181, 2022. https://doi.org/10.1186/s12874-022-01665-y
  12. P. Sai Nishant, S. Mehrotra, B. Mohan, and G. Devaraju, “Identifying Classification Technique for Medical Diagnosis,” 2020, pp. 95–104. https://doi.org/10.1007/978-981-15-0630-7_10
  13. R. Morales-Sánchez, S. Montalvo, A. Riaño, R. Mart’inez, and M. Velasco, “Early diagnosis of HIV cases by means of text mining and machine learning models on clinical notes,” Comput Biol Med, vol. 179, p. 108830, 2024. https://doi.org/10.1016/j.compbiomed.2024.108830
  14. D. Pak et al., “Application of text-classification based machine learning in predicting psychiatric diagnosis,” Korean Journal of Biological Psychiatry, vol. 27, no. 1, pp. 18–26, 2020. https://doi.org/10.22857/kjbp.2020.27.1.003
  15. S. Cohen, A.-S. Jannot, L. Iserin, D. Bonnet, A. Burgun, and J.-B. Escudié, “Accuracy of claim data in the identification and classification of adults with congenital heart diseases in electronic medical records,” Arch Cardiovasc Dis, vol. 112, no. 1, pp. 31–43, 2019. https://doi.org/10.1016/j.acvd.2018.07.002
  16. Z. I. Attia, D. M. Harmon, E. R. Behr, and P. A. Friedman, “Application of artificial intelligence to the electrocardiogram,” Eur Heart J, vol. 42, no. 46, pp. 4717–4730, 2021. https://doi.org/10.1093/eurheartj/ehab649
  17. R. Vliegenthart, A. Fouras, C. Jacobs, and N. Papanikolaou, “Innovations in thoracic imaging: CT, radiomics, AI and x-ray velocimetry,” Respirology, vol. 27, no. 10, pp. 818–833, 2022. https://doi.org/10.1111/resp.14344
  18. M. Jamaluddin and A. D. Wibawa, “Patient Diagnosis Classification based on Electronic Medical Record using Text Mining and Support Vector Machine,” in Proceedings - 2021 International Seminar on Application for Technology of Information and Communication, in Proceedings - 2021 International Seminar on Application for Technology of Information and Communication: IT Opportunities and Creativities for Digital Innovation and Communication within Global Pandemic, iSemantic 2021. United States: Institute of Electrical and Electronics Engineers Inc., Sep. 2021, pp. 243–248. https://doi.org/10.1109/iSemantic52711.2021.9573178
  19. X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks and defenses for deep learning,” IEEE Trans Neural Netw Learn Syst, vol. 30, no. 9, pp. 2805–2824, 2019. https://doi.org/10.1109/TNNLS.2018.2886017
  20. H. Xu et al., “Adversarial attacks and defenses in images, graphs and text: A review,” International journal of automation and computing, vol. 17, pp. 151–178, 2020. https://doi.org/10.48550/arXiv.1909.08072
  21. Y. Li, M. Cheng, C.-J. Hsieh, and T. C. M. Lee, “A review of adversarial attack and defense for classification methods,” Am Stat, vol. 76, no. 4, pp. 329–345, 2022. https://doi.org/10.1080/00031305.2021.2006781
  22. J. Li, S. Ji, T. Du, B. Li, and T. Wang, “TextBugger: Generating Adversarial Text Against Real-world Applications,” in Proceedings 2019 Network and Distributed System Security Symposium, in NDSS 2019. Internet Society, 2019. https://doi.org/10.14722/ndss.2019.23138
  23. M. Cheng, J. Yi, P.-Y. Chen, H. Zhang, and C.-J. Hsieh, “Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples,” 2020. https://doi.org/10.48550/arXiv.1803.01128
  24. D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment,” 2020. https://doi.org/10.48550/arXiv.1907.11932
  25. G. Apruzzese, M. Colajanni, L. Ferretti, and M. Marchetti, “Addressing adversarial attacks against security systems based on machine learning,” in 2019 11th international conference on cyber conflict (CyCon), 2019, pp. 1–18. https://doi.org/10.23919/CYCON.2019.8756865
  26. E. Anthi, L. Williams, M. Rhode, P. Burnap, and A. Wedgbury, “Adversarial attacks on machine learning cybersecurity defences in industrial control systems,” Journal of Information Security and Applications, vol. 58, p. 102717, 2021. https://doi.org/10.1016/j.jisa.2020.102717
  27. I. Rosenberg, A. Shabtai, Y. Elovici, and L. Rokach, “Adversarial machine learning attacks and defense methods in the cyber security domain,” ACM Computing Surveys (CSUR), vol. 54, no. 5, pp. 1–36, 2021. https://doi.org/10.48550/arXiv.2007.02407
  28. M. Macas, C. Wu, and W. Fuertes, “Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems,” Expert Syst Appl, vol. 238, p. 122223, 2024. https://doi.org/10.1016/j.eswa.2023.122223
  29. S. G. Finlayson, H. W. Chung, I. S. Kohane, and A. L. Beam, “Adversarial attacks against medical deep learning systems,” arXiv preprint arXiv:1804.05296, 2018. https://doi.org/10.48550/arXiv.1804.05296
  30. X. Li and D. Zhu, “Robust detection of adversarial attacks on medical images,” in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020, pp. 1154–1158. https://doi.org/10.1109/ISBI45749.2020.9098628
  31. S. G. Finlayson, J. D. Bowers, J. Ito, J. L. Zittrain, A. L. Beam, and I. S. Kohane, “Adversarial attacks on medical machine learning,” Science (1979), vol. 363, no. 6433, pp. 1287–1289, 2019. https://doi.org/10.1126/science.aaw4399
  32. M.-J. Tsai, P.-Y. Lin, and M.-E. Lee, “Adversarial attacks on medical image classification,” Cancers (Basel), vol. 15, no. 17, p. 4228, 2023. https://doi.org/10.3390/cancers15174228
  33. E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing NLP,” arXiv preprint arXiv:1908.07125, 2019. https://doi.org/10.48550/arXiv.1908.07125
  34. X. Han et al., “BFS2Adv: black-box adversarial attack towards hard-to-attack short texts,” Comput Secur, vol. 141, p. 103817, 2024. https://doi.org/10.1016/j.cose.2024.103817
  35. L. Song, X. Yu, H.-T. Peng, and K. Narasimhan, “Universal adversarial attacks with natural triggers for text classification,” arXiv preprint arXiv:2005.00174, 2020. https://doi.org/10.18653/v1/2021.naacl-main.291
  36. L. Xu, L. Berti-Equille, A. Cuesta-Infante, and K. Veeramachaneni, “Improving textual adversarial attacks using metric-guided rewrite and rollback,” 2024.
  37. C. Guo, A. Sablayrolles, H. Jégou, and D. Kiela, “Gradient-based adversarial attacks against text transformers,” arXiv preprint arXiv:2104.13733, 2021. https://doi.org/10.48550/arXiv.2104.13733
  38. A. Huq, M. Pervin, and others, “Adversarial attacks and defense on texts: A survey,” arXiv preprint arXiv:2005.14108, 2020. https://doi.org/10.48550/arXiv.2005.14108
  39. H. Waghela, S. Rakshit, and J. Sen, “A modified word saliency-based adversarial attack on text classification models,” in International Conference on Computing, Intelligence and Data Analytics, 2024, pp. 371–382. https://doi.org/10.1007/978-981-96-0451-7_27
  40. A. Samadi and A. Sullivan, “Evaluating Text Classification Robustness to Part-of-Speech Adversarial Examples,” arXiv preprint arXiv:2408.08374, 2024. https://doi.org/10.48550/arXiv.2408.08374
  41. M. Mozes, M. Bartolo, P. Stenetorp, B. Kleinberg, and L. D. Griffin, “Contrasting human-and machine-generated word-level adversarial examples for text classification,” arXiv preprint arXiv:2109.04385, 2021. https://doi.org/10.18653/v1/2021.emnlp-main.651
  42. J. Hauser, Z. Meng, D. Pascual, and R. Wattenhofer, “Bert is robust! a case against synonym-based adversarial examples in text classification,” arXiv preprint arXiv:2109.07403, 2021. https://doi.org/10.48550/arXiv.2109.07403
  43. M. G. Hussain, B. Sultana, M. Rahman, and M. R. Hasan, “Comparison analysis of bangla news articles classification using support vector machine and logistic regression,” TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 21, no. 3, pp. 584–591, 2023. http://doi.org/10.12928/telkomnika.v21i3.23416
  44. X. Luo, “Efficient English text classification using selected machine learning techniques,” Alexandria Engineering Journal, vol. 60, no. 3, pp. 3401–3409, 2021. https://doi.org/10.1016/j.aej.2021.02.009
  45. A. Bhavani and B. S. Kumar, “A review of state art of text classification algorithms,” in 2021 5th international conference on computing methodologies and communication (ICCMC), 2021, pp. 1484–1490. https://doi.org/10.1109/ICCMC51019.2021.9418262
  46. L. Taherkhani, A. Daneshvar, H. Amoozad Khalili, and M. R. Sanaei, “Analysis of the Customer Churn Prediction Project in the Hotel Industry Based on Text Mining and the Random Forest Algorithm,” Advances in Civil Engineering, vol. 2023, no. 1, p. 6029121, 2023. https://doi.org/10.1155/2023/6029121
  47. S. Ghosal and A. Jain, “Depression and suicide risk detection on social media using fasttext embedding and xgboost classifier,” Procedia Comput Sci, vol. 218, pp. 1631–1639, 2023. https://doi.org/10.1016/j.procs.2023.01.141
  48. P. W. Khan, Y. C. Byun, and O.-R. Jeong, “A stacking ensemble classifier-based machine learning model for classifying pollution sources on photovoltaic panels,” Sci Rep, vol. 13, no. 1, p. 10256, 2023. https://doi.org/10.1038/s41598-023-35476-y
  49. A. Abdellatif et al., “Forecasting photovoltaic power generation with a stacking ensemble model,” Sustainability, vol. 14, no. 17, p. 11083, 2022. https://doi.org/10.3390/su141711083
  50. S. Chatterjee and Y.-C. Byun, “EEG-based emotion classification using stacking ensemble approach,” Sensors, vol. 22, no. 21, p. 8550, 2022. https://doi.org/10.3390/s22218550
  51. Y. Zhang, J. Ma, S. Liang, X. Li, and J. Liu, “A stacking ensemble algorithm for improving the biases of forest aboveground biomass estimations from multiple remotely sensed datasets,” GIsci Remote Sens, vol. 59, no. 1, pp. 234–249, 2022. https://doi.org/10.1080/15481603.2021.2023842
  52. N. Chattopadhyay, A. Goswami, and A. Chattopadhyay, “Adversarial Attacks and Dimensionality in Text Classifiers,” arXiv preprint arXiv:2404.02660, 2024. https://doi.org/10.48550/arXiv.2404.02660
  53. D. Li et al., “Contextualized perturbation for textual adversarial attack,” arXiv preprint arXiv:2009.07502, 2020. https://doi.org/10.18653/v1/2021.naacl-main.400
  54. C. Guo, A. Sablayrolles, H. Jégou, and D. Kiela, “Gradient-based adversarial attacks against text transformers,” arXiv preprint arXiv:2104.13733, 2021. https://doi.org/10.48550/arXiv.2104.13733
  55. Y. Gu et al., “Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing,” 2020. https://doi.org/10.1145/3458754
  56. J. X. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, and Y. Qi, “TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP,” 2020. https://doi.org/10.48550/arXiv.2005.05909
  57. N. Mrkšić et al., “Counter-fitting Word Vectors to Linguistic Constraints,” in Proceedings of HLT-NAACL, 2016. https://doi.org/10.18653/v1/N16-1018
  58. H. Henderi, W. Winarno, and others, “Text Mining an Automatic Short Answer Grading (ASAG), Comparison of Three Methods of Cosine Similarity, Jaccard Similarity and Dice’s Coefficient,” Journal of Applied Data Sciences, vol. 2, no. 2, 2021. https://doi.org/10.47738/jads.v2i2.31
Read More
Author biographies is not available.
Download this PDF file
PDF
Statistic
Read Counter : 0 Download : 0

Downloads

Download data is not yet available.