Issue

Vol. 10, No. 2, May 2025

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Integrating Adaptive Sampling with Ensembles Model for Software Defect Prediction

https://doi.org/10.22219/kinetik.v10i2.2191
Muhammad Yusuf
Institut Teknologi Sepuluh Nopember
Arinal Haq
Institut Teknologi Sepuluh Nopember
Siti Rochimah
Institut Teknologi Sepuluh Nopember

Corresponding Author(s) : Siti Rochimah

siti@its.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 10, No. 2, May 2025

Abstract

Handling class imbalance is a challenge in software defect prediction. Imbalanced datasets can cause bias in machine learning models, hindering their ability to detect defects. This paper proposes an integration of Adaptive Synthetic Sampling (ADASYN) and ensemble learning methods to improve prediction accuracy. ADASYN enhances the handling of imbalanced data by generating synthetic samples for hard-to-classify instances, while the ensemble stacking technique leverages the strengths of multiple models to reduce bias and variance. The machine learning model used in this study is K-Nearest Neighbors (KNN), Decision Tree (DT), and Random Forest (RF). The results demonstrate that ADASYN combined with ensemble stacking outperforms the traditional SMOTE technique in most cases. For instance, in the Ant-1.7 dataset, ADASYN achieved a stacking accuracy of 90.60% compared to 89.32% with SMOTE. Similarly, in the Camel-1.6 dataset, ADASYN achieved 91.56%, slightly exceeding SMOTE’s 91.32%. However, SMOTE performed better in simpler models like Decision Tree for certain datasets, highlighting the importance of choosing the appropriate resampling method. Across all datasets, ensemble stacking consistently provided the highest accuracy, benefiting from ADASYN's adaptive resampling strategy. These results underscore the importance of combining advanced sampling methods with ensemble learning techniques to address class imbalance effectively. This approach improves prediction accuracy and provides a practical framework for reliable software defect prediction in real-world scenarios. Future work will explore hybrid techniques and broader evaluations across diverse datasets and classifiers.

Keywords

Software Defect Prediction ADASYN Class Imbalance Ensemble Learning Stacking
Yusuf, M., Haq, A., & Rochimah, S. (2025). Integrating Adaptive Sampling with Ensembles Model for Software Defect Prediction. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 10(2). https://doi.org/10.22219/kinetik.v10i2.2191
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. [E. Izquierdo-Verdiguier and R. Zurita-Milla, “An evaluation of Guided Regularized Random Forest for classification and regression tasks in remote sensing,” Int. J. Appl. Earth Obs. Geoinformation, vol. 88, p. 102051, Jun. 2020, doi: 10.1016/j.jag.2020.102051.
  2. A. Kurani, P. Doshi, A. Vakharia, and M. Shah, “A Comprehensive Comparative Study of Artificial Neural Network (ANN) and Support Vector Machines (SVM) on Stock Forecasting,” Ann. Data Sci., vol. 10, no. 1, pp. 183–208, Feb. 2023, doi: 10.1007/s40745-021-00344-x.
  3. A. Antoniadis, S. Lambert-Lacroix, and J.-M. Poggi, “Random forests for global sensitivity analysis: A selective review,” Reliab. Eng. Syst. Saf., vol. 206, p. 107312, Feb. 2021, doi: 10.1016/j.ress.2020.107312.
  4. Y. Tang, Q. Dai, M. Yang, T. Du, and L. Chen, “Software defect prediction ensemble learning algorithm based on adaptive variable sparrow search algorithm,” Int. J. Mach. Learn. Cybern., vol. 14, no. 6, pp. 1967–1987, Jun. 2023, doi: 10.1007/s13042-022-01740-2.
  5. Y. Zakariyau Bala, P. Abdul Samat, K. Yatim Sharif, and N. Manshor, “Cross-project software defect prediction through multiple learning,” Bull. Electr. Eng. Inform., vol. 13, no. 3, pp. 2027–2035, Jun. 2024, doi: 10.11591/eei.v13i3.5258.
  6. M. Aria, C. Cuccurullo, and A. Gnasso, “A comparison among interpretative proposals for Random Forests,” Mach. Learn. Appl., vol. 6, p. 100094, Dec. 2021, doi: 10.1016/j.mlwa.2021.100094.
  7. K. K. Bejjanki, J. Gyani, and N. Gugulothu, “Class Imbalance Reduction (CIR): A Novel Approach to Software Defect Prediction in the Presence of Class Imbalance,” Symmetry, vol. 12, no. 3, p. 407, Mar. 2020, doi: 10.3390/sym12030407.
  8. N. A. A. Khleel and K. Nehéz, “Software defect prediction using a bidirectional LSTM network combined with oversampling techniques,” Clust. Comput., vol. 27, no. 3, pp. 3615–3638, Jun. 2024, doi: 10.1007/s10586-023-04170-z.
  9. N. S. Thomas and S. Kaliraj, “An Improved and Optimized Random Forest Based Approach to Predict the Software Faults,” SN Comput. Sci., vol. 5, no. 5, p. 530, May 2024, doi: 10.1007/s42979-024-02764-x.
  10. M. N. M. Rahman, R. A. Nugroho, M. R. Faisal, F. Abadi, and R. Herteno, “Optimized multi correlation-based feature selection in software defect prediction,” TELKOMNIKA Telecommun. Comput. Electron. Control, vol. 22, no. 3, p. 598, Jun. 2024, doi: 10.12928/telkomnika.v22i3.25793.
  11. Y. Sun et al., “Unsupervised Domain Adaptation Based on Discriminative Subspace Learning for Cross-Project Defect Prediction,” Comput. Mater. Contin., vol. 68, no. 3, pp. 3373–3389, 2021, doi: 10.32604/cmc.2021.016539.
  12. T. Zhang et al., “Intelligent fault diagnosis of machines with small & imbalanced data: A state-of-the-art review and possible extensions,” ISA Trans., vol. 119, pp. 152–171, Jan. 2022, doi: 10.1016/j.isatra.2021.02.042.
  13. W. Huang et al., “Railway dangerous goods transportation system risk identification: Comparisons among SVM, PSO-SVM, GA-SVM and GS-SVM,” Appl. Soft Comput., vol. 109, p. 107541, Sep. 2021, doi: 10.1016/j.asoc.2021.107541.
  14. Y. Tang, Q. Dai, M. Yang, T. Du, and L. Chen, “Software defect prediction ensemble learning algorithm based on adaptive variable sparrow search algorithm,” Int. J. Mach. Learn. Cybern., vol. 14, no. 6, pp. 1967–1987, Jun. 2023, doi: 10.1007/s13042-022-01740-2.
  15. A. Antoniadis, S. Lambert-Lacroix, and J.-M. Poggi, “Random forests for global sensitivity analysis: A selective review,” Reliab. Eng. Syst. Saf., vol. 206, p. 107312, Feb. 2021, doi: 10.1016/j.ress.2020.107312.
  16. M. Hammad, M. H. Alkinani, B. B. Gupta, and A. A. Abd El-Latif, “Myocardial infarction detection based on deep neural network on imbalanced data,” Multimed. Syst., vol. 28, no. 4, pp. 1373–1385, Aug. 2022, doi: 10.1007/s00530-020-00728-8.
  17. J. Zheng, X. Wang, D. Wei, B. Chen, and Y. Shao, “A Novel Imbalanced Ensemble Learning in Software Defect Predication,” IEEE Access, vol. 9, pp. 86855–86868, 2021, doi: 10.1109/ACCESS.2021.3072682.
  18. K. Vos, Z. Peng, C. Jenkins, M. R. Shahriar, P. Borghesani, and W. Wang, “Vibration-based anomaly detection using LSTM/SVM approaches,” Mech. Syst. Signal Process., vol. 169, p. 108752, Apr. 2022, doi: 10.1016/j.ymssp.2021.108752.
  19. M. Nevendra and P. Singh, “Empirical investigation of hyperparameter optimization for software defect count prediction,” Expert Syst. Appl., vol. 191, p. 116217, Apr. 2022, doi: 10.1016/j.eswa.2021.116217.
  20. X. Pu et al., “Improving Lower Limb Function and Frailty in Frail Older Patients with Acute Myocardial Infarction After Percutaneous Coronary Intervention: A Randomized Controlled Study of Neuromuscular Electrical Stimulation,” Clin. Interv. Aging, vol. Volume 19, pp. 1163–1176, Jul. 2024, doi: 10.2147/CIA.S460805.
  21. M. N. M. Rahman, R. A. Nugroho, M. R. Faisal, F. Abadi, and R. Herteno, “Optimized multi correlation-based feature selection in software defect prediction,” TELKOMNIKA Telecommun. Comput. Electron. Control, vol. 22, no. 3, p. 598, Jun. 2024, doi: 10.12928/telkomnika.v22i3.25793.
  22. M. Mohammadi et al., “A comprehensive survey and taxonomy of the SVM-based intrusion detection systems,” J. Netw. Comput. Appl., vol. 178, p. 102983, Mar. 2021, doi: 10.1016/j.jnca.2021.102983.
  23. Y. Wang, Z. Pan, and J. Dong, “A new two-layer nearest neighbor selection method for kNN classifier,” Knowl.-Based Syst., vol. 235, p. 107604, Jan. 2022, doi: 10.1016/j.knosys.2021.107604.
  24. M. Rizky Pribadi, H. Dwi Purnomo, and H. Hendry, “A three-step combination strategy for addressing outliers and class imbalance in software defect prediction,” IAES Int. J. Artif. Intell. IJ-AI, vol. 13, no. 3, p. 2987, Sep. 2024, doi: 10.11591/ijai.v13.i3.pp2987-2998.
  25. M. K. Suryadi, R. Herteno, S. W. Saputro, M. R. Faisal, and R. A. Nugroho, “Comparative Study of Various Hyperparameter Tuning on Random Forest Classification With SMOTE and Feature Selection Using Genetic Algorithm in Software Defect Prediction,” J. Electron. Electromed. Eng. Med. Inform., vol. 6, no. 2, pp. 137–147, Mar. 2024, doi: 10.35882/jeeemi.v6i2.375.
  26. A. W. Dar and S. U. Farooq, “Handling class overlap and imbalance using overlap driven under-sampling with balanced random forest in software defect prediction,” Innov. Syst. Softw. Eng., Jun. 2024, doi: 10.1007/s11334-024-00571-4.
  27. S. Goyal, “Heterogeneous Stacked Ensemble Classifier for Software Defect Prediction,” in 2020 Sixth International Conference on Parallel, Distributed and Grid Computing (PDGC), Waknaghat, India: IEEE, Nov. 2020, pp. 126–130. doi: 10.1109/PDGC50313.2020.9315754.
  28. S. Feng, J. Keung, X. Yu, Y. Xiao, and M. Zhang, “Investigation on the stability of SMOTE-based oversampling techniques in software defect prediction,” Inf. Softw. Technol., vol. 139, p. 106662, Nov. 2021, doi: 10.1016/j.infsof.2021.106662.
  29. A. Kurani, P. Doshi, A. Vakharia, and M. Shah, “A Comprehensive Comparative Study of Artificial Neural Network (ANN) and Support Vector Machines (SVM) on Stock Forecasting,” Ann. Data Sci., vol. 10, no. 1, pp. 183–208, Feb. 2023, doi: 10.1007/s40745-021-00344-x.
  30. T. Sharma, A. Jatain, S. Bhaskar, and K. Pabreja, “Ensemble Machine Learning Paradigms in Software Defect Prediction,” Procedia Comput. Sci., vol. 218, pp. 199–209, 2023, doi: 10.1016/j.procs.2023.01.002.
  31. Y. Z. Bala, P. Abdul Samat, K. Y. Sharif, and N. Manshor, “Improving Cross-Project Software Defect Prediction Method Through Transformation and Feature Selection Approach,” IEEE Access, vol. 11, pp. 2318–2326, 2023, doi: 10.1109/ACCESS.2022.3231456.
  32. M. H. D. M. Ribeiro, R. G. Da Silva, S. R. Moreno, V. C. Mariani, and L. D. S. Coelho, “Efficient bootstrap stacking ensemble learning model applied to wind power generation forecasting,” Int. J. Electr. Power Energy Syst., vol. 136, p. 107712, Mar. 2022, doi: 10.1016/j.ijepes.2021.107712.
  33. T. Wu, W. Zhang, X. Jiao, W. Guo, and Y. Alhaj Hamoud, “Evaluation of stacking and blending ensemble learning methods for estimating daily reference evapotranspiration,” Comput. Electron. Agric., vol. 184, p. 106039, May 2021, doi: 10.1016/j.compag.2021.106039.
  34. Z. Ding and L. Xing, “Improved software defect prediction using Pruned Histogram-based isolation forest,” Reliab. Eng. Syst. Saf., vol. 204, p. 107170, Dec. 2020, doi: 10.1016/j.ress.2020.107170.
  35. W. Huang et al., “Railway dangerous goods transportation system risk identification: Comparisons among SVM, PSO-SVM, GA-SVM and GS-SVM,” Appl. Soft Comput., vol. 109, p. 107541, Sep. 2021, doi: 10.1016/j.asoc.2021.107541.
  36. M. Rashid, J. Kamruzzaman, T. Imam, S. Wibowo, and S. Gordon, “A tree-based stacking ensemble technique with feature selection for network intrusion detection,” Appl. Intell., vol. 52, no. 9, pp. 9768–9781, Jul. 2022, doi: 10.1007/s10489-021-02968-1.
  37. N. A. A. Khleel and K. Nehéz, “Software defect prediction using a bidirectional LSTM network combined with oversampling techniques,” Clust. Comput., vol. 27, no. 3, pp. 3615–3638, Jun. 2024, doi: 10.1007/s10586-023-04170-z.
  38. N. Kardani, A. Zhou, M. Nazem, and S.-L. Shen, “Improved prediction of slope stability using a hybrid stacking ensemble method based on finite element analysis and field data,” J. Rock Mech. Geotech. Eng., vol. 13, no. 1, pp. 188–201, Feb. 2021, doi: 10.1016/j.jrmge.2020.05.011.
  39. A. Chatzimparmpas, R. M. Martins, K. Kucher, and A. Kerren, “StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics,” IEEE Trans. Vis. Comput. Graph., vol. 27, no. 2, pp. 1547–1557, Feb. 2021, doi: 10.1109/TVCG.2020.3030352.
  40. R. Lazzarini, H. Tianfield, and V. Charissis, “A stacking ensemble of deep learning models for IoT intrusion detection,” Knowl.-Based Syst., vol. 279, p. 110941, Nov. 2023, doi: 10.1016/j.knosys.2023.110941.
Read More
Author biographies is not available.
Download this PDF file
Statistic
Read Counter : 0

Downloads

Download data is not yet available.