This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Regularization Techniques to Improving the Stability and Accuracy of the MLC Algorithm
Corresponding Author(s) : Usman Sudibyo
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 11, No. 3, August 2026 (Article in Progress)
Abstract
Maximum Likelihood Classification (MLC) is a classification algorithm that has important applications in the fields of image processing and remote sensing. No use of MLC was found in other fields. MLC assumes that data come from a certain probability distribution (for example, a normal distribution), which may be too simple to describe complex data or data with a non-normal distribution. This can lead to poor performance in situations where distribution assumptions are not met. That is why, in the existing literature, there is no use of MLC for classification problems other than remote sensing. We propose a regularization technique to reduce distribution assumption errors in MLC called Regularized Maximum Likelihood Classification (RMLC). Regularization techniques are integrated into the covariance matrix, where regularization can make the data variance larger or smaller than the actual variance. This technique can also overcome singularities in the covariance matrix, non-Gaussian data, and data containing outliers. Experimental results on 13 public datasets show a significant increase in accuracy performance. The average accuracy increase reaches more than 11%, from 0.802 to 0.919, highlighting its potential for broader applicability and enhanced performance.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- D. G. Stork, Pattern Classification, 2nd ed. Sand Hill Road, 2003.
- D. Mccraine, S. Samiappan, L. Kohler, T. Sullivan, and D. J. Will, “Automated Hyperspectral Feature Selection and Classification of Wildlife Using Uncrewed Aerial Vehicles,” MDPI, pp. 1–18, 2024. https://doi.org/10.3390/rs16020406
- Y. Dian, S. Fang, Y. Le, Y. Xu, and C. Yao, “Comparison of the Different Classifiers in Vegetation Species Discrimination Using Hyperspectral Reflectance Data,” J. Indian Soc. Remote Sens., vol. 42, no. 1, pp. 61–72, 2014. https://doi.org/10.1007/s12524-013-0309-9
- X. Zhao, J. Ma, L. Wang, Z. Zhang, Y. Ding, and X. Xiao, “A review of hyperspectral image classification based on graph neural networks,” Artif. Intell. Rev., 2025. https://doi.org/10.1007/s10462-025-11169-y
- V. N. Balabathina and S. Mishra, “Comparative evaluation of fast-learning classi fi cation algorithms for urban forest tree species identi fi cation using EO-1 hyperion hyperspectral imagery,” Front. Environ. Sci., no. October, pp. 1–14, 2025. https://doi.org/10.3389/fenvs.2025.1668746
- L. Li, H. Ge, J. Gao, Y. Zhang, Y. Tong, and J. Sun, “Method for Hyperspectral Image Feature Extraction,” Neural Process. Lett., pp. 515–542, 2020. https://doi.org/10.1007/s11063-019-10101-0
- Y. Zhang, J. Ren, and J. Jiang, “Combining MLC and SVM classifiers for learning based decision making: Analysis and evaluations,” Comput. Intell. Neurosci., vol. 2015, 2015. https://doi.org/10.1155/2015/423581
- C. M. Bishop, Pattern Recognition and Machine Learning, vol. 27, no. 1. 2004.
- F. C. Pereira, A. M. Gonçalves, and M. Costa, “Outliers Impact on Parameter Estimation of Gaussian and Non-Gaussian State Space Models: A Simulation Study †,” Eng. Proc., vol. 18, no. 1, pp. 1–10, 2022. https://doi.org/10.3390/engproc2022018031
- P. S. Sisodia, V. Tiwari, and A. Kumar, “Analysis of Supervised Maximum Likelihood Classification for remote sensing image,” Int. Conf. Recent Adv. Innov. Eng. ICRAIE 2014, pp. 9–12, 2014. https://doi.org/10.1109/ICRAIE.2014.6909319
- M. A. Sohl, S. A. Mahmood, and M. U. Rasheed, “Comparative performance of four machine learning models for land cover classification in a low-cost UAV ultra-high-resolution RGB-only orthomosaic,” Earth Sci. Informatics, 2024. https://doi.org/10.1007/s12145-024-01318-2
- G. R. Liu, “Overfitting and Regularization,” Mach. Learn. with Python, no. 2, pp. 501–538, 2022. https://doi.org/10.1142/9789811254185_0014
- K. Elkhalil, A. Kammoun, R. Couillet, T. Y. Al-Naffouri, and M.-S. Alouini, “A Large Dimensional Study of Regularized Discriminant Analysis,” IEEE Trans. Signal Process., vol. 68, pp. 2464–2479, 2020. https://doi.org/10.1109/tsp.2020.2984160
- P. Zwiernik, C. Uhler, and D. Richards, “Maximum likelihood estimation for linear Gaussian covariance models,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 79, no. 4, pp. 1269–1292, 2017. https://doi.org/10.1111/rssb.12217
- J. Friedman, T. Hastie, and R. Tibshirani, “Journal of Statistical Software,” vol. 33, no. 1, 2010.
- H. Nugroho, W. Widodo, and A. Rachman, “Pattern Recognition Bird Sounds Based on Their Type Using Discreate Cosine Transform (DCT) and Gaussian Methods,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 3, pp. 233–240, 2019. https://doi.org/10.22219/kinetik.v4i3.791
- A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Commun., vol. 30, no. 2, pp. 169–190, 2017. https://doi.org/10.3233/AIC-170729
- N. Rastin, M. Z. Jahromi, and M. Taheri, “A generalized weighted distance k-Nearest Neighbor for multi-label problems,” Pattern Recognit., vol. 114, p. 107526, 2021. https://doi.org/10.1016/j.patcog.2020.107526
- A. Karatzoglou, D. Meyer, and K. Hornik, “Support Vector Algorithm in R,” J. Stat. Softw., vol. 15, no. 9, pp. 1–28, 2006. https://doi.org/10.18637/jss.v015.i09
- J. Liu, X. Xiong, P. Ren, C. N. Li, and Y. H. Shao, “Capped norm linear discriminant analysis and its applications,” Appl. Intell., pp. 18488–18507, 2023. https://doi.org/10.1007/s10489-022-04395-2
- C. N. Li, J. Liu, Y. Meng, and Y. H. Shao, “Recursive universum linear discriminant analysis,” Optim. Lett., no. 0123456789, 2023. https://doi.org/10.1007/s11590-023-02067-9
- F. Zhu, J. Gao, J. Yang, and N. Ye, “Neighborhood linear discriminant analysis,” Pattern Recognit., vol. 123, p. 108422, 2022. https://doi.org/10.1016/j.patcog.2021.108422
- E. Hamouda, A. S. Abohamama, and M. Tarek, “Random Projection-Based Feature Transformation Using Metaheuristic Optimization Algorithm,” Arab. J. Sci. Eng., vol. 46, no. 9, pp. 8345–8353, 2021. https://doi.org/10.1007/s13369-021-05474-1
- N. Auguin, D. Morales-Jimenez, and M. McKay, “Large-Dimensional Characterization of Robust Linear Discriminant Analysis,” IEEE Trans. Signal Process., vol. 69, pp. 2625–2638, 2021. https://doi.org/10.1109/TSP.2021.3075150
- K. K. Huang, D. Q. Dai, and C. X. Ren, “Regularized coplanar discriminant analysis for dimensionality reduction,” Pattern Recognit., vol. 62, pp. 87–98, 2017. https://doi.org/10.1016/j.patcog.2016.08.024
- M. Kelly, R. Longjohn, and K. Nottingham, “The UCI Machine Learning Repository”.
- D. G. Z. Selcuk Korkmaz, “Package 'MVN": Multivariate normality tests.,” pp. 1–6, 2022.
- E. Gómez-Déniz, J. M. Sarabia, and E. Calderín-Ojeda, “Bimodal normal distribution: Extensions and applications,” J. Comput. Appl. Math., vol. 388, p. 113292, 2021. https://doi.org/10.1016/j.cam.2020.113292
References
D. G. Stork, Pattern Classification, 2nd ed. Sand Hill Road, 2003.
D. Mccraine, S. Samiappan, L. Kohler, T. Sullivan, and D. J. Will, “Automated Hyperspectral Feature Selection and Classification of Wildlife Using Uncrewed Aerial Vehicles,” MDPI, pp. 1–18, 2024. https://doi.org/10.3390/rs16020406
Y. Dian, S. Fang, Y. Le, Y. Xu, and C. Yao, “Comparison of the Different Classifiers in Vegetation Species Discrimination Using Hyperspectral Reflectance Data,” J. Indian Soc. Remote Sens., vol. 42, no. 1, pp. 61–72, 2014. https://doi.org/10.1007/s12524-013-0309-9
X. Zhao, J. Ma, L. Wang, Z. Zhang, Y. Ding, and X. Xiao, “A review of hyperspectral image classification based on graph neural networks,” Artif. Intell. Rev., 2025. https://doi.org/10.1007/s10462-025-11169-y
V. N. Balabathina and S. Mishra, “Comparative evaluation of fast-learning classi fi cation algorithms for urban forest tree species identi fi cation using EO-1 hyperion hyperspectral imagery,” Front. Environ. Sci., no. October, pp. 1–14, 2025. https://doi.org/10.3389/fenvs.2025.1668746
L. Li, H. Ge, J. Gao, Y. Zhang, Y. Tong, and J. Sun, “Method for Hyperspectral Image Feature Extraction,” Neural Process. Lett., pp. 515–542, 2020. https://doi.org/10.1007/s11063-019-10101-0
Y. Zhang, J. Ren, and J. Jiang, “Combining MLC and SVM classifiers for learning based decision making: Analysis and evaluations,” Comput. Intell. Neurosci., vol. 2015, 2015. https://doi.org/10.1155/2015/423581
C. M. Bishop, Pattern Recognition and Machine Learning, vol. 27, no. 1. 2004.
F. C. Pereira, A. M. Gonçalves, and M. Costa, “Outliers Impact on Parameter Estimation of Gaussian and Non-Gaussian State Space Models: A Simulation Study †,” Eng. Proc., vol. 18, no. 1, pp. 1–10, 2022. https://doi.org/10.3390/engproc2022018031
P. S. Sisodia, V. Tiwari, and A. Kumar, “Analysis of Supervised Maximum Likelihood Classification for remote sensing image,” Int. Conf. Recent Adv. Innov. Eng. ICRAIE 2014, pp. 9–12, 2014. https://doi.org/10.1109/ICRAIE.2014.6909319
M. A. Sohl, S. A. Mahmood, and M. U. Rasheed, “Comparative performance of four machine learning models for land cover classification in a low-cost UAV ultra-high-resolution RGB-only orthomosaic,” Earth Sci. Informatics, 2024. https://doi.org/10.1007/s12145-024-01318-2
G. R. Liu, “Overfitting and Regularization,” Mach. Learn. with Python, no. 2, pp. 501–538, 2022. https://doi.org/10.1142/9789811254185_0014
K. Elkhalil, A. Kammoun, R. Couillet, T. Y. Al-Naffouri, and M.-S. Alouini, “A Large Dimensional Study of Regularized Discriminant Analysis,” IEEE Trans. Signal Process., vol. 68, pp. 2464–2479, 2020. https://doi.org/10.1109/tsp.2020.2984160
P. Zwiernik, C. Uhler, and D. Richards, “Maximum likelihood estimation for linear Gaussian covariance models,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 79, no. 4, pp. 1269–1292, 2017. https://doi.org/10.1111/rssb.12217
J. Friedman, T. Hastie, and R. Tibshirani, “Journal of Statistical Software,” vol. 33, no. 1, 2010.
H. Nugroho, W. Widodo, and A. Rachman, “Pattern Recognition Bird Sounds Based on Their Type Using Discreate Cosine Transform (DCT) and Gaussian Methods,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 3, pp. 233–240, 2019. https://doi.org/10.22219/kinetik.v4i3.791
A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Commun., vol. 30, no. 2, pp. 169–190, 2017. https://doi.org/10.3233/AIC-170729
N. Rastin, M. Z. Jahromi, and M. Taheri, “A generalized weighted distance k-Nearest Neighbor for multi-label problems,” Pattern Recognit., vol. 114, p. 107526, 2021. https://doi.org/10.1016/j.patcog.2020.107526
A. Karatzoglou, D. Meyer, and K. Hornik, “Support Vector Algorithm in R,” J. Stat. Softw., vol. 15, no. 9, pp. 1–28, 2006. https://doi.org/10.18637/jss.v015.i09
J. Liu, X. Xiong, P. Ren, C. N. Li, and Y. H. Shao, “Capped norm linear discriminant analysis and its applications,” Appl. Intell., pp. 18488–18507, 2023. https://doi.org/10.1007/s10489-022-04395-2
C. N. Li, J. Liu, Y. Meng, and Y. H. Shao, “Recursive universum linear discriminant analysis,” Optim. Lett., no. 0123456789, 2023. https://doi.org/10.1007/s11590-023-02067-9
F. Zhu, J. Gao, J. Yang, and N. Ye, “Neighborhood linear discriminant analysis,” Pattern Recognit., vol. 123, p. 108422, 2022. https://doi.org/10.1016/j.patcog.2021.108422
E. Hamouda, A. S. Abohamama, and M. Tarek, “Random Projection-Based Feature Transformation Using Metaheuristic Optimization Algorithm,” Arab. J. Sci. Eng., vol. 46, no. 9, pp. 8345–8353, 2021. https://doi.org/10.1007/s13369-021-05474-1
N. Auguin, D. Morales-Jimenez, and M. McKay, “Large-Dimensional Characterization of Robust Linear Discriminant Analysis,” IEEE Trans. Signal Process., vol. 69, pp. 2625–2638, 2021. https://doi.org/10.1109/TSP.2021.3075150
K. K. Huang, D. Q. Dai, and C. X. Ren, “Regularized coplanar discriminant analysis for dimensionality reduction,” Pattern Recognit., vol. 62, pp. 87–98, 2017. https://doi.org/10.1016/j.patcog.2016.08.024
M. Kelly, R. Longjohn, and K. Nottingham, “The UCI Machine Learning Repository”.
D. G. Z. Selcuk Korkmaz, “Package 'MVN": Multivariate normality tests.,” pp. 1–6, 2022.
E. Gómez-Déniz, J. M. Sarabia, and E. Calderín-Ojeda, “Bimodal normal distribution: Extensions and applications,” J. Comput. Appl. Math., vol. 388, p. 113292, 2021. https://doi.org/10.1016/j.cam.2020.113292