This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Regularization Techniques to Improve the Stability and Accuracy of MLC Algorithm
Corresponding Author(s) : Usman Sudibyo
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 11, No. 3, August 2026 (Article in Progress)
Abstract
Maximum Likelihood Classification (MLC) is a classification algorithm that has important applications in the fields of image processing and remote sensing. No use of MLC was found in other fields. MLC assumes that data comes from a certain probability distribution (for example, a normal distribution), which may be too simple to describe complex data or have a non-normal distribution. This can lead to poor performance in situations where distribution assumptions are not met. That is why in various literatures there is no use of MLC for classification problems other than remote sensing. We propose a regularization technique to reduce distribution assumption errors in MLC called Regularization on maximum likelihood classification (RMLC). Regularization techniques are integrated into the covariance matrix, where regularization can make the data variance larger or smaller than the actual variance. This technique can also overcome singularities in the covariance matrix, non-Gaussian data, and data containing outliers. Experimental results on 13 public datasets show a significant increase in accuracy performance. The average accuracy increase reaches more than 11%, from 0.802 to 0.919, highlighting its potential for broader applicability and enhanced performance
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- Y. Dian, S. Fang, Y. Le, Y. Xu, and C. Yao, “Comparison of the Different Classifiers in Vegetation Species Discrimination Using Hyperspectral Reflectance Data,” J. Indian Soc. Remote Sens., vol. 42, no. 1, pp. 61–72, 2014, doi: 10.1007/s12524-013-0309-9.
- Y. Zhang, J. Ren, and J. Jiang, “Combining MLC and SVM classifiers for learning based decision making: Analysis and evaluations,” Comput. Intell. Neurosci., vol. 2015, 2015, doi: 10.1155/2015/423581.
- C. M. Bishop, Pattern Recognition and Machine Learning, vol. 27, no. 1. 2004.
- F. C. Pereira, A. M. Gonçalves, and M. Costa, “Outliers Impact on Parameter Estimation of Gaussian and Non-Gaussian State Space Models: A Simulation Study †,” Eng. Proc., vol. 18, no. 1, pp. 1–10, 2022, doi: 10.3390/engproc2022018031.
- L. Li, H. Ge, J. Gao, Y. Zhang, Y. Tong, and J. Sun, “Method for Hyperspectral Image Feature Extraction,” Neural Process. Lett., pp. 515–542, 2020, doi: 10.1007/s11063-019-10101-0.
- P. S. Sisodia, V. Tiwari, and A. Kumar, “Analysis of Supervised Maximum Likelihood Classification for remote sensing image,” Int. Conf. Recent Adv. Innov. Eng. ICRAIE 2014, pp. 9–12, 2014, doi: 10.1109/ICRAIE.2014.6909319.
- N. Dahiya, S. Singh, and S. Gupta, “Comparative Analysis and Implication of Hyperion Hyperspectral and Landsat-8 Multispectral Dataset in Land Classification,” J. Indian Soc. Remote Sens., vol. 51, no. 11, pp. 2201–2213, 2023, doi: 10.1007/s12524-023-01760-7.
- M. A. Sohl, S. A. Mahmood, and M. U. Rasheed, “Comparative performance of four machine learning models for land cover classification in a low-cost UAV ultra-high-resolution RGB-only orthomosaic,” Earth Sci. Informatics, 2024, doi: 10.1007/s12145-024-01318-2.
- G. R. Liu, “Overfitting and Regularization,” Mach. Learn. with Python, no. 2, pp. 501–538, 2022, doi: 10.1142/9789811254185_0014.
- K. Elkhalil, A. Kammoun, R. Couillet, T. Y. Al-Naffouri, and M.-S. Alouini, “A Large Dimensional Study of Regularized Discriminant Analysis,” IEEE Trans. Signal Process., vol. 68, pp. 2464–2479, 2020, doi: 10.1109/tsp.2020.2984160.
- P. Zwiernik, C. Uhler, and D. Richards, “Maximum likelihood estimation for linear Gaussian covariance models,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 79, no. 4, pp. 1269–1292, 2017, doi: 10.1111/rssb.12217.
- H. Nugroho, W. Widodo, and A. Rachman, “Pattern Recognition Bird Sounds Based on Their Type Using Discreate Cosine Transform (DCT) and Gaussian Methods,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 3, pp. 233–240, 2019, doi: 10.22219/kinetik.v4i3.791.
- N. Auguin, D. Morales-Jimenez, and M. McKay, “Large-Dimensional Characterization of Robust Linear Discriminant Analysis,” IEEE Trans. Signal Process., vol. 69, pp. 2625–2638, 2021, doi: 10.1109/TSP.2021.3075150.
- K. K. Huang, D. Q. Dai, and C. X. Ren, “Regularized coplanar discriminant analysis for dimensionality reduction,” Pattern Recognit., vol. 62, pp. 87–98, 2017, doi: 10.1016/j.patcog.2016.08.024.
- M. Kelly, R. Longjohn, and K. Nottingham, “The UCI Machine Learning Repository”, [Online]. Available: https://archive.ics.uci.edu
- D. G. Z. Selcuk Korkmaz, “Package 'MVN": Multivariate normality tests.,” pp. 1–6, 2022, [Online]. Available: https://orcid.org/0000-0003-4632-6850
- E. Gómez-Déniz, J. M. Sarabia, and E. Calderín-Ojeda, “Bimodal normal distribution: Extensions and applications,” J. Comput. Appl. Math., vol. 388, p. 113292, 2021, doi: 10.1016/j.cam.2020.113292.
- A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Commun., vol. 30, no. 2, pp. 169–190, 2017, doi: 10.3233/AIC-170729.
- N. Rastin, M. Z. Jahromi, and M. Taheri, “A generalized weighted distance k-Nearest Neighbor for multi-label problems,” Pattern Recognit., vol. 114, p. 107526, 2021, doi: 10.1016/j.patcog.2020.107526.
- A. Karatzoglou, D. Meyer, and K. Hornik, “Support Vector Algorithm in R,” J. Stat. Softw., vol. 15, no. 9, pp. 1–28, 2006, doi: 10.18637/jss.v081.b02.
- J. Liu, X. Xiong, P. Ren, C. N. Li, and Y. H. Shao, “Capped norm linear discriminant analysis and its applications,” Appl. Intell., pp. 18488–18507, 2023, doi: 10.1007/s10489-022-04395-2.
- C. N. Li, J. Liu, Y. Meng, and Y. H. Shao, “Recursive universum linear discriminant analysis,” Optim. Lett., no. 0123456789, 2023, doi: 10.1007/s11590-023-02067-9.
- F. Zhu, J. Gao, J. Yang, and N. Ye, “Neighborhood linear discriminant analysis,” Pattern Recognit., vol. 123, p. 108422, 2022.
- E. Hamouda, A. S. Abohamama, and M. Tarek, “Random Projection-Based Feature Transformation Using Metaheuristic Optimization Algorithm,” Arab. J. Sci. Eng., vol. 46, no. 9, pp. 8345–8353, 2021, doi: 10.1007/s13369-021-05474-1.
References
Y. Dian, S. Fang, Y. Le, Y. Xu, and C. Yao, “Comparison of the Different Classifiers in Vegetation Species Discrimination Using Hyperspectral Reflectance Data,” J. Indian Soc. Remote Sens., vol. 42, no. 1, pp. 61–72, 2014, doi: 10.1007/s12524-013-0309-9.
Y. Zhang, J. Ren, and J. Jiang, “Combining MLC and SVM classifiers for learning based decision making: Analysis and evaluations,” Comput. Intell. Neurosci., vol. 2015, 2015, doi: 10.1155/2015/423581.
C. M. Bishop, Pattern Recognition and Machine Learning, vol. 27, no. 1. 2004.
F. C. Pereira, A. M. Gonçalves, and M. Costa, “Outliers Impact on Parameter Estimation of Gaussian and Non-Gaussian State Space Models: A Simulation Study †,” Eng. Proc., vol. 18, no. 1, pp. 1–10, 2022, doi: 10.3390/engproc2022018031.
L. Li, H. Ge, J. Gao, Y. Zhang, Y. Tong, and J. Sun, “Method for Hyperspectral Image Feature Extraction,” Neural Process. Lett., pp. 515–542, 2020, doi: 10.1007/s11063-019-10101-0.
P. S. Sisodia, V. Tiwari, and A. Kumar, “Analysis of Supervised Maximum Likelihood Classification for remote sensing image,” Int. Conf. Recent Adv. Innov. Eng. ICRAIE 2014, pp. 9–12, 2014, doi: 10.1109/ICRAIE.2014.6909319.
N. Dahiya, S. Singh, and S. Gupta, “Comparative Analysis and Implication of Hyperion Hyperspectral and Landsat-8 Multispectral Dataset in Land Classification,” J. Indian Soc. Remote Sens., vol. 51, no. 11, pp. 2201–2213, 2023, doi: 10.1007/s12524-023-01760-7.
M. A. Sohl, S. A. Mahmood, and M. U. Rasheed, “Comparative performance of four machine learning models for land cover classification in a low-cost UAV ultra-high-resolution RGB-only orthomosaic,” Earth Sci. Informatics, 2024, doi: 10.1007/s12145-024-01318-2.
G. R. Liu, “Overfitting and Regularization,” Mach. Learn. with Python, no. 2, pp. 501–538, 2022, doi: 10.1142/9789811254185_0014.
K. Elkhalil, A. Kammoun, R. Couillet, T. Y. Al-Naffouri, and M.-S. Alouini, “A Large Dimensional Study of Regularized Discriminant Analysis,” IEEE Trans. Signal Process., vol. 68, pp. 2464–2479, 2020, doi: 10.1109/tsp.2020.2984160.
P. Zwiernik, C. Uhler, and D. Richards, “Maximum likelihood estimation for linear Gaussian covariance models,” J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 79, no. 4, pp. 1269–1292, 2017, doi: 10.1111/rssb.12217.
H. Nugroho, W. Widodo, and A. Rachman, “Pattern Recognition Bird Sounds Based on Their Type Using Discreate Cosine Transform (DCT) and Gaussian Methods,” Kinet. Game Technol. Inf. Syst. Comput. Network, Comput. Electron. Control, vol. 4, no. 3, pp. 233–240, 2019, doi: 10.22219/kinetik.v4i3.791.
N. Auguin, D. Morales-Jimenez, and M. McKay, “Large-Dimensional Characterization of Robust Linear Discriminant Analysis,” IEEE Trans. Signal Process., vol. 69, pp. 2625–2638, 2021, doi: 10.1109/TSP.2021.3075150.
K. K. Huang, D. Q. Dai, and C. X. Ren, “Regularized coplanar discriminant analysis for dimensionality reduction,” Pattern Recognit., vol. 62, pp. 87–98, 2017, doi: 10.1016/j.patcog.2016.08.024.
M. Kelly, R. Longjohn, and K. Nottingham, “The UCI Machine Learning Repository”, [Online]. Available: https://archive.ics.uci.edu
D. G. Z. Selcuk Korkmaz, “Package 'MVN": Multivariate normality tests.,” pp. 1–6, 2022, [Online]. Available: https://orcid.org/0000-0003-4632-6850
E. Gómez-Déniz, J. M. Sarabia, and E. Calderín-Ojeda, “Bimodal normal distribution: Extensions and applications,” J. Comput. Appl. Math., vol. 388, p. 113292, 2021, doi: 10.1016/j.cam.2020.113292.
A. Tharwat, T. Gaber, A. Ibrahim, and A. E. Hassanien, “Linear discriminant analysis: A detailed tutorial,” AI Commun., vol. 30, no. 2, pp. 169–190, 2017, doi: 10.3233/AIC-170729.
N. Rastin, M. Z. Jahromi, and M. Taheri, “A generalized weighted distance k-Nearest Neighbor for multi-label problems,” Pattern Recognit., vol. 114, p. 107526, 2021, doi: 10.1016/j.patcog.2020.107526.
A. Karatzoglou, D. Meyer, and K. Hornik, “Support Vector Algorithm in R,” J. Stat. Softw., vol. 15, no. 9, pp. 1–28, 2006, doi: 10.18637/jss.v081.b02.
J. Liu, X. Xiong, P. Ren, C. N. Li, and Y. H. Shao, “Capped norm linear discriminant analysis and its applications,” Appl. Intell., pp. 18488–18507, 2023, doi: 10.1007/s10489-022-04395-2.
C. N. Li, J. Liu, Y. Meng, and Y. H. Shao, “Recursive universum linear discriminant analysis,” Optim. Lett., no. 0123456789, 2023, doi: 10.1007/s11590-023-02067-9.
F. Zhu, J. Gao, J. Yang, and N. Ye, “Neighborhood linear discriminant analysis,” Pattern Recognit., vol. 123, p. 108422, 2022.
E. Hamouda, A. S. Abohamama, and M. Tarek, “Random Projection-Based Feature Transformation Using Metaheuristic Optimization Algorithm,” Arab. J. Sci. Eng., vol. 46, no. 9, pp. 8345–8353, 2021, doi: 10.1007/s13369-021-05474-1.