Issue

Vol. 10, No. 1, February 2025

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

An Ensemble Learning Layer for Wayang Recognition using CNN-based ResNet-50 and LSTM

https://doi.org/10.22219/kinetik.v10i1.2053
Candra Irawan
University of Dian Nuswantoro
Eko Hari Rachmawanto
University of Dian Nuswantoro
Heru Pramono Hadi
University of Dian Nuswantoro

Corresponding Author(s) : Candra Irawan

candra.irawan@dsn.dinus.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 10, No. 1, February 2025

Abstract

Wayang is commonly used to tell epic stories of Mahabharata and Ramayana, as well as local legends and myths. There are various types of wayang, such as wayang kulit (made of buffalo or goat leather), wayang golek (made of wood), and wayang klithik (combination of leather and wood). Although it indicates cultural richness, such diversity also makes it difficult for the general public to identify the character of wayang they are seeing because each type has unique characteristics and details. Recognizing   wayang characters is a challenging task due to their intricate designs and subtle variations. This research addresses this problem by leveraging machine learning technology, specifically CNN-based classification methods, to accurately identify wayang characters. This study proposed a novel method that integrates ResNet-50 transfer learning with LSTM, enhancing the model's ability to capture both spatial and sequential features of wayang images. The proposed model achieved an impressive accuracy of 97.92%, with precision, recall, and F1-scores all reaching 100%. Despite the extended training time of 188 minutes and 21 seconds, the results demonstrate the model's superior performance. This advancement can significantly aid in the preservation and educational dissemination of Indonesian cultural heritage. Future research can focus on optimizing the training process to reduce the time while maintaining or even improving the accuracy, potentially expanding the model's application scope and effectiveness.

Keywords

Wayang Classification CNN RestNet-50 LSTM
Irawan, C., Rachmawanto, E. H., & Hadi, H. P. (2025). An Ensemble Learning Layer for Wayang Recognition using CNN-based ResNet-50 and LSTM. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 10(1). https://doi.org/10.22219/kinetik.v10i1.2053
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. A. B. Wibawa, “Analyzing the Evolution of Indonesian Wayang Puppetry and Its Fusion with Modern Theater and Performance Arts,” Studies in Art and Architecture, vol. 3, no. 1, pp. 1–9, Mar. 2024. https://doi.org/10.56397/SAA.2024.03.01
  2. R. Siringo-ringo, A. Siagian, and N. Wahyuni, “The Resilient Tradition: Exploring the Cultural Significance of Javanese Wayang Kulit in Heritage Preservation,” Jurnal Ilmu Pendidikan dan Humaniora, vol. 11, no. 1, pp. 69–84, Jan. 2022. https://doi.org/10.35335/jiph.v11i1.16
  3. T. Nur Fitria, “The Performance of Wayang Orang Sriwedari Surakarta: A Cultural Preservation,” Jurnal Humaya: Jurnal Hukum, Humaniora, Masyarakat, dan Budaya, vol. 3, no. 2, pp. 123–138, Dec. 2023. https://doi.org/10.33830/humaya.v3i2.6276
  4. E. H. Rachmawanto, C. A. Sari, and F. O. Isinkaye, “A good result of brain tumor classification based on simple convolutional neural network architecture,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 22, no. 3, pp. 711–719, Jun. 2024. https://doi.org/10.12928/TELKOMNIKA.v22i3.25863
  5. A. Susanto, I. U. W. Mulyono, C. A. Sari, E. H. Rachmawanto, D. R. I. M. Setiadi, and M. K. Sarker, “Improved Javanese script recognition using custom model of convolution neural network,” International Journal of Electrical and Computer Engineering, vol. 13, no. 6, pp. 6629–6636, Dec. 2023. https://doi.org/10.11591/ijece.v13i6.pp6629-6636
  6. D. del-Pozo-Bueno, D. Kepaptsoglou, F. Peiró, and S. Estradé, “Comparative of machine learning classification strategies for electron energy loss spectroscopy: Support vector machines and artificial neural networks,” Ultramicroscopy, vol. 253, Nov. 2023. https://doi.org/10.1016/j.ultramic.2023.113828
  7. K. M. O. Nahar et al., “Recognition of Arabic Air-Written Letters: Machine Learning, Convolutional Neural Networks, and Optical Character Recognition (OCR) Techniques,” Preprints (Basel), Sep. 2023. https://doi.org/10.20944/preprints202309.1806.v1
  8. A. A. Khan, A. A. Laghari, and S. A. Awan, “Machine Learning in Computer Vision: A Review,” EAI Endorsed Transactions on Scalable Information Systems, vol. 8, no. 32, pp. 1–11, 2021. https://doi.org/10.4108/eai.21-4-2021.169418
  9. J. Gupta, S. Pathak, and G. Kumar, “Deep Learning (CNN) and Transfer Learning: A Review,” in Journal of Physics: Conference Series, Institute of Physics, 2022. https://doi.org/10.1088/1742-6596/2273/1/012029
  10. M. Ahmad, S. Abbas, A. Fatima, G. F. Issa, T. M. Ghazal, and M. A. Khan, “Deep Transfer Learning-Based Animal Face Identification Model Empowered with Vision-Based Hybrid Approach,” Applied Sciences (Switzerland), vol. 13, no. 2, Jan. 2023. https://doi.org/10.3390/app13021178
  11. A. P. Wibawa, W. A. Yudha Pratama, A. N. Handayani, and A. Ghosh, “Convolutional Neural Network (CNN) to determine the character of wayang kulit,” International Journal of Visual and Performing Arts, vol. 3, no. 1, pp. 1–8, Jun. 2021. https://doi.org/10.31763/viperarts.v3i1.373
  12. M. Banjaransari, A. Prahara, M. Banjaransari, and A. Prahara, “Image Classification of Wayang Using Transfer Learning and Fine-Tuning of CNN Models,” Buletin Ilmiah Sarjana Teknik Elektro, vol. 5, no. 4, pp. 632–641, 2023. https://doi.org/10.12928/biste.v5i4.9977
  13. I. B. K. Sudiatmika, M. Artana, N. W. Utami, M. A. P. Putra, and E. G. A. Dewi, “Mask R-CNN for Indonesian Shadow Puppet Recognition and Classification,” in Journal of Physics: Conference Series, IOP Publishing Ltd, Feb. 2021. https://doi.org/10.1088/1742-6596/1783/1/012032
  14. K. Wisnudhanti and F. Candra, “Image Classification of Pandawa Figures Using Convolutional Neural Network on Raspberry Pi 4,” in Journal of Physics: Conference Series, IOP Publishing Ltd, Nov. 2020. https://doi.org/10.1088/1742-6596/1655/1/012103
  15. “14 Indonesian Wayang Types Datasets.” Accessed: Jul. 04, 2024.
  16. N. R. D. Cahyo, C. A. Sari, E. H. Rachmawanto, C. Jatmoko, R. R. A. Al-Jawry, and M. A. Alkhafaji, “A Comparison of Multi Class Support Vector Machine vs Deep Convolutional Neural Network for Brain Tumor Classification,” in 2023 International Seminar on Application for Technology of Information and Communication (iSemantic), IEEE, Sep. 2023, pp. 358–363. https://doi.org/10.1109/iSemantic59612.2023.10295336
  17. M. M. I. Al-Ghiffary, C. A. Sari, E. H. Rachmawanto, N. M. Yacoob, N. R. D. Cahyo, and R. R. Ali, “Milkfish Freshness Classification Using Convolutional Neural Networks Based on Resnet50 Architecture,” Advance Sustainable Science Engineering and Technology, vol. 5, no. 3, p. 0230304, Oct. 2023. https://doi.org/10.26877/asset.v5i3.17017
  18. G. S. Nugraha, M. I. Darmawan, and R. Dwiyansaputra, “Comparison of CNN’s Architecture GoogleNet, AlexNet, VGG-16, Lenet -5, Resnet-50 in Arabic Handwriting Pattern Recognition,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, May 2023. https://doi.org/10.22219/kinetik.v8i2.1667
  19. A. Deshpande, V. V. Estrela, and P. Patavardhan, “The DCT-CNN-ResNet50 architecture to classify brain tumors with super-resolution, convolutional neural network, and the ResNet50,” Neuroscience Informatics, vol. 1, no. 4, p. 100013, Dec. 2021. https://doi.org/10.1016/j.neuri.2021.100013
  20. M. M. I. Al-Ghiffary, N. R. D. Cahyo, E. H. Rachmawanto, C. Irawan, and N. Hendriyanto, “Adaptive deep learning based on FaceNet convolutional neural network for facial expression recognition,” Journal of Soft Computing, vol. 05, no. 03, pp. 271–280, 2024. https://doi.org/10.52465/joscex.v5i3.450
  21. N. R. D. Cahyo and M. M. I. Al-Ghiffary, “An Image Processing Study: Image Enhancement, Image Segmentation, and Image Classification using Milkfish Freshness Images,” IJECAR) International Journal of Engineering Computing Advanced Research, vol. 1, no. 1, pp. 11–22, 2024.
  22. X. X. Li, D. Li, W. X. Ren, and J. S. Zhang, “Loosening Identification of Multi-Bolt Connections Based on Wavelet Transform and ResNet-50 Convolutional Neural Network,” Sensors, vol. 22, no. 18, Sep. 2022. https://doi.org/10.3390/s22186825
  23. F. Farhan, C. A. Sari, E. H. Rachmawanto, and N. R. D. Cahyo, “Mangrove Tree Species Classification Based on Leaf, Stem, and Seed Characteristics Using Convolutional Neural Networks with K-Folds Cross Validation Optimalization,” Advance Sustainable Science Engineering and Technology, vol. 5, no. 3, p. 02303011, Oct. 2023. https://doi.org/10.26877/asset.v5i3.17188
  24. A. K. Ozcanli and M. Baysal, “Islanding detection in microgrid using deep learning based on 1D CNN and CNN-LSTM networks,” Sustainable Energy, Grids and Networks, vol. 32, p. 100839, 2022. https://doi.org/10.1016/j.segan.2022.100839
  25. P. Kumari and D. Toshniwal, “Long short term memory–convolutional neural network based deep hybrid approach for solar irradiance forecasting,” Appl Energy, vol. 295, p. 117061, 2021. https://doi.org/10.1016/j.apenergy.2021.117061
  26. I. P. Kamila, C. A. Sari, E. H. Rachmawanto, and N. R. D. Cahyo, “A Good Evaluation Based on Confusion Matrix for Lung Diseases Classification using Convolutional Neural Networks,” Advance Sustainable Science, Engineering and Technology, vol. 6, no. 1, p. 0240102, Dec. 2023. https://doi.org/10.26877/asset.v6i1.17330
  27. S. C. Kim and Y. S. Cho, “Predictive System Implementation to Improve the Accuracy of Urine Self-Diagnosis with Smartphones: Application of a Confusion Matrix-Based Learning Model through RGB Semiquantitative Analysis,” Sensors, vol. 22, no. 14, Jul. 2022. https://doi.org/10.3390/s22145445
  28. A. Theissler, M. Thomas, M. Burch, and F. Gerschner, “ConfusionVis: Comparative evaluation and selection of multi-class classifiers based on confusion matrices,” Knowl Based Syst, vol. 247, Jul. 2022. https://doi.org/10.1016/j.knosys.2022.108651
Read More

Author Biographies

Eko Hari Rachmawanto, University of Dian Nuswantoro

https://orcid.org/0000-0001-9984-0592

Scopus Author ID: 57194878429

Heru Pramono Hadi, University of Dian Nuswantoro

https://scholar.google.com/citations?user=WxH4etAAAAAJ&hl=en&oi=ao 

Download this PDF file
PDF
Statistic
Read Counter : 0 Download : 0

Downloads

Download data is not yet available.