Analyzing the Quality of Game-based Assessment Design in Basic Arithmetic Operations

M. Naufal Azzmi. H.; Umi Laili Yuhana; Nawang Sulistyani; Lailatul Husniah

doi:10.22219/kinetik.v8i1.1599

Issue

Vol. 8, No. 1, February 2023

Issue Published : Feb 28, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Analyzing the Quality of Game-based Assessment Design in Basic Arithmetic Operations

https://doi.org/10.22219/kinetik.v8i1.1599

M. Naufal Azzmi. H.

Institut Teknologi Sepuluh Nopember

Umi Laili Yuhana

Institut Teknologi Sepuluh Nopember Surabaya

Nawang Sulistyani

Universitas Muhammadiyah Malang

Lailatul Husniah

Universitas Muhammadiyah Malang

Corresponding Author(s) : Umi Laili Yuhana

yuhana@if.its.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 8, No. 1, February 2023
Article Published : Feb 28, 2023

Abstract

The use of Serious Games in education has experienced rapid development. However, not all studies have been able to show evidence that game-based methods are superior to other methods. It is important to analyze the quality of game designs used in learning and assessment used to assess students. This study focuses on how to design a serious game called B-block used in the assessment. The researcher validates the quality of the serious game first before conducting a pilot study regarding the feasibility of the Serious Game given for assessment. The game focuses on basic arithmetic, including addition, subtraction, division, and multiplication, and involves positive and negative numbers. The study was conducted in one of the schools in Indonesia and given to 35 students with an age range of 11-12 years with different student backgrounds in their experience with game-based exams. Based on these results, 85.7% of respondents agreed that this game could be used as a substitute for paper-based exams, with the analysis of game design quality having an average value of 78% pedagogic specifications and 73% playful and 80% technical specifications. Thus, the average value of this game quality analysis is considered superior and meets almost all the specifications needed for assessment. We also argue that serious game is closely related to how game design meets specifications for use as educational tools.

Keywords

Seriuous Game Game Based Assessment Game Design Game Design Quality Arithmetic Game

Azzmi. H., M. N., Yuhana, U. L., Sulistyani, N., & Husniah, L. (2023). Analyzing the Quality of Game-based Assessment Design in Basic Arithmetic Operations. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 8(1), 453-460. https://doi.org/10.22219/kinetik.v8i1.1599

Download Citation

References

T. Jordaan, M. Havenga, and B. Bunt, “Mathematical game-based learning: education students’ collaboration and on-line experiences during disrupted Covid-19 circumstances,” World Transactions on Engineering and Technology Education, pp. 263–270, 2021.
J. Luhan, V. Novotna, and J. Kriz, “ICT support for creative teaching of mathematic disciplines,” Interdisciplinary Studies Journal, vol. 2, no. 3, p. 89, 2013.
M. Taub, R. Sawyer, A. Smith, J. Rowe, R. Azevedo, and J. Lester, “The agency effect: The impact of student agency on learning, emotions, and problem-solving behaviors in a game-based learning environment,” Comput Educ, vol. 147, p. 103781, 2020. https://doi.org/10.1016/j.compedu.2019.103781
S. D. Putra and V. Yasin, “MDA framework approach for gamification-based elementary mathematics learning design,” International Journal of Engineering, Science and Information Technology, vol. 1, no. 3, pp. 35–39, 2021. https://doi.org/10.52088/ijesty.v1i3.83
A. Arsyad, “Media pembelajaran, edisi 1,” Jakarta: PT. Raja Grafindo Persada, vol. 36, 2002.
U. Tokac, E. Novak, and C. G. Thompson, “Effects of game‐based learning on students’ mathematics achievement: A meta‐analysis,” J Comput Assist Learn, vol. 35, no. 3, pp. 407–420, 2019. https://doi.org/10.1111/jcal.12347
J. Zhao, K. Zhou, and Y. Ding, “Digital games-based learning pedagogy enhances the quality of medical education: a systematic review and meta-analysis,” The Asia-Pacific Education Researcher, pp. 1–12, 2021. https://doi.org/10.1007/s40299-021-00587-5
S. Y. Cheung and K. Y. Ng, “Application of the educational game to enhance student learning,” in Frontiers in Education, 2021, vol. 6, p. 623793. https://doi.org/10.3389/feduc.2021.623793
D. A. Amanda, “Pengembangan Game Edukasi Pada Mata Pelajaran Matematika Materi Bangun Datar Berbasis Android di SDN 1 Jepun,” JoEICT (Journal of Education And ICT), vol. 3, no. 2, 2019. https://doi.org/10.29100/joeict.v3i2.1241
V. Shute, F. Ke, and L. Wang, “Assessment and adaptation in games,” in Instructional techniques to facilitate learning and motivation of serious games, Springer, 2017, pp. 59–78. https://doi.org/10.1007/978-3-319-39298-1_4
C. Wang and L. Huang, “A Systematic Review of Serious Games for Collaborative Learning: Theoretical Framework, Game Mechanic and Efficiency Assessment.,” International Journal of Emerging Technologies in Learning, vol. 16, no. 6, 2021. https://doi.org/10.3991/ijet.v16i06.18495
M. J. Gomez, J. A. Ruipérez-Valiente, and F. J. G. Clemente, “A systematic literature review of digital game-based assessment empirical studies: Current trends and open challenges,” arXiv preprint arXiv:2207.07369, 2022. https://doi.org/10.48550/arXiv.2207.07369
N. N. Omar, “The efffectiveness of Kahoot application towards students’ good feedback practice,” PEOPLE: International Journal of Social Sciences, vol. 3, no. 2, pp. 2551–2562, 2017. https://doi.org/10.20319/pijss.2017.32.25512562
D. Grier, S. F. Lindt, and S. C. Miller, “Formative assessment with game-based technology,” International Journal of Technology in Education and Science (IJTES), vol. 5, no. 2, pp. 193–202, 2021. https://doi.org/10.46328/ijtes.97
H. DURAK and M. Saritepeci, “Investigating the effect of technology use in education on classroom management within the scope of the FATİH project,” Cukurova university faculty of education journal, vol. 46, no. 2, pp. 441–457, 2017. https://doi.org/10.14812/CUEFD.303511
K. Kiili, K. Devlin, A. Perttula, P. Tuomi, and A. Lindstedt, “Using video games to combine learning and assessment in mathematics education,” International Journal of Serious Games, vol. 2, no. 4, pp. 37–55, 2015. https://doi.org/10.17083/ijsg.v2i4.98
H. Bai, W. Pan, A. Hirumi, and M. Kebritchi, “Assessing the effectiveness of a 3‐D instructional game on improving mathematics achievement and motivation of middle school students,” British Journal of Educational Technology, vol. 43, no. 6, pp. 993–1003, 2012. https://doi.org/10.1111/j.1467-8535.2011.01269.x
S. Bugden, E. Szkudlarek, and E. M. Brannon, “Approximate arithmetic training does not improve symbolic math in third and fourth grade children,” Trends Neurosci Educ, vol. 22, p. 100149, 2021. https://doi.org/10.1016/j.tine.2021.100149
Y.-K. Wong, C.-F. Wu, and Y.-H. Tu, “Effectiveness of a Serious Game Design and Game Mechanic Factors for Attention and Executive Function Improvement in the Elderly: A Pretest-Posttest Study,” Applied Sciences, vol. 12, no. 14, p. 6923, 2022. https://doi.org/10.3390/app12146923
T. M. Joo and W. Y. Seng, “Validating the effectiveness of game-based learning approach in the form of video game for assessing computational thinking,” Journal of ICT in Education, vol. 8, no. 4, pp. 1–12, 2021. https://doi.org/10.37134/jictie.vol8.sp.2.1.2021
Y. Zhonggen, “A meta-analysis of use of serious games in education over a decade,” International Journal of Computer Games Technology, vol. 2019, 2019. https://doi.org/10.1155/2019/4797032
A. All, E. N. P. Castellar, and J. van Looy, “Digital Game-Based Learning effectiveness assessment: Reflections on study design,” Comput Educ, vol. 167, p. 104160, 2021. https://doi.org/10.1016/j.compedu.2021.104160
F. G. M. Silva, “Practical methodology for the design of educational serious games,” Information, vol. 11, no. 1, p. 14, 2019. https://doi.org/10.3390/info11010014
R. Hunicke, M. LeBlanc, and R. Zubek, “MDA: A formal approach to game design and game research,” in Proceedings of the AAAI Workshop on Challenges in Game AI, 2004, vol. 4, no. 1, p. 1722.
B. M. Winn, “The design, play, and experience framework,” in Handbook of research on effective electronic gaming in education, IGI Global, 2009, pp. 1010–1024. https://doi.org/10.4018/978-1-59904-808-6.CH058
J. Sherry and A. Pacheco, “Matching computer game genres to educational outcomes,” Electronic Journal of Communication, vol. 16, no. 1, p. 2, 2006.
N. Kara, “A systematic review of the use of serious games in science education,” Contemp Educ Technol, vol. 13, no. 2, p. ep295, 2021. https://doi.org/10.1016/j.vrih.2022.02.001
Y. el Borji and M. Khaldi, “Comparative study to develop a tool for the quality assessment of serious games intended to be used in education,” International Journal of Emerging Technologies in Learning (Online), vol. 9, no. 9, p. 50, 2014. https://doi.org/10.3991/ijet.v9i9.4150
Y. el Borji and M. Khaldi, “An IEEE LOM application profile to describe serious games «SG-LOM»,” Int J Comput Appl, vol. 86, no. 13, 2014. https://doi.org/10.5120/15042-3404

References

T. Jordaan, M. Havenga, and B. Bunt, “Mathematical game-based learning: education students’ collaboration and on-line experiences during disrupted Covid-19 circumstances,” World Transactions on Engineering and Technology Education, pp. 263–270, 2021.

J. Luhan, V. Novotna, and J. Kriz, “ICT support for creative teaching of mathematic disciplines,” Interdisciplinary Studies Journal, vol. 2, no. 3, p. 89, 2013.

M. Taub, R. Sawyer, A. Smith, J. Rowe, R. Azevedo, and J. Lester, “The agency effect: The impact of student agency on learning, emotions, and problem-solving behaviors in a game-based learning environment,” Comput Educ, vol. 147, p. 103781, 2020. https://doi.org/10.1016/j.compedu.2019.103781

S. D. Putra and V. Yasin, “MDA framework approach for gamification-based elementary mathematics learning design,” International Journal of Engineering, Science and Information Technology, vol. 1, no. 3, pp. 35–39, 2021. https://doi.org/10.52088/ijesty.v1i3.83

A. Arsyad, “Media pembelajaran, edisi 1,” Jakarta: PT. Raja Grafindo Persada, vol. 36, 2002.

U. Tokac, E. Novak, and C. G. Thompson, “Effects of game‐based learning on students’ mathematics achievement: A meta‐analysis,” J Comput Assist Learn, vol. 35, no. 3, pp. 407–420, 2019. https://doi.org/10.1111/jcal.12347

J. Zhao, K. Zhou, and Y. Ding, “Digital games-based learning pedagogy enhances the quality of medical education: a systematic review and meta-analysis,” The Asia-Pacific Education Researcher, pp. 1–12, 2021. https://doi.org/10.1007/s40299-021-00587-5

S. Y. Cheung and K. Y. Ng, “Application of the educational game to enhance student learning,” in Frontiers in Education, 2021, vol. 6, p. 623793. https://doi.org/10.3389/feduc.2021.623793

D. A. Amanda, “Pengembangan Game Edukasi Pada Mata Pelajaran Matematika Materi Bangun Datar Berbasis Android di SDN 1 Jepun,” JoEICT (Journal of Education And ICT), vol. 3, no. 2, 2019. https://doi.org/10.29100/joeict.v3i2.1241

V. Shute, F. Ke, and L. Wang, “Assessment and adaptation in games,” in Instructional techniques to facilitate learning and motivation of serious games, Springer, 2017, pp. 59–78. https://doi.org/10.1007/978-3-319-39298-1_4

C. Wang and L. Huang, “A Systematic Review of Serious Games for Collaborative Learning: Theoretical Framework, Game Mechanic and Efficiency Assessment.,” International Journal of Emerging Technologies in Learning, vol. 16, no. 6, 2021. https://doi.org/10.3991/ijet.v16i06.18495

M. J. Gomez, J. A. Ruipérez-Valiente, and F. J. G. Clemente, “A systematic literature review of digital game-based assessment empirical studies: Current trends and open challenges,” arXiv preprint arXiv:2207.07369, 2022. https://doi.org/10.48550/arXiv.2207.07369

N. N. Omar, “The efffectiveness of Kahoot application towards students’ good feedback practice,” PEOPLE: International Journal of Social Sciences, vol. 3, no. 2, pp. 2551–2562, 2017. https://doi.org/10.20319/pijss.2017.32.25512562

D. Grier, S. F. Lindt, and S. C. Miller, “Formative assessment with game-based technology,” International Journal of Technology in Education and Science (IJTES), vol. 5, no. 2, pp. 193–202, 2021. https://doi.org/10.46328/ijtes.97

H. DURAK and M. Saritepeci, “Investigating the effect of technology use in education on classroom management within the scope of the FATİH project,” Cukurova university faculty of education journal, vol. 46, no. 2, pp. 441–457, 2017. https://doi.org/10.14812/CUEFD.303511

K. Kiili, K. Devlin, A. Perttula, P. Tuomi, and A. Lindstedt, “Using video games to combine learning and assessment in mathematics education,” International Journal of Serious Games, vol. 2, no. 4, pp. 37–55, 2015. https://doi.org/10.17083/ijsg.v2i4.98

H. Bai, W. Pan, A. Hirumi, and M. Kebritchi, “Assessing the effectiveness of a 3‐D instructional game on improving mathematics achievement and motivation of middle school students,” British Journal of Educational Technology, vol. 43, no. 6, pp. 993–1003, 2012. https://doi.org/10.1111/j.1467-8535.2011.01269.x

S. Bugden, E. Szkudlarek, and E. M. Brannon, “Approximate arithmetic training does not improve symbolic math in third and fourth grade children,” Trends Neurosci Educ, vol. 22, p. 100149, 2021. https://doi.org/10.1016/j.tine.2021.100149

Y.-K. Wong, C.-F. Wu, and Y.-H. Tu, “Effectiveness of a Serious Game Design and Game Mechanic Factors for Attention and Executive Function Improvement in the Elderly: A Pretest-Posttest Study,” Applied Sciences, vol. 12, no. 14, p. 6923, 2022. https://doi.org/10.3390/app12146923

T. M. Joo and W. Y. Seng, “Validating the effectiveness of game-based learning approach in the form of video game for assessing computational thinking,” Journal of ICT in Education, vol. 8, no. 4, pp. 1–12, 2021. https://doi.org/10.37134/jictie.vol8.sp.2.1.2021

Y. Zhonggen, “A meta-analysis of use of serious games in education over a decade,” International Journal of Computer Games Technology, vol. 2019, 2019. https://doi.org/10.1155/2019/4797032

A. All, E. N. P. Castellar, and J. van Looy, “Digital Game-Based Learning effectiveness assessment: Reflections on study design,” Comput Educ, vol. 167, p. 104160, 2021. https://doi.org/10.1016/j.compedu.2021.104160

F. G. M. Silva, “Practical methodology for the design of educational serious games,” Information, vol. 11, no. 1, p. 14, 2019. https://doi.org/10.3390/info11010014

R. Hunicke, M. LeBlanc, and R. Zubek, “MDA: A formal approach to game design and game research,” in Proceedings of the AAAI Workshop on Challenges in Game AI, 2004, vol. 4, no. 1, p. 1722.

B. M. Winn, “The design, play, and experience framework,” in Handbook of research on effective electronic gaming in education, IGI Global, 2009, pp. 1010–1024. https://doi.org/10.4018/978-1-59904-808-6.CH058

J. Sherry and A. Pacheco, “Matching computer game genres to educational outcomes,” Electronic Journal of Communication, vol. 16, no. 1, p. 2, 2006.

N. Kara, “A systematic review of the use of serious games in science education,” Contemp Educ Technol, vol. 13, no. 2, p. ep295, 2021. https://doi.org/10.1016/j.vrih.2022.02.001

Y. el Borji and M. Khaldi, “Comparative study to develop a tool for the quality assessment of serious games intended to be used in education,” International Journal of Emerging Technologies in Learning (Online), vol. 9, no. 9, p. 50, 2014. https://doi.org/10.3991/ijet.v9i9.4150

Y. el Borji and M. Khaldi, “An IEEE LOM application profile to describe serious games «SG-LOM»,” Int J Comput Appl, vol. 86, no. 13, 2014. https://doi.org/10.5120/15042-3404

Issue

Vol. 8, No. 1, February 2023

Analyzing the Quality of Game-based Assessment Design in Basic Arithmetic Operations

Corresponding Author(s) : Umi Laili Yuhana

Abstract

Keywords

Download Citation

References

Downloads