This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Utilization of AR Technology for The Development of Speech Therapy Applications by Optimizing Marked-Based Tracking Method
Corresponding Author(s) : Linda Perdana Wanti
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control,
Vol. 8, No. 1, February 2023
Abstract
Autism is a developmental disorder that attacks children mentally and causes difficulties in interacting with the social environment. Therapy that can be done to people with autism to deal with communication disorders is speech therapy. Application usage health or better known as mobile health is easy and fast to help users in accessing information about various health problems, one of which is child development or better known as an autism spectrum disorder (ASD) which was developed using augmented reality (AR) technology. The purpose of the study is to optimizing the marked-based tracking method to augmented reality technology for speech therapy tutorials for children with autism. The results obtained from this study are the SELPY application (Self Autism Therapy) mobile-based speech therapy for people with autism which is a product of the application of appropriate technology in the field of information technology, especially in the health sector. The marked-based tracking method has been successfully implemented in the development of speech therapy AR applications for children with autism spectrum disorder (ASD). This is by the results of the tests that have been carried out, namely distance testing and angle testing. The most ideal distance to detect marker/image targets is 40cm to 50cm with a smartphone tilt angle of 200 to 300.
Keywords
Download Citation
Endnote/Zotero/Mendeley (RIS)BibTeX
- A. Syawaludin, Gunarhadi, and P. Rintayati, “Development of Augmented Reality-Based Interactive Multimedia to Improve Critical Thinking Skills in Science Learning.,” Int. J. Instr., vol. 12, no. 4, pp. 331–344, Oct. 2019, doi: 10.29333/iji.2019.12421a.
- O. P. Leonardo, F. E. Hardianto, and R. K. Budhi, “Efektifitas Pemanfaatan Aplikasi Berbasis Augmented Reality untuk Anak Berkebutuhan Khusus,” Pros. Semin. Nas. Pendidik. Tek. Inform., no. ISSN 2087-2658, pp. 20–25, 2018.
- A. Rega, A. Mennitto, S. Vita, and L. Iovino, “New Technologies and Autism: Can Augmented Reality (Ar) Increase the Motivation in Children With Autism?,” INTED2018 Proc., vol. 1, no. March, pp. 4904–4910, 2018, doi: 10.21125/inted.2018.0959.
- I. J. Lee, “Kinect-for-windows with augmented reality in an interactive roleplay system for children with an autism spectrum disorder,” Interact. Learn. Environ., vol. 29, no. 4, pp. 688–704, 2021, doi: 10.1080/10494820.2019.1710851.
- B. M. DeJesus, R. C. Oliveira, F. O. de Carvalho, J. de Jesus Mari, R. M. Arida, and L. Teixeira-Machado, “Dance promotes positive benefits for negative symptoms in autism spectrum disorder (ASD): A systematic review,” Complement. Ther. Med., vol. 49, no. September 2019, 2020, doi: 10.1016/j.ctim.2020.102299.
- E. A. Wise, “Aging in Autism Spectrum Disorder,” Am. J. Geriatr. Psychiatry, 2019, doi: 10.1016/j.jagp.2019.12.001.
- Z. Javadfar et al., “Effects of vitamin D supplementation on core symptoms, serum serotonin, and interleukin-6 in children with autism spectrum disorders: A randomized clinical trial,” Nutrition, vol. 79–80, p. 110986, 2020, doi: 10.1016/j.nut.2020.110986.
- J. A. Andoy Galvan et al., “Mode of delivery, order of birth, parental age gap and autism spectrum disorder among Malaysian children: A case-control study,” Heliyon, vol. 6, no. 10, p. e05068, 2020, doi: 10.1016/j.heliyon.2020.e05068.
- B. Eskandari, H. Pouretemad, M. Mousavi, and H. Farahani, “Common elements of parent management training programs for preschool children with autism spectrum disorder,” Asian J. Psychiatr., vol. 52, no. November 2019, p. 102149, 2020, doi: 10.1016/j.ajp.2020.102149.
- E. Puerto, J. Aguilar, C. López, and D. Chávez, “Using Multilayer Fuzzy Cognitive Maps to diagnose Autism Spectrum Disorder,” Appl. Soft Comput. J., vol. 75, pp. 58–71, 2019, doi: 10.1016/j.asoc.2018.10.034.
- B. Razi, D. Imani, M. Hassanzadeh Makoui, R. Rezaei, and S. Aslani, “Association between MTHFR gene polymorphism and susceptibility to autism spectrum disorders: Systematic review and meta-analysis,” Res. Autism Spectr. Disord., vol. 70, no. October 2019, p. 101473, 2020, doi: 10.1016/j.rasd.2019.101473.
- R. Grzadzinski, D. Janvier, and S. H. Kim, “Recent Developments in Treatment Outcome Measures for Young Children With Autism Spectrum Disorder (ASD),” Semin. Pediatr. Neurol., vol. 34, no. Cidd, p. 100806, 2020, doi: 10.1016/j.spen.2020.100806.
- J. A. Cascia and J. J. Barr, “Associations among parent and teacher ratings of systemizing, vocabulary and executive function in children with autism spectrum disorder,” Res. Dev. Disabil., vol. 106, no. July, p. 103779, 2020, doi: 10.1016/j.ridd.2020.103779.
- L. de Vries, I. Fouquaet, B. Boets, G. Naulaers, and J. Steyaert, “Autism spectrum disorder and pupillometry: A systematic review and meta-analysis,” Neurosci. Biobehav. Rev., vol. 120, pp. 479–508, 2021, doi: 10.1016/j.neubiorev.2020.09.032.
- K. Khowaja et al., “Augmented reality for learning of children and adolescents with autism spectrum disorder (ASD): A systematic review,” IEEE Access, vol. 8, pp. 78779–78807, 2020, doi: 10.1109/ACCESS.2020.2986608.
- M. F. Syahputra, D. Arisandi, A. F. Lumbanbatu, L. F. Kemit, E. B. Nababan, and O. Sheta, “Augmented reality social story for autism spectrum disorder,” J. Phys. Conf. Ser., vol. 978, no. 1, 2018, doi: 10.1088/1742-6596/978/1/012040.
- R. Liu, J. P. Salisbury, A. Vahabzadeh, and N. T. Sahin, “Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching,” Front. Pediatr., vol. 5, no. June, pp. 1–8, 2017, doi: 10.3389/fped.2017.00145.
- A. Bilesan et al., “Marker-based motion tracking using Microsoft Kinect,” IFAC-PapersOnLine, vol. 51, no. 22, pp. 399–404, 2018, doi: 10.1016/j.ifacol.2018.11.575.
- N. H. Adnan, I. Ahmad, and N. Abdullasim, “Systematic review on augmented reality application for autism children,” J. Adv. Res. Dyn. Control Syst., vol. 10, no. 11, pp. 26–32, 2018.
- A. Rohacz, S. Weißenfels, and S. Strassburger, “Concept for the comparison of intralogistics designs with real factory layout using augmented reality, SLAM and marker-based tracking,” Procedia CIRP, vol. 93, pp. 341–346, 2020, doi: 10.1016/j.procir.2020.03.039.
- N. Nurrisma, R. Munadi, S. Syahrial, and E. D. Meutia, “Perancangan Augmented Reality dengan Metode Marker Card Detection dalam Pengenalan Karakter Korea,” Inform. Mulawarman J. Ilm. Ilmu Komput., vol. 16, no. 1, p. 34, 2021, doi: 10.30872/jim.v16i1.5152.
- S. D. Y. Kusuma, “Perancangan Aplikasi Augmented Reality Pembelajaran Tata Surya dengan Menggunakan Marker Based Tracking,” J. Inform. Univ. Pamulang, vol. 3, no. 1, p. 33, 2018, doi: 10.32493/informatika.v3i1.1428.
- B. Marques, B. S. Santos, T. Araújo, N. C. Martins, J. Alves, and P. Dias, “Situated Visualization in the Decision Process through Augmented Reality,” Proc. Int. Conf. Inf. Vis., vol. 2019-July, no. Iv, pp. 13–18, 2019, doi: 10.1109/IV.2019.00012.
- M. Mirbabaie and J. Fromm, “REDUCING THE COGNITIVE LOAD OF DECISION-MAKERS IN EMERGENCY MANAGEMENT THROUGH AUGMENTED REALITY,” in ECIS 2019 Proceedings, 2019, pp. 0–11.
- V. Sangiorgio, S. Martiradonna, F. Fatiguso, and I. Lombillo, “Augmented reality based - decision making (AR-DM) to support multi-criteria analysis in constructions,” Autom. Constr., vol. 124, no. May 2020, p. 103567, 2021, doi: 10.1016/j.autcon.2021.103567.
- M. Lorenz, S. Knopp, and P. Klimant, “Industrial Augmented Reality: Requirements for an Augmented Reality Maintenance Worker Support System,” Adjun. Proc. - 2018 IEEE Int. Symp. Mix. Augment. Reality, ISMAR-Adjunct 2018, pp. 151–153, 2018, doi: 10.1109/ISMAR-Adjunct.2018.00055.
- Z. C. Rawis, V. Tulenan, and B. A. Sugiarso, “Penerapan Augmented Reality Berbasis Android Untuk Mengenalkan Pakaian Adat Tountemboan,” J. Tek. Inform., vol. 13, no. 1, Feb. 2018.
- M. A. Nazaruddin and M. Efendi, “The Book of Pop Up Augmented Reality to Increase Focus and Object Recognition Capabilities for Children with Autism,” J. ICSAR, vol. 2, no. 1, pp. 9–14, 2018, doi: 10.17977/um005v2i12018p009.
- E. Pantano, A. Rese, and D. Baier, “Enhancing the online decision-making process by using augmented reality: A two country comparison of youth markets,” J. Retail. Consum. Serv., vol. 38, no. May, pp. 81–95, 2017, doi: 10.1016/j.jretconser.2017.05.011.
- C. Berenguer, I. Baixauli, S. Gómez, M. de E. P. Andrés, and S. De Stasio, “Exploring the Impact of Augmented Reality in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review,” Int. J. Environ. Res. Public Health, vol. 17, no. 17, pp. 1–15, 2020, doi: 10.3390/ijerph17176143.
- E. Agustina Julisawaty et al., “Aplikasi Visualisasi Penyakit Jantung Manusia Berbasis Augmented Reality Menggunakan Unity,” Pros. Semnastek, vol. 0, no. 0, Dec. 2017.
References
A. Syawaludin, Gunarhadi, and P. Rintayati, “Development of Augmented Reality-Based Interactive Multimedia to Improve Critical Thinking Skills in Science Learning.,” Int. J. Instr., vol. 12, no. 4, pp. 331–344, Oct. 2019, doi: 10.29333/iji.2019.12421a.
O. P. Leonardo, F. E. Hardianto, and R. K. Budhi, “Efektifitas Pemanfaatan Aplikasi Berbasis Augmented Reality untuk Anak Berkebutuhan Khusus,” Pros. Semin. Nas. Pendidik. Tek. Inform., no. ISSN 2087-2658, pp. 20–25, 2018.
A. Rega, A. Mennitto, S. Vita, and L. Iovino, “New Technologies and Autism: Can Augmented Reality (Ar) Increase the Motivation in Children With Autism?,” INTED2018 Proc., vol. 1, no. March, pp. 4904–4910, 2018, doi: 10.21125/inted.2018.0959.
I. J. Lee, “Kinect-for-windows with augmented reality in an interactive roleplay system for children with an autism spectrum disorder,” Interact. Learn. Environ., vol. 29, no. 4, pp. 688–704, 2021, doi: 10.1080/10494820.2019.1710851.
B. M. DeJesus, R. C. Oliveira, F. O. de Carvalho, J. de Jesus Mari, R. M. Arida, and L. Teixeira-Machado, “Dance promotes positive benefits for negative symptoms in autism spectrum disorder (ASD): A systematic review,” Complement. Ther. Med., vol. 49, no. September 2019, 2020, doi: 10.1016/j.ctim.2020.102299.
E. A. Wise, “Aging in Autism Spectrum Disorder,” Am. J. Geriatr. Psychiatry, 2019, doi: 10.1016/j.jagp.2019.12.001.
Z. Javadfar et al., “Effects of vitamin D supplementation on core symptoms, serum serotonin, and interleukin-6 in children with autism spectrum disorders: A randomized clinical trial,” Nutrition, vol. 79–80, p. 110986, 2020, doi: 10.1016/j.nut.2020.110986.
J. A. Andoy Galvan et al., “Mode of delivery, order of birth, parental age gap and autism spectrum disorder among Malaysian children: A case-control study,” Heliyon, vol. 6, no. 10, p. e05068, 2020, doi: 10.1016/j.heliyon.2020.e05068.
B. Eskandari, H. Pouretemad, M. Mousavi, and H. Farahani, “Common elements of parent management training programs for preschool children with autism spectrum disorder,” Asian J. Psychiatr., vol. 52, no. November 2019, p. 102149, 2020, doi: 10.1016/j.ajp.2020.102149.
E. Puerto, J. Aguilar, C. López, and D. Chávez, “Using Multilayer Fuzzy Cognitive Maps to diagnose Autism Spectrum Disorder,” Appl. Soft Comput. J., vol. 75, pp. 58–71, 2019, doi: 10.1016/j.asoc.2018.10.034.
B. Razi, D. Imani, M. Hassanzadeh Makoui, R. Rezaei, and S. Aslani, “Association between MTHFR gene polymorphism and susceptibility to autism spectrum disorders: Systematic review and meta-analysis,” Res. Autism Spectr. Disord., vol. 70, no. October 2019, p. 101473, 2020, doi: 10.1016/j.rasd.2019.101473.
R. Grzadzinski, D. Janvier, and S. H. Kim, “Recent Developments in Treatment Outcome Measures for Young Children With Autism Spectrum Disorder (ASD),” Semin. Pediatr. Neurol., vol. 34, no. Cidd, p. 100806, 2020, doi: 10.1016/j.spen.2020.100806.
J. A. Cascia and J. J. Barr, “Associations among parent and teacher ratings of systemizing, vocabulary and executive function in children with autism spectrum disorder,” Res. Dev. Disabil., vol. 106, no. July, p. 103779, 2020, doi: 10.1016/j.ridd.2020.103779.
L. de Vries, I. Fouquaet, B. Boets, G. Naulaers, and J. Steyaert, “Autism spectrum disorder and pupillometry: A systematic review and meta-analysis,” Neurosci. Biobehav. Rev., vol. 120, pp. 479–508, 2021, doi: 10.1016/j.neubiorev.2020.09.032.
K. Khowaja et al., “Augmented reality for learning of children and adolescents with autism spectrum disorder (ASD): A systematic review,” IEEE Access, vol. 8, pp. 78779–78807, 2020, doi: 10.1109/ACCESS.2020.2986608.
M. F. Syahputra, D. Arisandi, A. F. Lumbanbatu, L. F. Kemit, E. B. Nababan, and O. Sheta, “Augmented reality social story for autism spectrum disorder,” J. Phys. Conf. Ser., vol. 978, no. 1, 2018, doi: 10.1088/1742-6596/978/1/012040.
R. Liu, J. P. Salisbury, A. Vahabzadeh, and N. T. Sahin, “Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching,” Front. Pediatr., vol. 5, no. June, pp. 1–8, 2017, doi: 10.3389/fped.2017.00145.
A. Bilesan et al., “Marker-based motion tracking using Microsoft Kinect,” IFAC-PapersOnLine, vol. 51, no. 22, pp. 399–404, 2018, doi: 10.1016/j.ifacol.2018.11.575.
N. H. Adnan, I. Ahmad, and N. Abdullasim, “Systematic review on augmented reality application for autism children,” J. Adv. Res. Dyn. Control Syst., vol. 10, no. 11, pp. 26–32, 2018.
A. Rohacz, S. Weißenfels, and S. Strassburger, “Concept for the comparison of intralogistics designs with real factory layout using augmented reality, SLAM and marker-based tracking,” Procedia CIRP, vol. 93, pp. 341–346, 2020, doi: 10.1016/j.procir.2020.03.039.
N. Nurrisma, R. Munadi, S. Syahrial, and E. D. Meutia, “Perancangan Augmented Reality dengan Metode Marker Card Detection dalam Pengenalan Karakter Korea,” Inform. Mulawarman J. Ilm. Ilmu Komput., vol. 16, no. 1, p. 34, 2021, doi: 10.30872/jim.v16i1.5152.
S. D. Y. Kusuma, “Perancangan Aplikasi Augmented Reality Pembelajaran Tata Surya dengan Menggunakan Marker Based Tracking,” J. Inform. Univ. Pamulang, vol. 3, no. 1, p. 33, 2018, doi: 10.32493/informatika.v3i1.1428.
B. Marques, B. S. Santos, T. Araújo, N. C. Martins, J. Alves, and P. Dias, “Situated Visualization in the Decision Process through Augmented Reality,” Proc. Int. Conf. Inf. Vis., vol. 2019-July, no. Iv, pp. 13–18, 2019, doi: 10.1109/IV.2019.00012.
M. Mirbabaie and J. Fromm, “REDUCING THE COGNITIVE LOAD OF DECISION-MAKERS IN EMERGENCY MANAGEMENT THROUGH AUGMENTED REALITY,” in ECIS 2019 Proceedings, 2019, pp. 0–11.
V. Sangiorgio, S. Martiradonna, F. Fatiguso, and I. Lombillo, “Augmented reality based - decision making (AR-DM) to support multi-criteria analysis in constructions,” Autom. Constr., vol. 124, no. May 2020, p. 103567, 2021, doi: 10.1016/j.autcon.2021.103567.
M. Lorenz, S. Knopp, and P. Klimant, “Industrial Augmented Reality: Requirements for an Augmented Reality Maintenance Worker Support System,” Adjun. Proc. - 2018 IEEE Int. Symp. Mix. Augment. Reality, ISMAR-Adjunct 2018, pp. 151–153, 2018, doi: 10.1109/ISMAR-Adjunct.2018.00055.
Z. C. Rawis, V. Tulenan, and B. A. Sugiarso, “Penerapan Augmented Reality Berbasis Android Untuk Mengenalkan Pakaian Adat Tountemboan,” J. Tek. Inform., vol. 13, no. 1, Feb. 2018.
M. A. Nazaruddin and M. Efendi, “The Book of Pop Up Augmented Reality to Increase Focus and Object Recognition Capabilities for Children with Autism,” J. ICSAR, vol. 2, no. 1, pp. 9–14, 2018, doi: 10.17977/um005v2i12018p009.
E. Pantano, A. Rese, and D. Baier, “Enhancing the online decision-making process by using augmented reality: A two country comparison of youth markets,” J. Retail. Consum. Serv., vol. 38, no. May, pp. 81–95, 2017, doi: 10.1016/j.jretconser.2017.05.011.
C. Berenguer, I. Baixauli, S. Gómez, M. de E. P. Andrés, and S. De Stasio, “Exploring the Impact of Augmented Reality in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review,” Int. J. Environ. Res. Public Health, vol. 17, no. 17, pp. 1–15, 2020, doi: 10.3390/ijerph17176143.
E. Agustina Julisawaty et al., “Aplikasi Visualisasi Penyakit Jantung Manusia Berbasis Augmented Reality Menggunakan Unity,” Pros. Semnastek, vol. 0, no. 0, Dec. 2017.