Sound plays a crucial role in daily life, with music being essential for emotional connection [1]. For cochlear implant (CI) users, hearing music is challenging as the brain must adapt to the implant’s signals, which differ from normal hearing [2]. Music training is vital for improving auditory skills but is particularly difficult for CI users, especially those who enjoyed music prior to their implant. Serious games offer a promising way to make this training more engaging and effective.

This study explores how a serious game can enhance music listening skills in CI users, focusing on pedagogical training to support active listening and analytical skills in cognitive audio processing. The game is designed to aid auditory training through top-down restoration [3], helping users better perceive music by focusing on spectral and timbral elements. The game was developed in a User-Centered Design approach together with a musician with mixed hearing loss, drawing on his particular challenges and background as a music producer. The game, set in a music store, involves repairing broken musical components in a 2D puzzle format, with three stages that focus on subtle auditory cues and pitch tuning through slider stages. Dynamic auditory feedback adjusts in real-time based on player actions.

The Learning Mechanics-Game Mechanics (LM-GM) model was employed to integrate educational content with gameplay [4]. This model helps align the educational objectives of the game with its interactive components, ensuring that the learning process is both engaging and effective. The game mechanics are designed to train problem-solving, cognitive audio processing, and auditory discrimination while providing meaningful feedback that supports skill development.
The participant responded positively to the game, enjoying its balance of challenge and usability. The game’s light-hearted narrative encourages altruism, and there is potential for adding a progression system, customizable auditory profiles, and social features. Also, AI-driven adaptability could make the game more personalized, adjusting difficulty based on user performance in real-time. Future studies will incorporate additional in-game training activities to tackle challenges such as audio scene analysis, while also expanding the game's accessibility to areas beyond hearing.

References:
[1] Barton, G. (2018). The Relationship Between Music, Culture, and Society: Meaning in Music. In Music Learning and Teaching in Culturally and Socially Diverse Contexts (pp. 23–41). Springer International Publishing. https://doi.org/10.1007/978-3-319-95408-0_2
[2] Warren, S., Messersmith, J., & Entwisle, L. (2018). Cochlear Implantation for Children and Adults with Severe-to-Profound Hearing Loss. Seminars in Hearing, 39(04), 390–404. https://doi.org/10.1055/s-0038-1670705
[3] Schierholz, I., Schönermark, C., Ruigendijk, E., Kral, A., Kopp, B., & Büchner, A. (2021). An event-related brain potential study of auditory attention in cochlear implant users. Clinical Neurophysiology, 132(9), 2290–2305. https://doi.org/10.1016/j.clinph.2021.03.055
[4] Arnab, S., Lim, T., Carvalho, M. B., Bellotti, F., de Freitas, S., Louchart, S., Suttie, N., Berta, R., & De Gloria, A. (2014). Mapping learning and game mechanics for serious games analysis. British Journal of Educational Technology, 46(2), 391–411. https://doi.org/10.1111/bjet.12113

Keywords: Serious games, music appreciation, auditory rehabilitation, hearing training, cochlear implant, sound and music computing, Learning Mechanics-Game Mechanics.