This Ph.D. project by Francesco Ganis is a collaboration between the Multisensory Experience Lab and the Center for Hearing and Balance from Rigshospitalet of Copenhagen.

The project integrates vibrotactile augmentation into musical training to enhance the music perception of cochlear implanted children. The objective is to incorporate musical training into the existing framework provided by clinicians (e.g. during audio-verbal therapy sessions). This training is conveyed through gamification and, where applicable, supported by vibrotactile feedback. Touch is a sensory modality that we use to perceive the shape, texture, and temperature of objects. Previous research demonstrates that providing individuals with hearing impairments with both sound and vibrations enhances their perception of music and speech.

This study specifically focuses on musical training for children and teenagers, with the ultimate goal of offering an engaging and effective method for practicing at home, thereby improving the listening skills of patients. The software is paired with appropriate hardware, such as controllers and mobile interfaces, to provide high-quality vibrotactile feedback when necessary.

The musical aspects taken into consideration include fundamental descriptors such as pitch, amplitude, timbre, and rhythm, as well as higher-level elements like emotion and enjoyment.