This Ph.d project by Razvan Paisa investigates the use of vibrotactile augmentation of music for cochlear implant users.

After receiving a cochlear implant, individuals do not appreciate music any longer, as these hearing devices aim to restore speech capabilities, often disregarding musical percepts or appreciation.

An exciting new possibility to improve the perceived hearing capabilities of CI users is to use multisensory stimulation – be it from the visual domain or the tactile domain.

This project aims to explore the possibility of augmenting music with vibrotactile stimuli so that individuals with cochlear implants would find more enjoyment in music. The focus is on creating tightly integrated hardware, software, and musical experiences (e.g., concerts) that aim to improve music characteristics such as melody or timber perception in a supra-additive fashion. Furthermore, we are exploring new training music paradigms that include vibrotactile stimulation for CI users, that are assisting them throughout the training process. As a result, the project will develop novel methodologies for multisensory signal processing, focusing on the physiological and perceptual characteristics of the touch and hearing, as well as new training schemes for CI users willing to pursue a continuous restoration of music hearing and understanding capabilities.

This project is conducted in collaboration with the Center for Hearing and Balance from Rigshospitalet as well as The Royal Danish Academy of Music.