Sound Mind

The term "Sound Mind" was first coined by Nina Kraus to capture the intricate relationship between both "sound" and the "mind", as she describes:

"I think of the sound mind as a force behind a continuum from the past to the present and into the future. The sounds we have engaged with over our whole lifetimes have shaped what our brain is today. Our brain today, in turn, can make decisions about how we shape our sonic world going forward, not just our personal futures but those of our children and of society as a whole." - Of Sound Mind: How Our brain Constructs a Meaningful Sonic World

When listening to music, we not only receive auditory stimulation but also experience a fusion of visual and tactile sensations [1, 2]. Hence, the way in wich music engages multiple senses simultaneously in turn shapes the way we interact with sound. Think of for example how vibro-tactile feedback assists drummers in maintaining precise timing and rhythm.

While there has been a significant number of studies dedicated to the contribution of visual information on musical perception, there exist a comparatively limited comprehension with respect to the role played by the tactile system [3. 4, 5]. This project aims to address this gap by employing a technique called the Frequency Following Response (FFR).

Frequency Following Response (FFR)

The FFR is a neurological brain measure that physically resembles the features of an evoking sound stimulus [, ]. Hence, this allows us to directly examine how different sound attributes (e.g., pitch, timbre) are encoded within an individual's brain as an objective index for musical perception.

This project leverages the technique of FFR while integrating the vibro-tactile modality. Accordingly, vibro-tactile stimuli are randomly presented while recording the FFR under continuous auditory stimulation. Subsequently, both conditions (audio vs audio-tactile) can be contrasted to objectively evaluate the effect of the vibro-tactile modality on the FFR signal accordingly.

Demonstration

The animation below displays the FFR to a 40-ms /da/ consonant vowel stimulus. The signal of interest is on the order of nanovolts, and to average out the noise many trials have to be collected. Hence, the trial counter for both animations indicate how many trials have been collected and averaged successively.

The animation on the left displays the temporal domain. Subsequently, the Fast Fourier Transform is taken over the [19.5 ms-44.2 ms] time window which is displayed on the right side accordingly.