This internship was a preliminary study within a larger ANR project named RASPUTIN project (Room Acoustic Simulation for Perceptually Realistic Users in Real-Time Immersive and Navigation Experiences). This project is aimed at the visually impaired public.
Its purpose is to build a multi-modal virtual reality simulation, intended to help blind people to learn how to localize themselves and move safely inside a real enclosed place, such as a building, by exploring it first virtually. The idea is to get a mental representation of an environment, without vision but using acoustic properties of the architecture and ambient sounds of the real place modeled in the game, to learn its layout and then to be able to find their way in the reality.
The aim of my M1 internship was to search for some acoustic binaural cues related to the geometric properties of the environment (for example the sound variation that would make an opened door in a room, a turn in a corridor, a staircase rising somewhere) and that they are used to rely on to move safely inside the building. Once those cues are identified, they will be validated in the virtual simulation.
The work consisted in trying to extract those cues from audio signals, recorded by a binaural head moved along different environments, and in the future to have blind people listen to them to search for a correlation between their impressions and some acoustical metrics we extract using MATLAB scripts.
First part of the work was to define scenarios containing some geometrical variations that would be interesting to analyze. I set up 2 different environments: the first one was called “corridor crossing” and represented a long corridor joined by a smaller one. The idea was to know if there was any acoustic variation when walking through the long corridor, that would indicate the presence of the crossroads.
The second scenario was called corridor widening, it was representing a volume variation on a corridor linked to a bigger room. The idea was in this case to search for any acoustical parameter that would change when the corridor gets wider.
The binaural head was mounted on a rolling cart and was used to record audio signals in the two scenarios: it was pushed by an extensible cane, and its position was known at any instant thanks to a camera recording landmarks on the floor.
Different sounds were used in the scenarios: on the rolling cart, a sound source was emitting some finger snaps sounds to reproduce echolocation phenomenon (ability to produce a sound and to analyze its echo after reverberation on the environment), and another sound source was emitting babble noise in different positions of the environment, to create landmarks and furnish the sound space.
Among the many clues I tried to reveal in the measurements, the most expressive were the Interaural Level Difference (ILD), which corresponds to a sound intensity difference between left and right ears ; and the variance of the auto-correlation function, which traduces irregularities in the signals of both ears. For example, we can see an emergence of the sound intensity on the left of the head at the moment when it passes in front of the crossroads. The babble noise sound source was set in the office, represented by a red circle.
The echolocation analysis was not easy to implement because of poor signal to noise ratio, but it gave information on the reverberation time evolution in the second scenario, between the corridor and the larger room.
