The purpose of our project is to provide a method and tools for the online transmission of spatial audio information with a low computational cost. The online transmission of spatial audio is illustrated in Fig. 1.
Spatial audio information is captured, transmitted, and presented to the listener at the remote site. The reproduced sound gives the listener a sensation of being listening to the sound in the space where the sound is actually recorded.
A dummy head that simulates a head and torso of a human has been used for the binaural recording. Figure 2 shows pictures of a dummy head (Head And Torso Simulator). The dummy head has microphones in its left and right ear canals.
When the sounds recorded by the microphones in the dummy head are presented to the listener through headphones, a sensation of being listening to these sounds in the real world environment can be given to the listener (Fig.3).
Because of its size and weight, however, it is not very easy to use the conventional dummy head.
Spatial audio information can be captured by a microphone array with tens or hundreds of microphones and reproduced by a loudspeaker array as illustrated in Fig. 4. In this case, headphones are not necessary and a wide listening area can be provided. It reqiures, however, high bit rate and high computational cost for transmitting and processing the data.
A miniature head simulator system is a new technology for capturing and processing spatial sound. It consists of a tetrahedral microphone system and a signal processor . Figure 5 shows the microphone systems that consist of 4 omnidirectional microphones. These microphones are at the vertices of a tetrahedron. Since each side of the tetrahedron is as short as a few cm, the microphone system is far smaller than the conventional dummy head (Figure 6).
Figure 5 Microphone systems of a miniature head simulator
Each side of a tetrahedron is 30 mm (left) and 15 mm (right).
Figure 6 Comparison with the conventional dummy head
A unique feature of this technology is that it allows for interactive manipulation
In the dummy head recording, since the front and back are physically determined by the shape of the dummy head, directions of sound images are also determined by the relative position of the sound sources and the dummy head.
In the miniature head simulator, the front can be arbitrary determined and therefore panning of the sound can be controlled by the listener during the reproduction.
Since the spatial property of the sound at the observation point can be given mathematically, any desired polar pattern can be generated programmatically. For more information, see Panoramic beamformer.
Here, we provide samples of sounds recorded by the miniature head simulator.
In these demonstrations, stereo headphones are necessary.
The demo can be opened by clicking the thumbnail below.
Once the image is loaded and the page says "Click to start" or "Tap to start," you can start reproducing the sound by clicking (tapping) on the image.
If the instruction saying "Tap on the image" appears, tap on the screen and the browser starts loading data. Then the page will say "Tap to start" and you can start reproducing panoramic sound by tapping the image.
During the reproduction, panning of sound changes according to the motion of the mouse pointer.
If you are using the mobile device, pan and tilt can be controlled by the rotation of the device instead of the motion of the mouse.
In this demonstration, spatial audio information is streamed via a
SOPA format that is a custom data format proposed by us.
An array of more than one miniature head simulator has been proposed -> Panoramic sound morphing.
Thanks to Three.js (http://threejs.org/)
The miniature head simulator system can be used even for inaudible ultrasound .
By making the microphone distance very short, the upper limit of the frequency range expands into the ultrasonic frequency. For more information, see Panoramic ultrasound.