Stream Of Panoramic Audio
Since 26 June, 2012 7:16
Last update; 15 July, 2016
To view this page ensure that the latest version of Adobe Flash Player is installed.
In real life, localization of sound sources are rarely static. When listeners
turn their head to the left or right in the real-world environment, the
flow of sound from their viewpoint instantly changes. Is it possible to
transmit such sort of sensation online?
Our answer is, 'Yes it is.'
On this site, we introduce our technology for online transmission of spatial audio information. This technology is named 'SOPA' which stands for 'Stream Of Panoramic Audio' and its features can be summarized as follows.
A unique feature of this technology is that it allows for interactive manipulation of panning.
In real-world situations, the locations of sound sources vary according to the movement of the listener's head. If a listener directly facing the sound source, for instance, a loudspeaker, turns his or her head to the left or right, the sound no longer arrives from the front. Therefore head movements can be interpreted as movements of the world from the viewpoint of the listener. In reality, this never occurs when listening to music through headphones since the headphones rotate along with the head.
SOPA makes it possible to control the panning of the sound during its reproduction.
SOPA data contain not only an audio data stream but also its spatial information. Spatial information can be used to produce directionality so that the listener can pick up sounds from any desired direction and reject sound from other directions.
The listener can control the target direction while listening to the sound.
Spatial audio information captured by the miniature head simulator system can be encoded to a custom data format. Even though its size is euivalent to the conventional stereo WAV file, a SOPA file contains not only the acoustic data stream but also the spatial information.
SOPA data can be decoded by using only a simple trigonometric function and Fast Fourier Transform. Since this process does not require much computer resources, a SOPA file can be reproduced not only on the desktop computers but also on mobile devices.
By using the SOPA format, spatial audio information can be streamed to the browsers, smartphones, and tablets with a low bit rate.
A SOPA file contains only the monaural audio data stream and the directional information. How to reproduce the data is not specified anywhere in the SOPA file. How to reconstruct the space acoustically from the SOPA file is open to the developers.
Although all demonstrations presented on this site require stereo headphones, spatial audio information can be reconstructed from a SOPA file also for the multi-channel loudspeaker system.
We constructed, for instance, a 6-channel loudspeaker system to reproduce SOPA data that is illustrated in the figure below. The system is named 'Panoramic Sound Spot System .'
Fig. 6-channel Panoramic Sound Spot System
Although this 6-channel loudspeaker system is rather small and for the personal use, the system might be expanded for the theatrical performance.
A Java program for the 6-channel Panoramic Sound Spot System 'pss6tr' is available. You can download a jar file from pss6tr.jar (SOPA archives).
A Java program for the 4-channel Panoramic Sound Spot System 'pss4tr' is also available. You can download a jar file from pss4tr.jar (SOPA archives).
Here, we provide samples demonstrating the capabilities of SOPA.
In the demonstrations, stereo headphones are necessary.
To start the demonstration, click on the thumbnail below and the Flash demo will appear in the new window.
Once the image is loaded, you can start reproducing the sound by clicking on the image. During the reproduction, the panning of sound can be controlled by moving the mouse cursor to the left or the right.
Thanks to AWAY3D (http://away3d.com/) for 3d graphics
The latest version of Adobe Flash Player has to be installed on your PC.
If you use smartphones or tablets, you can probably use Puffin Browser (http://www.puffinbrowser.com/index.php) to see the Flash contents. In this case, however, the performance may be limited to some extent and there may be some degradation.
The images in the demonstrations are made of 6 different video files each of which is attached to each plane of a cube. Since they are not completely synchronized, you will ocasionally see image discontinuities especially when network traffic is heavy.