Locus Sonus, the audio art research group I work with, created an installation using the virtual world Second Life. For the Seconde Nature festival in Aix-en-Provence, we recreated a portion of the courtyard of the Cité du Livre, where the installation physically existed, in Second Life. This representation of the real world was joined to a series of rooms in the virtual space. Inside the rooms were a number of object that denizens of Second Life could move about. Each of these objects created a unique sound.The objects’ sound was reverberated through the virtual space and played back in the physical space of the installation. When objects were moved into the recreated space of the Cité du Livre, the sound was spatialized as if they were moving around in the physical world. The audio from the real world was streamed back into Second Life, completing the fusion of the virtual and real spaces.
Technically, this required getting the coordinates of the object in Second Life, sending them to a webserver, which then passed the data to a Pure Data patch that ran the audio into the real space. A microphone picked up the sound and streamed it back into Second Life.
You can read more about the project here, see the code used, and check out some conclusions I drew based on the experience.