Skip to main content
EURASIP Journal on Advances in Signal Processing
Download PDF
  • Research Article
  • Open access
  • Published:

Virtual Reality System with Integrated Sound Field Simulation and Reproduction

EURASIP Journal on Advances in Signal Processing volume 2007, Article number: 070540 (2007) Cite this article

Abstract

A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.

References

  1. Begault DR: Challenges to the successful implementation of 3-D sound. Journal of the Audio Engineering Society 1991,39(11):864-870.

    Google Scholar 

  2. Naef M, Staadt O, Gross M: Spatialized audio rendering for immersive virtual environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '02), November 2002, Hong Kong 65–72.

    Chapter  Google Scholar 

  3. Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC: The CAVE: audio visual experience automatic virtual environment. Communications of the ACM 1992,35(6):65-72.

    Article  Google Scholar 

  4. Burgess DA, Verlinden JC: An architecture for spatial audio servers. Proceedings of Virtual Reality Systems Conference (Fall '93), November 1993, New York, NY, USA

    Google Scholar 

  5. Mulder JD, Dooijes EH: Spatial audio in graphical applications. In Visualization in Scientific Computing. Edited by: Göbel M, Müller H, Urban B. Springer, Wien, Austria; 1994:215-229.

    Google Scholar 

  6. Lake Huron 2005.https://doi.org/www.lake.com.au/

  7. Savioja L: Modeling Techniques for Virtual Acoustics, Ph.D. thesis. Helsinki University of Technology, Helsinki, Finland; 1999.

    Google Scholar 

  8. Savioja L, Huopaniemi J, Lokki T, Väänänen R: Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society 1999,47(9):675-705.

    Google Scholar 

  9. Funkhouser T, Min P, Carlbom I: Real-time acoustic modeling for distributed virtual environments. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 365–374.

    Chapter  Google Scholar 

  10. Storms RL: Npsnet-3D Sound Server: An Effective Use of the Auditory Channel. 1995.

    Google Scholar 

  11. Kuttruff H: Room Acoustics. 4th edition. Elsevier Science Publisher, New York, NY, USA; 2000.

    Google Scholar 

  12. Allen JB, Berkley DA: Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 1979,65(4):943-950. 10.1121/1.382599

    Article  Google Scholar 

  13. Borish J: Extension of the image model to arbitrary polyhedra. The Journal of the Acoustical Society of America 1984,75(6):1827-1836. 10.1121/1.390983

    Article  Google Scholar 

  14. Dalenbäck B-IL: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. The Journal of the Acoustical Society of America 1996,100(2):899-909. 10.1121/1.416249

    Article  Google Scholar 

  15. Forsberg P-A: Fully discrete ray tracing. Applied Acoustics 1985,18(6):393-397. 10.1016/0003-682X(85)90021-0

    Article  Google Scholar 

  16. Funkhouser T, Tsingos N, Carlbom I, et al.: A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 2004,115(2):739-756. 10.1121/1.1641020

    Article  Google Scholar 

  17. Naylor GM: ODEON—another hybrid room acoustical model. Applied Acoustics 1993,38(2–4):131-143.

    Article  Google Scholar 

  18. Stephenson UM: Quantized pyramidal beam tracing—a new algorithm for room acoustics and noise immission prognosis. Acta Acustica United with Acustica 1996,82(3):517-525.

    Google Scholar 

  19. van Maercke D: Simulation of sound fields in time and frequency domain using a geometrical model. Proceedings of the 12th International Congress on Acoustics (ICA '86), July 1986, Toronto, Ontario, Canada 2: paper E11-7

    Google Scholar 

  20. Vorländer M: Simulation of the transient and steady state sound propagation in rooms using a new combined sound particle—image source algorithm. The Journal of the Acoustical Society of America 1989, 86: 172–178. 10.1121/1.398336

    Article  Google Scholar 

  21. Bork I: A comparison of room simulation software—the 2nd round Robin on room acoustical computer simulation. Acta Acustica United with Acustica 2000,86(6):943-956.

    Google Scholar 

  22. Vorländer M: International round Robin on room acoustical computer simulations. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, Norway 689–692.

    Google Scholar 

  23. Kuttruff H: A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries. The Journal of the Acoustical Society of America 1995,98(1):288-293. 10.1121/1.413727

    Article  Google Scholar 

  24. Christensen CL, Rindel JH: A new scattering method that combines roughness and diffraction effects. Forum Acousticum, 2005, Budapest, Hungary

    Google Scholar 

  25. Heinz R: Binaural room simulation based on an image source model with addition of statistical methods to include the diffuse sound scattering of walls and to predict the reverberant tail. Applied Acoustics 1993,38(2–4):145-159.

    Article  Google Scholar 

  26. Lam YW: A comparison of three reflection modelling methods used in room acoustics computer models. The Journal of the Acoustical Society of America 1996,100(4):2181-2192. 10.1121/1.417927

    Article  Google Scholar 

  27. Vorländer M: Ein Strahlverfolgungsverfahren zur Berechnung von Schallfeldern in Räumen. Acustica 1988,65(3):138-148.

    Google Scholar 

  28. Suh JS, Nelson PA: Measurement of transient response of rooms and comparison with geometrical acoustic models. The Journal of the Acoustical Society of America 1999,105(4):2304-2317. 10.1121/1.426837

    Article  Google Scholar 

  29. Svensson UP, Fred RI, Vanderkooy J: An analytic secondary source model of edge diffraction impulse responses. The Journal of the Acoustical Society of America 1999,106(5):2331-2344. 10.1121/1.428071

    Article  Google Scholar 

  30. Tsingos N, Funkhouser T, Ngan A, Carlbom I: Modeling acoustics in virtual environments using the uniform theory of diffraction. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), August 2001, Los Angeles, Calif, USA 545–552.

    Chapter  Google Scholar 

  31. Stephenson UM: Beugungssimulation ohne Rechenzeitexplosion: die Methode der quantisierten Pyramidenstrahlen; ein neues Berechnungsverfahren für Raumakustik und Lärmimmissionsprognose; Vergleiche, Ansätze, Lösungen, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 2004.

    Google Scholar 

  32. Slater M, Steed A, Chrysanthou Y: Computer Graphics and Virtual Environments: From Realism to Real-Time. Addison Wesley, New York, NY, USA; 2001.

    Google Scholar 

  33. Cremer L, Müller HA: Die wissenschaftlichen Grundlagen der Raumakustik—Band 1. 2nd edition. S. Hirzel, Stuttgart, Germany; 1978.

    Google Scholar 

  34. Akenine-Möller T, Haines E: Real-Time Rendering. 2nd edition. A. K. Peters, Natick, Mass, USA; 2002.

    Google Scholar 

  35. Foley JD, van Dam A, Feiner SK, Hughes JF: Computer Graphics, Principles and Practice. 2nd edition. Addison Wesley, Reading, Mass, USA; 1996.

    MATH  Google Scholar 

  36. Shumacker R, Brand R, Gilliland M, Sharp W: Study for applying computer-generated images to visual simulations. In Report AFHRL-TR-69-14. U.S. Air Force Human Resources Laboratory, San Antonio, Tex, USA; 1969.

    Google Scholar 

  37. Cormen TH, Leiserson CE, Rivest RL, Stein C: Introduction to Algorithms. 2nd edition. MIT Press, Cambridge, Mass, USA; 2001.

    MATH  Google Scholar 

  38. Schröder D, Lentz T: Real-time processing of image sources using binary space partitioning. Journal of the Audio Engineering Society 2006,54(7-8):604-619.

    Google Scholar 

  39. Heinz R: Entwicklung und Beurteilung von computergestützten Methoden zur binauralen Raumsimulation, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 1994.

    Google Scholar 

  40. Bradley JS, Soulodre GA: The influence of late arriving energy on spatial impression. The Journal of the Acoustical Society of America 1995,97(4):2263-2271. 10.1121/1.411951

    Article  Google Scholar 

  41. Rindel JH: Evaluation of room acoustic qualities and defects by use of auralization. Proceedings of the 148th Meeting of the Acoustical Society of America, November 2004, San Diego, Calif, USA

    Google Scholar 

  42. Schröder D, Dross P, Vorländer M: A fast reverberation estimator for virtual environments. Proceedings of the AES 30th International Conference, March 2007, Saariselkä, Finland

    Google Scholar 

  43. Brookes T, Treble C: The effect of non-symmetrical left/right recording pinnae on the perceived externalisation of binaural recordings. Proceedings of the 118th Audio Engineering Society Convention, May 2005, Barcelona, Spain

    Google Scholar 

  44. Brungart DS, Rabinowitz WM, Durlach NI: Auditory localization of a nearby point source. The Journal of the Acoustical Society of America 1996,100(4):2593.

    Article  Google Scholar 

  45. Kulkarni A, Colburn HS: Role of spectral detail in sound-source localization. Nature 1998,396(6713):747-749. 10.1038/25526

    Article  Google Scholar 

  46. Lehnert H, Richter M: Auditory virtual environment: simplified treatment of reflections. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, Norway

    Google Scholar 

  47. Romanenko G, Vorländer M: Employment of spherical wave reflection coefficient in room acoustics. IoA Symposium Surface Acoustics, 2003, Salford, UK

    Google Scholar 

  48. Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93), August 1993, Anaheim, Calif, USA. ACM Press; 135–142.

    Chapter  Google Scholar 

  49. Bauer BB: Stereophonic earphones and binaural loudspeakers. Journal of the Audio Engineering Society 1961,9(2):148-151.

    Google Scholar 

  50. Kirkeby O, Nelson PA, Hamada H: Local sound field reproduction using two closely spaced loudspeakers. The Journal of the Acoustical Society of America 1998,104(4):1973-1981. 10.1121/1.423763

    Article  Google Scholar 

  51. Møller H: Reproduction of artificial-head recordings through loudspeakers. Journal of the Audio Engineering Society 1989,37(1-2):30-33.

    Google Scholar 

  52. Gardner WG: 3-D audio using loudspeakers, Ph.D. thesis. Massachusetts Institute of Technology, Cambridge, Mass, USA; 1997.

    Google Scholar 

  53. Lentz T, Schmitz O: Realisation of an adaptive cross-talk cancellation system for a moving listener. Proceedings of the 21st Audio Engineering Society Conference, June 2002, St. Petersburg, Russia

    Google Scholar 

  54. Lentz T, Behler GK: Dynamic cross-talk cancellation for binaural synthesis in virtual reality environments. Proceedings of the 117th Audio Engineering Society Convention, October 2004, San Francisco, Calif, USA

    Google Scholar 

  55. Steinberg : ASIO 2.0 Audio Streaming Input Output Development Kit. 2004.

    Google Scholar 

  56. Lentz T: Dynamic crosstalk cancellation for binaural synthesis in virtual reality environments. Journal of the Audio Engineering Society 2006,54(4):283-294.

    Google Scholar 

  57. Takeuchi T, Nelson P, Kirkeby O, Hamada H: The effects of reflections on the performance of virtual acoustic imaging systems. Proceedings of the International Symposium on Active Control of Sound and Vibration (ACTIVE '97), August 1997, Budapest, Hungary 955–966.

    Google Scholar 

  58. Ward DB: On the performance of acoustic crosstalk cancellation in a reverberant environment. The Journal of the Acoustical Society of America 2001,110(2):1195-1198. 10.1121/1.1386635

    Article  MathSciNet  Google Scholar 

  59. Lentz T, Sokoll J, Assenmacher I: Performance of spatial audio using dynamic cross-talk cancellation. Proceedings of the 119th Audio Engineering Society Convention, October 2005, New York, NY, USA

    Google Scholar 

  60. Gardner WG: Efficient convolution without input-output delay. Journal of the Audio Engineering Society 1995,43(3):127-136.

    Google Scholar 

  61. La Viola JJ Jr.: A testbed for studying and choosing predictive tracking algorithms in virtual environments. Proceedings of the 7th International Immersive Projection Technologies Workshop, 9th Eurographics Workshop on Virtual Environments, May 2003, Zurich, Switzerland 189–198.

    Google Scholar 

  62. Azuma R, Bishop G: A frequency-domain analysis of head-motion prediction. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), August 1995, Los Angeles, Calif, USA. ACM Press; 401–408.

    Chapter  Google Scholar 

  63. Chai L, Hoff WA, Vincent T: Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence: Teleoperators and Virtual Environments 2002,11(5):474-492. 10.1162/105474602320935829

    Article  Google Scholar 

  64. Wu J-R, Ouhyoung M: A 3D tracking experiment on latency and its compensation methods in virtual environments. In Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST '95), November 1995, Pittsburgh, Pa, USA. ACM Press; 41–49.

    Chapter  Google Scholar 

  65. Witew IB: Spatial variation of lateral measures in different concert halls. Proceedings of the 18th International Congress on Acoustics (ICA '04), April 2004, Kyoto, Japan 4: 2949.

    Google Scholar 

  66. Azuma R, Bishop G: Improving static and dynamic registration in an optical see-through HMD. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '94), July 1994, New York, NY, USA. ACM Press; 197–204.

    Chapter  Google Scholar 

  67. Pompetzki W: Psychoakustische Verifikation von Computermodellen zur binauralen Raumsimulation, Ph.D. thesis. Ruhr-Universität Bochum, Bochum, Germany; 1993.

    Google Scholar 

  68. Vorländer M, Mommertz E: Definition and measurement of random-incidence scattering coefficients. Applied Acoustics 2000,60(2):187-199. 10.1016/S0003-682X(99)00056-0

    Article  Google Scholar 

  69. ISO 354 : Acoustics, Measurement of sound absorption in a reverberant room. 2003.

    Google Scholar 

  70. ISO/DIS 17497-1 : Acoustics Measurement of the sound scattering properties of surfaces—part 1: measurement of the randomincidence scattering coefficient in a reverberation room.

  71. Tsingos N: Scalable perceptual mixing and filtering of audio signals using an augmented spectral representation. Proceedings of the 8th International Conference on Digital Audio Effects (DAFx '05), September 2005, Madrid, Spain

    Google Scholar 

  72. Tsingos N, Gallo E, Drettakis G: Perceptual audio rendering of complex virtual environments. Proceedings of the 31st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '04), August 2004, Los Angeles, Calif, USA 249–258.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Technical Acoustics, RWTH Aachen University, Neustrasse 50, Aachen, 52066, Germany

    Tobias Lentz, Dirk Schröder & Michael Vorländer

  2. Virtual Reality Group, RWTH Aachen University, Seffenter Weg 23, Aachen, 52074, Germany

    Ingo Assenmacher

Authors
  1. Tobias Lentz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dirk Schröder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Vorländer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ingo Assenmacher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Lentz.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Lentz, T., Schröder, D., Vorländer, M. et al. Virtual Reality System with Integrated Sound Field Simulation and Reproduction. EURASIP J. Adv. Signal Process. 2007, 070540 (2007). https://doi.org/10.1155/2007/70540

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/70540

Keywords