Skip to main content

Advertisement

You are viewing the new BMC article page. Let us know what you think. Return to old version

Research Article | Open | Published:

Virtual Reality System with Integrated Sound Field Simulation and Reproduction

Abstract

A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.

References

  1. 1.

    Begault DR: Challenges to the successful implementation of 3-D sound. Journal of the Audio Engineering Society 1991,39(11):864-870.

  2. 2.

    Naef M, Staadt O, Gross M: Spatialized audio rendering for immersive virtual environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '02), November 2002, Hong Kong 65–72.

  3. 3.

    Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC: The CAVE: audio visual experience automatic virtual environment. Communications of the ACM 1992,35(6):65-72.

  4. 4.

    Burgess DA, Verlinden JC: An architecture for spatial audio servers. Proceedings of Virtual Reality Systems Conference (Fall '93), November 1993, New York, NY, USA

  5. 5.

    Mulder JD, Dooijes EH: Spatial audio in graphical applications. In Visualization in Scientific Computing. Edited by: Göbel M, Müller H, Urban B. Springer, Wien, Austria; 1994:215-229.

  6. 6.

    Lake Huron 2005.https://doi.org/www.lake.com.au/

  7. 7.

    Savioja L: Modeling Techniques for Virtual Acoustics, Ph.D. thesis. Helsinki University of Technology, Helsinki, Finland; 1999.

  8. 8.

    Savioja L, Huopaniemi J, Lokki T, Väänänen R: Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society 1999,47(9):675-705.

  9. 9.

    Funkhouser T, Min P, Carlbom I: Real-time acoustic modeling for distributed virtual environments. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 365–374.

  10. 10.

    Storms RL: Npsnet-3D Sound Server: An Effective Use of the Auditory Channel. 1995.

  11. 11.

    Kuttruff H: Room Acoustics. 4th edition. Elsevier Science Publisher, New York, NY, USA; 2000.

  12. 12.

    Allen JB, Berkley DA: Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 1979,65(4):943-950. 10.1121/1.382599

  13. 13.

    Borish J: Extension of the image model to arbitrary polyhedra. The Journal of the Acoustical Society of America 1984,75(6):1827-1836. 10.1121/1.390983

  14. 14.

    Dalenbäck B-IL: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. The Journal of the Acoustical Society of America 1996,100(2):899-909. 10.1121/1.416249

  15. 15.

    Forsberg P-A: Fully discrete ray tracing. Applied Acoustics 1985,18(6):393-397. 10.1016/0003-682X(85)90021-0

  16. 16.

    Funkhouser T, Tsingos N, Carlbom I, et al.: A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 2004,115(2):739-756. 10.1121/1.1641020

  17. 17.

    Naylor GM: ODEON—another hybrid room acoustical model. Applied Acoustics 1993,38(2–4):131-143.

  18. 18.

    Stephenson UM: Quantized pyramidal beam tracing—a new algorithm for room acoustics and noise immission prognosis. Acta Acustica United with Acustica 1996,82(3):517-525.

  19. 19.

    van Maercke D: Simulation of sound fields in time and frequency domain using a geometrical model. Proceedings of the 12th International Congress on Acoustics (ICA '86), July 1986, Toronto, Ontario, Canada 2: paper E11-7

  20. 20.

    Vorländer M: Simulation of the transient and steady state sound propagation in rooms using a new combined sound particle—image source algorithm. The Journal of the Acoustical Society of America 1989, 86: 172–178. 10.1121/1.398336

  21. 21.

    Bork I: A comparison of room simulation software—the 2nd round Robin on room acoustical computer simulation. Acta Acustica United with Acustica 2000,86(6):943-956.

  22. 22.

    Vorländer M: International round Robin on room acoustical computer simulations. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, Norway 689–692.

  23. 23.

    Kuttruff H: A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries. The Journal of the Acoustical Society of America 1995,98(1):288-293. 10.1121/1.413727

  24. 24.

    Christensen CL, Rindel JH: A new scattering method that combines roughness and diffraction effects. Forum Acousticum, 2005, Budapest, Hungary

  25. 25.

    Heinz R: Binaural room simulation based on an image source model with addition of statistical methods to include the diffuse sound scattering of walls and to predict the reverberant tail. Applied Acoustics 1993,38(2–4):145-159.

  26. 26.

    Lam YW: A comparison of three reflection modelling methods used in room acoustics computer models. The Journal of the Acoustical Society of America 1996,100(4):2181-2192. 10.1121/1.417927

  27. 27.

    Vorländer M: Ein Strahlverfolgungsverfahren zur Berechnung von Schallfeldern in Räumen. Acustica 1988,65(3):138-148.

  28. 28.

    Suh JS, Nelson PA: Measurement of transient response of rooms and comparison with geometrical acoustic models. The Journal of the Acoustical Society of America 1999,105(4):2304-2317. 10.1121/1.426837

  29. 29.

    Svensson UP, Fred RI, Vanderkooy J: An analytic secondary source model of edge diffraction impulse responses. The Journal of the Acoustical Society of America 1999,106(5):2331-2344. 10.1121/1.428071

  30. 30.

    Tsingos N, Funkhouser T, Ngan A, Carlbom I: Modeling acoustics in virtual environments using the uniform theory of diffraction. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), August 2001, Los Angeles, Calif, USA 545–552.

  31. 31.

    Stephenson UM: Beugungssimulation ohne Rechenzeitexplosion: die Methode der quantisierten Pyramidenstrahlen; ein neues Berechnungsverfahren für Raumakustik und Lärmimmissionsprognose; Vergleiche, Ansätze, Lösungen, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 2004.

  32. 32.

    Slater M, Steed A, Chrysanthou Y: Computer Graphics and Virtual Environments: From Realism to Real-Time. Addison Wesley, New York, NY, USA; 2001.

  33. 33.

    Cremer L, Müller HA: Die wissenschaftlichen Grundlagen der Raumakustik—Band 1. 2nd edition. S. Hirzel, Stuttgart, Germany; 1978.

  34. 34.

    Akenine-Möller T, Haines E: Real-Time Rendering. 2nd edition. A. K. Peters, Natick, Mass, USA; 2002.

  35. 35.

    Foley JD, van Dam A, Feiner SK, Hughes JF: Computer Graphics, Principles and Practice. 2nd edition. Addison Wesley, Reading, Mass, USA; 1996.

  36. 36.

    Shumacker R, Brand R, Gilliland M, Sharp W: Study for applying computer-generated images to visual simulations. In Report AFHRL-TR-69-14. U.S. Air Force Human Resources Laboratory, San Antonio, Tex, USA; 1969.

  37. 37.

    Cormen TH, Leiserson CE, Rivest RL, Stein C: Introduction to Algorithms. 2nd edition. MIT Press, Cambridge, Mass, USA; 2001.

  38. 38.

    Schröder D, Lentz T: Real-time processing of image sources using binary space partitioning. Journal of the Audio Engineering Society 2006,54(7-8):604-619.

  39. 39.

    Heinz R: Entwicklung und Beurteilung von computergestützten Methoden zur binauralen Raumsimulation, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 1994.

  40. 40.

    Bradley JS, Soulodre GA: The influence of late arriving energy on spatial impression. The Journal of the Acoustical Society of America 1995,97(4):2263-2271. 10.1121/1.411951

  41. 41.

    Rindel JH: Evaluation of room acoustic qualities and defects by use of auralization. Proceedings of the 148th Meeting of the Acoustical Society of America, November 2004, San Diego, Calif, USA

  42. 42.

    Schröder D, Dross P, Vorländer M: A fast reverberation estimator for virtual environments. Proceedings of the AES 30th International Conference, March 2007, Saariselkä, Finland

  43. 43.

    Brookes T, Treble C: The effect of non-symmetrical left/right recording pinnae on the perceived externalisation of binaural recordings. Proceedings of the 118th Audio Engineering Society Convention, May 2005, Barcelona, Spain

  44. 44.

    Brungart DS, Rabinowitz WM, Durlach NI: Auditory localization of a nearby point source. The Journal of the Acoustical Society of America 1996,100(4):2593.

  45. 45.

    Kulkarni A, Colburn HS: Role of spectral detail in sound-source localization. Nature 1998,396(6713):747-749. 10.1038/25526

  46. 46.

    Lehnert H, Richter M: Auditory virtual environment: simplified treatment of reflections. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, Norway

  47. 47.

    Romanenko G, Vorländer M: Employment of spherical wave reflection coefficient in room acoustics. IoA Symposium Surface Acoustics, 2003, Salford, UK

  48. 48.

    Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93), August 1993, Anaheim, Calif, USA. ACM Press; 135–142.

  49. 49.

    Bauer BB: Stereophonic earphones and binaural loudspeakers. Journal of the Audio Engineering Society 1961,9(2):148-151.

  50. 50.

    Kirkeby O, Nelson PA, Hamada H: Local sound field reproduction using two closely spaced loudspeakers. The Journal of the Acoustical Society of America 1998,104(4):1973-1981. 10.1121/1.423763

  51. 51.

    Møller H: Reproduction of artificial-head recordings through loudspeakers. Journal of the Audio Engineering Society 1989,37(1-2):30-33.

  52. 52.

    Gardner WG: 3-D audio using loudspeakers, Ph.D. thesis. Massachusetts Institute of Technology, Cambridge, Mass, USA; 1997.

  53. 53.

    Lentz T, Schmitz O: Realisation of an adaptive cross-talk cancellation system for a moving listener. Proceedings of the 21st Audio Engineering Society Conference, June 2002, St. Petersburg, Russia

  54. 54.

    Lentz T, Behler GK: Dynamic cross-talk cancellation for binaural synthesis in virtual reality environments. Proceedings of the 117th Audio Engineering Society Convention, October 2004, San Francisco, Calif, USA

  55. 55.

    Steinberg : ASIO 2.0 Audio Streaming Input Output Development Kit. 2004.

  56. 56.

    Lentz T: Dynamic crosstalk cancellation for binaural synthesis in virtual reality environments. Journal of the Audio Engineering Society 2006,54(4):283-294.

  57. 57.

    Takeuchi T, Nelson P, Kirkeby O, Hamada H: The effects of reflections on the performance of virtual acoustic imaging systems. Proceedings of the International Symposium on Active Control of Sound and Vibration (ACTIVE '97), August 1997, Budapest, Hungary 955–966.

  58. 58.

    Ward DB: On the performance of acoustic crosstalk cancellation in a reverberant environment. The Journal of the Acoustical Society of America 2001,110(2):1195-1198. 10.1121/1.1386635

  59. 59.

    Lentz T, Sokoll J, Assenmacher I: Performance of spatial audio using dynamic cross-talk cancellation. Proceedings of the 119th Audio Engineering Society Convention, October 2005, New York, NY, USA

  60. 60.

    Gardner WG: Efficient convolution without input-output delay. Journal of the Audio Engineering Society 1995,43(3):127-136.

  61. 61.

    La Viola JJ Jr.: A testbed for studying and choosing predictive tracking algorithms in virtual environments. Proceedings of the 7th International Immersive Projection Technologies Workshop, 9th Eurographics Workshop on Virtual Environments, May 2003, Zurich, Switzerland 189–198.

  62. 62.

    Azuma R, Bishop G: A frequency-domain analysis of head-motion prediction. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), August 1995, Los Angeles, Calif, USA. ACM Press; 401–408.

  63. 63.

    Chai L, Hoff WA, Vincent T: Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence: Teleoperators and Virtual Environments 2002,11(5):474-492. 10.1162/105474602320935829

  64. 64.

    Wu J-R, Ouhyoung M: A 3D tracking experiment on latency and its compensation methods in virtual environments. In Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST '95), November 1995, Pittsburgh, Pa, USA. ACM Press; 41–49.

  65. 65.

    Witew IB: Spatial variation of lateral measures in different concert halls. Proceedings of the 18th International Congress on Acoustics (ICA '04), April 2004, Kyoto, Japan 4: 2949.

  66. 66.

    Azuma R, Bishop G: Improving static and dynamic registration in an optical see-through HMD. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '94), July 1994, New York, NY, USA. ACM Press; 197–204.

  67. 67.

    Pompetzki W: Psychoakustische Verifikation von Computermodellen zur binauralen Raumsimulation, Ph.D. thesis. Ruhr-Universität Bochum, Bochum, Germany; 1993.

  68. 68.

    Vorländer M, Mommertz E: Definition and measurement of random-incidence scattering coefficients. Applied Acoustics 2000,60(2):187-199. 10.1016/S0003-682X(99)00056-0

  69. 69.

    ISO 354 : Acoustics, Measurement of sound absorption in a reverberant room. 2003.

  70. 70.

    ISO/DIS 17497-1 : Acoustics Measurement of the sound scattering properties of surfaces—part 1: measurement of the randomincidence scattering coefficient in a reverberation room.

  71. 71.

    Tsingos N: Scalable perceptual mixing and filtering of audio signals using an augmented spectral representation. Proceedings of the 8th International Conference on Digital Audio Effects (DAFx '05), September 2005, Madrid, Spain

  72. 72.

    Tsingos N, Gallo E, Drettakis G: Perceptual audio rendering of complex virtual environments. Proceedings of the 31st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '04), August 2004, Los Angeles, Calif, USA 249–258.

Download references

Author information

Correspondence to Tobias Lentz.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Keywords

  • Information Technology
  • Virtual Reality
  • Quantum Information
  • Virtual Environment
  • Reality System