Open Access

Virtual Reality System with Integrated Sound Field Simulation and Reproduction

  • Tobias Lentz1Email author,
  • Dirk Schröder1,
  • Michael Vorländer1 and
  • Ingo Assenmacher2
EURASIP Journal on Advances in Signal Processing20072007:070540

https://doi.org/10.1155/2007/70540

Received: 1 May 2006

Accepted: 3 January 2007

Published: 13 March 2007

Abstract

A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.

Keywords

Information TechnologyVirtual RealityQuantum InformationVirtual EnvironmentReality System

[123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172]

Authors’ Affiliations

(1)
Institute of Technical Acoustics, RWTH Aachen University, Aachen, Germany
(2)
Virtual Reality Group, RWTH Aachen University, Aachen, Germany

References

  1. Begault DR: Challenges to the successful implementation of 3-D sound. Journal of the Audio Engineering Society 1991,39(11):864-870.Google Scholar
  2. Naef M, Staadt O, Gross M: Spatialized audio rendering for immersive virtual environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '02), November 2002, Hong Kong 65-72.View ArticleGoogle Scholar
  3. Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC: The CAVE: audio visual experience automatic virtual environment. Communications of the ACM 1992,35(6):65-72.View ArticleGoogle Scholar
  4. Burgess DA, Verlinden JC: An architecture for spatial audio servers. Proceedings of Virtual Reality Systems Conference (Fall '93), November 1993, New York, NY, USAGoogle Scholar
  5. Mulder JD, Dooijes EH: Spatial audio in graphical applications. In Visualization in Scientific Computing. Edited by: Göbel M, Müller H, Urban B. Springer, Wien, Austria; 1994:215-229.Google Scholar
  6. Lake Huron 2005.http://www.lake.com.au/
  7. Savioja L: Modeling Techniques for Virtual Acoustics, Ph.D. thesis. Helsinki University of Technology, Helsinki, Finland; 1999.Google Scholar
  8. Savioja L, Huopaniemi J, Lokki T, Väänänen R: Creating interactive virtual acoustic environments. Journal of the Audio Engineering Society 1999,47(9):675-705.Google Scholar
  9. Funkhouser T, Min P, Carlbom I: Real-time acoustic modeling for distributed virtual environments. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 365-374.View ArticleGoogle Scholar
  10. Storms RL: Npsnet-3D Sound Server: An Effective Use of the Auditory Channel. 1995.Google Scholar
  11. Kuttruff H: Room Acoustics. 4th edition. Elsevier Science Publisher, New York, NY, USA; 2000.Google Scholar
  12. Allen JB, Berkley DA: Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 1979,65(4):943-950. 10.1121/1.382599View ArticleGoogle Scholar
  13. Borish J: Extension of the image model to arbitrary polyhedra. The Journal of the Acoustical Society of America 1984,75(6):1827-1836. 10.1121/1.390983View ArticleGoogle Scholar
  14. Dalenbäck B-IL: Room acoustic prediction based on a unified treatment of diffuse and specular reflection. The Journal of the Acoustical Society of America 1996,100(2):899-909. 10.1121/1.416249View ArticleGoogle Scholar
  15. Forsberg P-A: Fully discrete ray tracing. Applied Acoustics 1985,18(6):393-397. 10.1016/0003-682X(85)90021-0View ArticleGoogle Scholar
  16. Funkhouser T, Tsingos N, Carlbom I, et al.: A beam tracing method for interactive architectural acoustics. The Journal of the Acoustical Society of America 2004,115(2):739-756. 10.1121/1.1641020View ArticleGoogle Scholar
  17. Naylor GM: ODEON—another hybrid room acoustical model. Applied Acoustics 1993,38(2–4):131-143.View ArticleGoogle Scholar
  18. Stephenson UM: Quantized pyramidal beam tracing—a new algorithm for room acoustics and noise immission prognosis. Acta Acustica United with Acustica 1996,82(3):517-525.MathSciNetGoogle Scholar
  19. van Maercke D: Simulation of sound fields in time and frequency domain using a geometrical model. Proceedings of the 12th International Congress on Acoustics (ICA '86), July 1986, Toronto, Ontario, Canada 2: paper E11-7Google Scholar
  20. Vorländer M: Simulation of the transient and steady state sound propagation in rooms using a new combined sound particle—image source algorithm. The Journal of the Acoustical Society of America 1989, 86: 172-178. 10.1121/1.398336View ArticleGoogle Scholar
  21. Bork I: A comparison of room simulation software—the 2nd round Robin on room acoustical computer simulation. Acta Acustica United with Acustica 2000,86(6):943-956.Google Scholar
  22. Vorländer M: International round Robin on room acoustical computer simulations. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, Norway 689-692.Google Scholar
  23. Kuttruff H: A simple iteration scheme for the computation of decay constants in enclosures with diffusely reflecting boundaries. The Journal of the Acoustical Society of America 1995,98(1):288-293. 10.1121/1.413727View ArticleGoogle Scholar
  24. Christensen CL, Rindel JH: A new scattering method that combines roughness and diffraction effects. Forum Acousticum, 2005, Budapest, HungaryGoogle Scholar
  25. Heinz R: Binaural room simulation based on an image source model with addition of statistical methods to include the diffuse sound scattering of walls and to predict the reverberant tail. Applied Acoustics 1993,38(2–4):145-159.View ArticleGoogle Scholar
  26. Lam YW: A comparison of three reflection modelling methods used in room acoustics computer models. The Journal of the Acoustical Society of America 1996,100(4):2181-2192. 10.1121/1.417927View ArticleGoogle Scholar
  27. Vorländer M: Ein Strahlverfolgungsverfahren zur Berechnung von Schallfeldern in Räumen. Acustica 1988,65(3):138-148.Google Scholar
  28. Suh JS, Nelson PA: Measurement of transient response of rooms and comparison with geometrical acoustic models. The Journal of the Acoustical Society of America 1999,105(4):2304-2317. 10.1121/1.426837View ArticleGoogle Scholar
  29. Svensson UP, Fred RI, Vanderkooy J: An analytic secondary source model of edge diffraction impulse responses. The Journal of the Acoustical Society of America 1999,106(5):2331-2344. 10.1121/1.428071View ArticleGoogle Scholar
  30. Tsingos N, Funkhouser T, Ngan A, Carlbom I: Modeling acoustics in virtual environments using the uniform theory of diffraction. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), August 2001, Los Angeles, Calif, USA 545-552.View ArticleGoogle Scholar
  31. Stephenson UM: Beugungssimulation ohne Rechenzeitexplosion: die Methode der quantisierten Pyramidenstrahlen; ein neues Berechnungsverfahren für Raumakustik und Lärmimmissionsprognose; Vergleiche, Ansätze, Lösungen, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 2004.Google Scholar
  32. Slater M, Steed A, Chrysanthou Y: Computer Graphics and Virtual Environments: From Realism to Real-Time. Addison Wesley, New York, NY, USA; 2001.Google Scholar
  33. Cremer L, Müller HA: Die wissenschaftlichen Grundlagen der Raumakustik—Band 1. 2nd edition. S. Hirzel, Stuttgart, Germany; 1978.Google Scholar
  34. Akenine-Möller T, Haines E: Real-Time Rendering. 2nd edition. A. K. Peters, Natick, Mass, USA; 2002.Google Scholar
  35. Foley JD, van Dam A, Feiner SK, Hughes JF: Computer Graphics, Principles and Practice. 2nd edition. Addison Wesley, Reading, Mass, USA; 1996.MATHGoogle Scholar
  36. Shumacker R, Brand R, Gilliland M, Sharp W: Study for applying computer-generated images to visual simulations. In Report AFHRL-TR-69-14. U.S. Air Force Human Resources Laboratory, San Antonio, Tex, USA; 1969.Google Scholar
  37. Cormen TH, Leiserson CE, Rivest RL, Stein C: Introduction to Algorithms. 2nd edition. MIT Press, Cambridge, Mass, USA; 2001.MATHGoogle Scholar
  38. Schröder D, Lentz T: Real-time processing of image sources using binary space partitioning. Journal of the Audio Engineering Society 2006,54(7-8):604-619.Google Scholar
  39. Heinz R: Entwicklung und Beurteilung von computergestützten Methoden zur binauralen Raumsimulation, Ph.D. thesis. RWTH Aachen University, Aachen, Germany; 1994.Google Scholar
  40. Bradley JS, Soulodre GA: The influence of late arriving energy on spatial impression. The Journal of the Acoustical Society of America 1995,97(4):2263-2271. 10.1121/1.411951View ArticleGoogle Scholar
  41. Rindel JH: Evaluation of room acoustic qualities and defects by use of auralization. Proceedings of the 148th Meeting of the Acoustical Society of America, November 2004, San Diego, Calif, USAGoogle Scholar
  42. Schröder D, Dross P, Vorländer M: A fast reverberation estimator for virtual environments. Proceedings of the AES 30th International Conference, March 2007, Saariselkä, FinlandGoogle Scholar
  43. Brookes T, Treble C: The effect of non-symmetrical left/right recording pinnae on the perceived externalisation of binaural recordings. Proceedings of the 118th Audio Engineering Society Convention, May 2005, Barcelona, SpainGoogle Scholar
  44. Brungart DS, Rabinowitz WM, Durlach NI: Auditory localization of a nearby point source. The Journal of the Acoustical Society of America 1996,100(4):2593.View ArticleGoogle Scholar
  45. Kulkarni A, Colburn HS: Role of spectral detail in sound-source localization. Nature 1998,396(6713):747-749. 10.1038/25526View ArticleGoogle Scholar
  46. Lehnert H, Richter M: Auditory virtual environment: simplified treatment of reflections. Proceedings of the 15th International Congress on Acoustics (ICA '95), June 1995, Trondheim, NorwayGoogle Scholar
  47. Romanenko G, Vorländer M: Employment of spherical wave reflection coefficient in room acoustics. IoA Symposium Surface Acoustics, 2003, Salford, UKGoogle Scholar
  48. Cruz-Neira C, Sandin DJ, DeFanti TA: Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93), August 1993, Anaheim, Calif, USA. ACM Press; 135-142.View ArticleGoogle Scholar
  49. Bauer BB: Stereophonic earphones and binaural loudspeakers. Journal of the Audio Engineering Society 1961,9(2):148-151.Google Scholar
  50. Kirkeby O, Nelson PA, Hamada H: Local sound field reproduction using two closely spaced loudspeakers. The Journal of the Acoustical Society of America 1998,104(4):1973-1981. 10.1121/1.423763View ArticleGoogle Scholar
  51. Møller H: Reproduction of artificial-head recordings through loudspeakers. Journal of the Audio Engineering Society 1989,37(1-2):30-33.Google Scholar
  52. Gardner WG: 3-D audio using loudspeakers, Ph.D. thesis. Massachusetts Institute of Technology, Cambridge, Mass, USA; 1997.Google Scholar
  53. Lentz T, Schmitz O: Realisation of an adaptive cross-talk cancellation system for a moving listener. Proceedings of the 21st Audio Engineering Society Conference, June 2002, St. Petersburg, RussiaGoogle Scholar
  54. Lentz T, Behler GK: Dynamic cross-talk cancellation for binaural synthesis in virtual reality environments. Proceedings of the 117th Audio Engineering Society Convention, October 2004, San Francisco, Calif, USAGoogle Scholar
  55. Steinberg : ASIO 2.0 Audio Streaming Input Output Development Kit. 2004.Google Scholar
  56. Lentz T: Dynamic crosstalk cancellation for binaural synthesis in virtual reality environments. Journal of the Audio Engineering Society 2006,54(4):283-294.Google Scholar
  57. Takeuchi T, Nelson P, Kirkeby O, Hamada H: The effects of reflections on the performance of virtual acoustic imaging systems. Proceedings of the International Symposium on Active Control of Sound and Vibration (ACTIVE '97), August 1997, Budapest, Hungary 955-966.Google Scholar
  58. Ward DB: On the performance of acoustic crosstalk cancellation in a reverberant environment. The Journal of the Acoustical Society of America 2001,110(2):1195-1198. 10.1121/1.1386635View ArticleGoogle Scholar
  59. Lentz T, Sokoll J, Assenmacher I: Performance of spatial audio using dynamic cross-talk cancellation. Proceedings of the 119th Audio Engineering Society Convention, October 2005, New York, NY, USAGoogle Scholar
  60. Gardner WG: Efficient convolution without input-output delay. Journal of the Audio Engineering Society 1995,43(3):127-136.Google Scholar
  61. La Viola JJ Jr.: A testbed for studying and choosing predictive tracking algorithms in virtual environments. Proceedings of the 7th International Immersive Projection Technologies Workshop, 9th Eurographics Workshop on Virtual Environments, May 2003, Zurich, Switzerland 189-198.Google Scholar
  62. Azuma R, Bishop G: A frequency-domain analysis of head-motion prediction. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), August 1995, Los Angeles, Calif, USA. ACM Press; 401-408.View ArticleGoogle Scholar
  63. Chai L, Hoff WA, Vincent T: Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence: Teleoperators and Virtual Environments 2002,11(5):474-492. 10.1162/105474602320935829View ArticleGoogle Scholar
  64. Wu J-R, Ouhyoung M: A 3D tracking experiment on latency and its compensation methods in virtual environments. In Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology (UIST '95), November 1995, Pittsburgh, Pa, USA. ACM Press; 41-49.View ArticleGoogle Scholar
  65. Witew IB: Spatial variation of lateral measures in different concert halls. Proceedings of the 18th International Congress on Acoustics (ICA '04), April 2004, Kyoto, Japan 4: 2949.Google Scholar
  66. Azuma R, Bishop G: Improving static and dynamic registration in an optical see-through HMD. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '94), July 1994, New York, NY, USA. ACM Press; 197-204.View ArticleGoogle Scholar
  67. Pompetzki W: Psychoakustische Verifikation von Computermodellen zur binauralen Raumsimulation, Ph.D. thesis. Ruhr-Universität Bochum, Bochum, Germany; 1993.Google Scholar
  68. Vorländer M, Mommertz E: Definition and measurement of random-incidence scattering coefficients. Applied Acoustics 2000,60(2):187-199. 10.1016/S0003-682X(99)00056-0View ArticleGoogle Scholar
  69. ISO 354 : Acoustics, Measurement of sound absorption in a reverberant room. 2003.Google Scholar
  70. ISO/DIS 17497-1 : Acoustics Measurement of the sound scattering properties of surfaces—part 1: measurement of the randomincidence scattering coefficient in a reverberation room.Google Scholar
  71. Tsingos N: Scalable perceptual mixing and filtering of audio signals using an augmented spectral representation. Proceedings of the 8th International Conference on Digital Audio Effects (DAFx '05), September 2005, Madrid, SpainGoogle Scholar
  72. Tsingos N, Gallo E, Drettakis G: Perceptual audio rendering of complex virtual environments. Proceedings of the 31st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '04), August 2004, Los Angeles, Calif, USA 249-258.Google Scholar

Copyright

© Tobias Lentz et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.