Faster sound propagation algorithm could lead to more realistic VR

Cal Jeffrey

TS Evangelist
Staff member

Traditional sound propagation modeling can take hours or even days of computational time, making it useless in virtual reality due to the unscripted nature of the platform. However, researchers at Stanford University have developed an algorithm that can perform the calculations needed in seconds. It is not close enough to real-time to be able to produce noises as they happen in VR, but it could be used to pre-render sounds on-the-fly.

“Making it easier to create models makes it practical to build interactive environments with realistic sound effects,” said Stanford Professor of Computer Science Doug James.

Up until now, sound propagation models relied on algorithms based on work from 19th-century scientist Herrmann von Helmholtz. Traditional modeling uses Helmholtz equation and the boundary element method (BEM), which takes a very long time to render. James and one of his grad students, Jui-Hsien Wang, threw out the old Helmholtz model and focused on work done by the 20th-century Austrian composer Fritz Heinrich Klein.

Klein combined twelve piano notes into a single chord called the Mutterakkord (Mother chord) for his 1921 composition Die Maschine. James and Wang applied a similar approach to their algorithm, which they dubbed KleinPAT.

“[The] Mother chord is arranged so that it contains one instance of each interval within the octave,” James and Wang wrote in their paper, KleinPAT: Optimal Mode Conflation For Time-Domain Precomputation Of Acoustic Transfer. “Our KleinPAT algorithm optimally arranges different modal tones of a vibrating 3D object into chords, which are then played together by a time-domain vector wavesolver in order to efficiently estimate all acoustic transfer fields.”

The research is still considered to be in its early phase and requires specialized equipment. So it could be some time before we see it applied in commercial VR rigs. However, Stanford's GPU-boosted system can compute the same sounds as BEM modeling, except hundreds or even thousands of times faster.

Image credit: The Reality of Design

Permalink to story.



This is pretty amazing. It should be noted that this isn't just useful for VR, it's useful for video games in general. It allows game devs to give objects in their game sound propagation dynamics by pre-baking some files. No longer would you have 1 sound for a cup breaking, it would be completely dependent on where the cup was dropped. The same applies to every object in the game. Breaking vases in Zelda? Each vase should generate a different sound based on where it hit it.

This in addition to binaural audio would be pretty sweet.


TS Member
The product should be usable across a huge range of AI. If sound then why not speech realism? An end to 'Hawkins speak'?
BTW 'KleinPAT: Optimal Mode Conflation For Time-Domain Precomputation Of Acoustic Transfer' - snappy title guys!.


TS Booster
Not something I've been aware of before but very interesting. I wonder if GPU accelerated BEM method would be comparable in speed. I see there has been some research done but nothing I can find by way of comparison. I can imagine GPU vs non-GPU accelerated could well be a large portion of the 3000x speed up.