Visualising the invisible: the sound of real and virtual worlds

Originally posted on Unreal Engine website

Real-time technology is inevitably associated with the instantaneous rendering of visuals, for example the stunning backdrops of The Mandalorian or photorealistic digital humans.

But game engines like Unreal Engine are also a powerful tool for simulating and visualising an invisible energy: sound. This expertise can be applied in interesting ways for the architecture, engineering and construction (AEC) industry.

Most audio simulation efforts use specialised tools that can take hours of processing to produce a result. These are not real-time simulations, as any tweak of the parameters would require hours of reprocessing. This is analogous to the situation in rendering where using an offline path-tracer to compute the bouncing of light in a scene can take minutes or hours per frame.

In a similar way, sound simulators compute how sound waves are reflected and refracted by the material they encounter. The results from these simulators can be brought into Unreal Engine to explore the soundscape using the engine’s graphics capability along with its sound capabilities. Cundall is one of the companies currently exploring the use of the advanced sound capabilities in Unreal Engine for real-time simulation.

It won’t be long before real-time audio simulations can be used to demonstrate how different surface finishes facilitate speech intelligibility in a lecture theatre, how music sounds in an auditorium, or how the choice of finishes in a shopping mall drives a sense of calm.

Real-time technology for real-world acoustics

Andrew Parkin is a Partner at Cundall and as an acoustics consultant, his role is to help clients and stakeholders understand how sound will behave in spaces that are often yet to be built or refurbished. “By optimizing the accuracy of acoustic predictions during a building design process, we’re able to minimise the disconnect between a client’s expectations and eventual reality—game engines have a key part to play, as they help us do this,” he explains.

When Cundall is designing a space, its advice relates to everything from internal finishes to the geometry of the room. The client or stakeholders need to understand the reason behind decisions in order to justify Cundall’s recommendations.

Recommendations based on technical data, such as numerical reverberation time, mean little to anyone except acousticians or particularly informed laypeople. The ability to communicate in an experiential way instead can provide a huge step towards full engagement.

This is where Cundall’s real-time experiential system, Virtual Acoustic Reality (VAR), comes in. The company’s original system for simulating audio uses CATT Acoustic for audio prediction, the CATT-Walker plugin for walkthroughs, a game engine for front-end graphics, and Oculus Rift as a visual out, with a high-quality soundcard and noise-cancelling headphones for audio.

While the accuracy of the audio simulations this system produces is high, it is also clunky, slow, and very processor hungry. The system requires pre-analysis of a space based on set internal finishes. Every time a change is made to the design, the model needs to be taken offline and re-calculated, which takes significant time and effort, and in the world of consultancy, time is money.

This is fine for presentation of a final design but is not conducive to an iterative design process where changes are often made and various options need to be explored and discounted quickly. Being able to review how these changes will sound in a dynamic, accurate fashion can greatly enhance the design and engagement process.

Dynamic audio simulations are so complex, however, it is not currently possible to use them accurately in real time, hence the requirement to pre-bake. By moving away from traditional acoustic modelling software to a more gamified solution, Cundall hopes to optimise processor use by rationalising processes, enabling more to be done in real time.

The ultimate goal is to get real-time simulations as close in accuracy to proprietary software as possible, with minimal or zero pre-baking, along with the ability to make changes to the model on the fly and get results instantaneously, rather than up to 24 hours later, as is the case at present.

It’s with this in mind that Cundall has been experimenting with a new updated system based on Unreal Engine. “We would ideally like to streamline the system so that it uses a single platform, Unreal Engine, which will make it quicker and more stable,” says Parkin.

The Unreal Marketplace was Cundall’s gateway into the engine. “The Marketplace was the instigating element that brought us into the Unreal world as a business, initially utilising third-party plugins to research the potential of the engine for development of the VAR system,” explains Parkin.

As acousticians, rather than computer programmers, the Blueprint visual scripting system has given Parkin’s team the ability to use virtually the full range of concepts and tools generally only available to programmers. “The Blueprint visual scripting system has proven very intuitive to understanding the mechanics of creating interactive content,” he says. “This is especially important for acoustic consultants such as ourselves that may not have a coding background.”

Cundall’s experimentation with Unreal Engine as a platform for acoustic simulation is only just beginning. Early tests have shown huge benefits in speed and performance, with further work needed to match the levels of accuracy achieved with the old system. “The ultimate goal for us would be to have a virtual model of a building that accurately portrays how a building or room will sound when in the design stage,” says Parkin. “Being then able to press a button and have the internal finishes, or room shape change, with a corresponding instant change in acoustic conditions, would be awesome.”

Improving auditory experiences for the future

Cundall is at the cutting edge of sound simulation and visualisation. Their work illustrates how real-time technology can be a powerful tool to improve immersion in virtual worlds and enhance auditory experiences in the real one.

As pioneers like these further explore the potential of real-time technology for creating auditory experiences, we’re sure to see more immersive virtual worlds and better-designed urban spaces in the future.

For more information about Cundall’s acoustic capabilities touch with Andrew Parkin.