Snap Plays Catch Up to Apple, Meta with Spatial Audio AR Filing

Snap wants to sync up sight and sound. 

The social media firm is seeking to patent a system for “augmented reality spatial audio.” Snap’s patent details a system that localizes audio to where an AR object is when wearing a pair of smart glasses. 

Essentially, Snap aims to create a more immersive AR experience using sensor data that’s collected via cameras on their smart glasses. The system then feeds that image data to an AI model to get an understanding of the objects in the user’s surrounding environment, as well as the user’s position within the environment. 

The system would then identify objects in the user’s surroundings to overly digital objects over them (or just choose a random location that makes sense to display a virtual object). Snap’s glasses would then monitor the wearer’s position throughout the space, and adjust the display accordingly. For visuals, it may change the animation as a user moves closer or farther away from the virtual object. 

The audio, meanwhile, may get louder as a user gets closer, or proximity may trigger certain sounds to play. The speakers that sit near each of the user’s ears would each have “directional audio zones,” Snap note, which play audio at different volumes depending on where they are in relation to the virtual object. 

For example, if Snaps’ AR glasses displayed a virtual butterfly moving around the user’s space, it may reflect that in the audio by making it sound as if it’s passing by one ear. 

A user can also manipulate these virtual objects with gestures, and the sound will change as the user makes changes. 

Though Snap’s patent history is littered with filings for creating interactive AR visuals, audio is a major part of creating realistic and immersive user experiences in extended reality, said Jake Maymar, AI strategist at The Glimpse Group. Though virtual objects and animations can create visual interest in AR and VR, seeing isn’t always believing, Maymar noted.

“Seventy-five percent of an experience is sound,” said Maymar. “And with spatial audio, you can completely change your world just with sound. It makes sense that (Snap) would really start leveraging spatial audio.” 

Along with creating a better user experience, spatial audio could help Snap create more immersive advertising, Maymar noted. Finding ways to draw in advertisers certainly wouldn’t hurt the social media firm as digital ad spend continues its downward trajectory. Though Snap reported an uptick in revenue of about 5% in the most recent quarter, the company has seen six straight quarters of either single-digit growth or sales declines. 

While the company may be trying to figure out its sound, plenty of other Big Tech firms have already beat Snap to the punch. Apple, for example, already has spatial audio available in several of its devices, including its Airpods and the Vision Pro mixed reality headset. The Smart Glasses borne from Meta’s recent partnership with Ray-Ban also include support for spatial audio. So with a patent application like this, Maymar noted, Snap is essentially “playing a little bit of catch up.” 

Plus, while Snap’s application was updated most recently in December, other companies’ patents for spatial audio experiences have definitely come before it. Similar to how almost every company in extended reality has a different way of doing eye and gaze tracking, Snap will have to make sure its tech is different enough from others to avoid running into patent legal trouble, Maymar noted.