Snap Patent Could Use Your Phone to Make AR Glasses Lighter

Snap may want to lighten the load on its AR glasses. 

The social media firm is seeking to patent “external computer vision of an eyewear device.” Snap’s tech relies on external devices for motion tracking and processing of AR experiences for higher-quality interactions and reduced computational needs. 

Interaction with sensors in typical smart glasses is “not very intuitive and has a very steep learning curve,” Snap said. “Also, performing image processing to detect hand gestures involves multiple machine learning models, which consumes a great deal of hardware resources of the smart glasses.” 

Snap’s tech gives the job of movement and gesture detection to a user’s smartphone. The user’s phone and the glasses are in constant communication, so the visuals in the eyewear correspond with the users’ movements and interactions with AR objects in their vicinity. 

In the patent’s primary example, a user would set up their phone so that its front camera captures their movements. Those movements would then be mimed in real time by an AR avatar pictured in the user’s display. 

Snap isn’t the only company that has sought to lighten the load on AR headsets. Meta has filed a number of patents that aim to move processing and tracking responsibilities to external devices. For any company developing AR, a sleeker, longer-last form factor is the “end goal,” said Jake Maymar, AI strategist at The Glimpse Group.

“If it costs almost no energy to run the actual glasses and everything is offset to an external phone, you can have very lightweight batteries … and the cost of manufacturing the glasses can be very inexpensive,” he said.

Cost and wearability remain the two biggest hurdles in making true AR glasses a reality, said Maymar. Both Snap and Meta unveiled their latest smart glasses last month, and both face those issues. While Snap’s new glasses in the Spectacles line are more powerful than previous models, they’re also quite large and bulky. Meanwhile, Meta’s Project Orion AR glasses are slightly sleeker, but reportedly cost $10,000 per unit to make. 

Creating a lighter and cheaper headset may only be half the battle, said Maymar. To get people to actually use these devices, they need to seamlessly integrate into a user’s existing tech and everyday activities, he said. This is where a company like Apple — which only jumped into artificial reality at the start of this year with the Vision Pro — may have a leg-up. 

Even if Apple isn’t first, if the company puts out AR glasses, Maymar said, “ultimately, Apple is going to win. Just think: How many people have iPhones? That’s a pretty clear indicator.”