Snap Continues Smart Glasses Push with Energy-Efficient Processing Patent

Snap wants to make sure your smart glasses aren’t overworked.

The company filed a patent application for “adaptive image processing” for an augmented reality device. Snap’s system improves the accuracy and energy efficiency of a set of AR glasses by adjusting how these devices process images. 

When capturing an image in a set of AR glasses, Snap’s tech first identifies a “region of interest” within that image. The region of interest may be a person or object within the wearer’s field of view, for example. The system then tracks that person or object’s position relative to the rest of the environment. 

However, in order to track that person or object properly, the object-tracking tech will often need an image that is cropped to the subject and scaled to a certain size — a process that can be quite “computationally expensive” when performed indiscriminately for every image. 

This is where Snap’s innovation comes in: This system dynamically adjusts the “crop-and-scale order” based on the specific needs of each frame. Snap’s tech takes in several other types of data, such as the object’s velocity or the motion of the AR device itself, to adapt image processing needs moment by moment. 

The outcome is more accurate object tracking that avoids burning through tons of power via constant on-device image processing. 

It’s no secret that Snap is working to be a power player in the nascent AR headset market. The company has filed tons of previous patents related to the tech, including ways to track hand motions to control AR objects, prescription AR glasses, and even mixed-reality contact lenses. 

But the company isn’t the only one that sees promise in the market. Google, Samsung, and Qualcomm are working together on their own set of mixed-reality glasses. Meta has long been seeking to build a pair of lightweight AR glasses, and recently extended its partnership with Ray-Ban through 2030 for “fashionable” smart glasses.

And while it’s been a hot minute since Snap has put a new pair of its Spectacles on the market, the company unveiled a set of specs, as well as a new operating system for them, at its annual Partner Summit last week. The glasses, which mark the fifth generation of the company’s Spectacles endeavor, feature a richer and more immersive display as well as a longer-lasting battery. 

“It is truly amazing how much capability we packed into this device,” Snap CEO Evan Spiegel said at the event. 

Snap, however, is taking a slightly different approach with its Spectacles than competitors. Rather than focusing on a broader audience, the glasses are targeted specifically at developers, similar to the 2021 version of the Spectacles. The company is asking developers to apply to access the glasses, and pay $99 per month to rent a pair for a year.