Snap’s Latest Patent Focuses on Hands to Control AR

Snap wants to augment the world around you – including your hands. 

The company is seeking to patent a system for “selecting AR buttons on a hand.” Snap’s patent essentially aims to create more intuitive overlap and interaction between augmented reality content and the real world, specifically between its AR creations and your hands. 

With a set of smart glasses, “The interaction with the embedded sensor to perform various modifications of the virtual content is not very intuitive and has a very steep learning curve,” Snap said in the filing. 

Snap’s system aims to improve the “efficiency, appeal, and utility” of these smart glasses by displaying selectable virtual content on a “first real-world object,” a user’s hand, and then tracking movement of a “second real-world object,” or a user’s other hand or finger, to figure out which content the user selected. 

The system takes into account “spatial relationship factors” between the user’s hands and the displayed AR objects to dynamically adjust the virtual world in response to user movements. If there is an overlap between the user’s second hand and the AR object — meaning, if a user taps a certain selection of the content displayed on their other hand — it will activate whatever content they selected. 

“This improves the accuracy of detection of the correct AR object the user intends to select,” Snap said. 

Snap’s patent activity is filled with AR glasses-related inventions as it seemingly tries to snag as much IP as possible in the growing space. The company has sought to patent ways to bring books to life with AR glasses, gaze-tracking techniques, high-tech prescription lenses, and even AR-enabled contacts. The company also received a $20 million grant from the State of California to expand manufacturing of these devices in December. 

Snap has offered AR Spectacles since 2016, dropping three generations of the glasses. Despite its efforts, the company has struggled to scale adoption of these devices, as most of the people interacting with its AR innovations do so through its flagship app.

The road to creating long-lasting and scalable artificial reality glasses faces a number of obstacles. The devices lose power quickly due to high processing power on a small amount of hardware, they overheat, and the field of view is still quite small. These factors are hard to overcome from an engineering perspective, limiting broader adoption by consumers. 

Snap also has quite a bit of competition as far as AR wearables go. Both Meta and Apple are plugging away at lightweight and comfortable mixed reality headwear, all seemingly to make the experience usable for more than a short burst. Plus, if the diagrams are any indication, the form factor of this invention in particular has a similar look and feel to the Humane AI pin palm projector, potentially presenting another competitor if glasses with this function make it onto people’s faces.