Meta may use AI to Build its Metaverse

The patent shows that, despite the company’s public interest in large language models, it’s still hard at work on artificial reality.

Photo by Maxim Hopman on Unsplash

Sign up to uncover the latest in emerging technology.

Meta is tired of building the metaverse. It may give the task to AI instead. 

The company filed a wide-ranging patent application for “AI curation and customization” of extended reality environments. This filing goes into several ways this could work in practice, but at its core, the system uses AI to let users build their own extended reality objects, no technical design experience necessary. 

The first implementation is a way to create a “customized pet avatar” by using AI to analyze photographs and videos of a user’s actual pet to determine their physical and behavioral features. The system then either recreates the pet in virtual reality from “predetermined graphical pet models,” or it modifies a generic graphical model to look like your pet. 

Next up, Meta details a way to generate VR travel itineraries using historical trip data of “the paths of past users in a particular area.” Essentially, an AI model takes in photos, videos or reviews taken by past users of popular travel destinations in a particular area. The system then identifies common walking paths in the area, and visualizes it in a VR environment for the user. Imagine a VR version of Yelp. 

Keep in mind: these concepts can likely be applied in lots of other ways than just travel itineraries and pets. The basis of it is simply a method to create VR content from real-life things by throwing image data at AI.

Photo via the U.S. Patent and Trademark Office.

The final example, however, seemingly doesn’t require user image data at all: The company’s filing describes a “virtual object builder” for extended reality, which takes users’ verbal commands or gestures to generate objects at their whim with AI. If a user’s command is ambiguous, the system will present the user with several options for objects to build. 

Photo via the U.S. Patent and Trademark Office.

Part of this Franken-patent references a project that Meta’s research lab actually unveiled last year: Builder Bot, a prototype of an AI tool that lets users speak virtual objects into existence. In its debut, a 3D cartoonish bust of CEO Mark Zuckerberg prompted the bot with “Let’s go to the beach,” to which it surrounded him with a low-resolution vision of sand and water. Since then, the company has shared no news of the project, but this filing gives a peek at how the potential feature could work. 

Publicly, Meta has shifted its focus to AI, working on putting AI and chatbots everywhere and reportedly working on a large language model that’s several times more powerful than Llama 2, its commercial product. But if the company’s patent activity is any indication, Meta is still plugging away at the metaverse. 

While many have written the concept off, the company continues to go after patents for AR, VR and XR-related hardware and software inventions, including things like eye-tracking capabilities, methods to reduce glitching and latency, and ways to track advertising engagement in virtual experiences.  Plus, with the arrival of the Meta Quest 3 and the company’s Ray-Ban smart glasses debuted at Meta Connect this week, artificial reality still seems a big part of the company’s vision. 

Meta’s Head of Global Affairs told reporters in March that, while it may take time to come to fruition, the company still believes in the metaverse, and “all the early evidence suggests that something like this will be the heart of the new computing platform.”
However, the company’s metaverse bet isn’t doing so hot in the earnings department. Its Reality Labs unit reported a $3.7 billion operating loss and just $276 million in revenues in the latest quarter. The division has also lost more than $21 billion since the start of 2022. And the news that it’s working on an ad campaign marketing the VR headsets as work tools may also suggest that, though Meta’s not giving up on artificial reality completely, they might at least be pivoting the demographic they’re targeting.