DeepMind Patent Adds to Robotic AI Ambitions

DeepMind wants to make sure its AI models know the world around them. 

Google’s AI division filed a patent application for a way to generate a “model of a target environment” based on interactions between an autonomous agent and a “source environment.” DeepMind’s system helps AI models improve decision-making by getting a thorough understanding of several different environments, allowing AI models to be more adaptable. 

Controlling an agent, such as a robot or autonomous device, in any environment requires a certain level of uncertainty in decision-making, DeepMind noted. “The agent must navigate the trade-off between learning about reward distributions by exploring the effects of actions versus exploiting the most promising actions based on the current interaction history of the agent with the environment.” 

DeepMind’s system selects a series of actions that an autonomous agent can take in its current environment using what’s called an “environment model,” or a simulation of the agent’s surroundings. 

Using that simulation, as well as historical data about different actions and interactions that the agent has taken, the environment model is “parameterized,” or tuned to give the agent better situational awareness. Additionally, the system takes in data from several different “source environments,” or places that are different from the agent’s location, which provides diverse data to help the model improve its reflexes and generalize its training.

Finally, this system weighs the risk and reward of every action that a robot can take in a given environment, aiming to come up with the most effective outcome. This process is repeated over time, allowing the agent to essentially learn from its mistakes. 

DeepMind’s system essentially aims to let robots explore their options, said Rhonda Dibachi, CEO of Manufacturing-as-a-Service platform HeyScottie. Imagine driving the same route to work every day, but one day it’s blocked by construction. Rather than return to the same route the next day, you explore new routes and learn the trade-offs of different options. “It is a balance between exploring and exploiting,” Dibachi said. 

This kind of decisiveness seems obvious to us as humans, but has to be clearly laid out to a robot or autonomous machine, Dibachi noted. These machines are capable of doing a lot of different actions, but often they can only do what they’re pre-programmed for, rather than reacting or using “probabilistic” decision-making, she said. 

Breaking past this decision-making barrier is vital in further integrating AI-powered robotics throughout operations like logistics, manufacturing, and construction. As it stands, robots exist behind cages in factory settings to limit their range of motion and prevent safety hazards. DeepMind’s patent, however, may “help them get out of the cage,” said Dibachi. 

“If you take those walls down, the dangers multiply infinitely,” said Dibachi. “So what you have to do is go to AI, because it allows the robot to interact with the environment in an open-ended way. This is the next big step.” 

Melding AI and robotics has piqued the interest of many big tech firms in recent months. Several big tech firms have gone after AI-powered robotics patents for things like battery life, action planning, and reactivity, and several startups have been hard at work on bringing all-purpose humanoid robots to market. 

Aside from this patent, DeepMind has similarly been tinkering with robotics. The company announced in January several advances in its robotics research that help with fast decision-making and environmental navigation. And earlier this month, Ayzaan Wahid, a research engineer at the company, posted a series of videos on X showing off ALOHA Unleashed, a project aiming to push the boundaries of dexterity in autonomous robots. The videos featured robotic arms putting shirts on hangers, tying shoes, replacing machine parts, and tightening gears.