Google Patent Makes Controlling Robots Easier

Are you ready for robotic coworkers? 

Google may be able to make collaborating with them a little simpler: The company is seeking to patent a system for “controlling robots using multi-modal language models” that essentially allows the machines to operate based on both language commands and environmental surroundings. 

To do this, Google’s system gathers environmental data collected via sensors, such as cameras or motion detectors, and textual data that dictates commands, into an “embedding space.” There, the data is turned into representations that an AI model can actually use which are then fed to a neural network that outputs commands for the robot to follow. 

On the user end, the tech makes it easier for humans to control and interact with AI-powered robots. Google noted its system “allows a robot to be controlled by a user using natural language input, without being limited to particular predefined command words or phrases in the input.” 

This isn’t the first time Google has taken an interest in robotics. The company previously sought to patent a system for simulating robotic training data to train real-world systems. And in March, the company introduced AI models specifically tailored to robotics.

Google isn’t the only company keeping pace with the AI robotics industry, though. Intel, Nvidia, Amazon and more have all sought to lock down IP in the rapidly expanding space. Additionally, Amazon last week announced a new robot called Vulcan, which it claims has a sense of touch. And as humanoid robots enter the chat, competitors such as Figure AI and Tesla’s Optimus robots are garnering more attention.  

One of the biggest obstacles standing in the way of human-robot harmony is safety, however. The closer humans work with autonomous machines, the more dangers arise. Starting with automation of tedious tasks – and giving these machines more data and environmental context as this patent intends to – could reduce the risk of error.