Nvidia, Google Patents Signal Big Tech’s AI Robot Craze

Big Tech wants to make sure its machines don’t crash into you: Both Nvidia and Google filed patents for systems to make their robots more environmentally aware.  

Nvidia is seeking to patent a “prompt generator” for use in machine learning processes that takes cues from its environment. Nvidia’s tech essentially gives machine learning models and autonomous machines tasks in real time, aiming to create a more reactive AI. 

“Certain circumstances can cause less than optimal performance of task planning based on the environment,” Nvidia said. “This approach may be difficult to scale in environments that include many feasible actions and/or many objects.” 

Nvidia’s system would include a large language model that generates prompts for an autonomous machine based on its surroundings. The system then sends those prompts to the machine’s internal task planner to perform instantaneously. For example, if the task is to put away a glass in a cupboard, the system would instruct the robot to complete some intermediary tasks to achieve that goal, such as “go to counter,” “pick up glass,” “open cupboard,” and “place glass in cupboard.”

Meanwhile, Google is seeking to patent robot navigation depending on “gesture(s) of human(s)” in its environment. This system creates a sequence of positions that allow a robot to reach a certain position based on the direction a human operator gestures. 

In response to a certain gesture, such as pointing left or right, Google’s system uses a neural network to create a navigation path based on a series of “position deltas,” which represents how a robot’s position may change based on its surroundings. The system then sends the generated path to the robot’s control system to act on it.  

This tech aims to overcome the issues of conventional “low-level” navigation methods, which “are not reactive to various behaviors of humans or other dynamic object(s) in the environment,” Google said. 

While robots aren’t taking over the world any time soon, tech companies seem intent on creating AI-enabled machines that walk among us. Google and Nvidia’s patents add to several from tech firms that aim to give you mechanical coworkers, many of which are meant for factory and manufacturing settings. For example, Intel sought to patent a robot path planner that senses “occupancy information” of a building, and Amazon filed a patent application that assigned tasks to robots based on how charged they are.

These efforts add to a broader conversation about what the future of humanoid AI robots could look like. Google, Microsoft, and Tesla are already tinkering with fusing AI models and robotics. Plus, AI robotics startup Figure raised $675 million in Series B funding in late February and announced a collaboration with OpenAI.

Nvidia recently joined the host of companies working towards that future, announcing a “moonshot” plan, called Project GR00T, at the company’s annual GTC conference to build a general-purpose foundational model for humanoid robots. 

“The next generation of robotics will likely be humanoid robotics,” Nvidia CEO Jensen Huang said in the conference’s keynote. “We now have the necessary technology to imagine generalized human robotics.”

Widening the range and reactivity of robots could unlock tons of potential and productivity across industries, particularly in labor-intensive industries like manufacturing, supply chain, and logistics. But to broaden robots’ horizons, developers need to ensure that they aren’t going to go haywire and potentially put the safety of their human co-workers at risk.