Nvidia’s Latest Patent Gives Self-Driving Robots Better Eyes

Nvidia wants to give its neural networks some depth perception. 

The company is seeking to patent techniques to “increase inference accuracy” with respect to autonomous machines. Nvidia’s method essentially allows neural networks to identify objects irrespective of their “size or perspective.” 

Neural networks cannot always identify objects if they are not “specifically trained to perform object detection for images having various objects with different size,” Nvidia said. “For example, objects that are far from said camera will appear smaller than other objects that are close to said camera.” 

Nvidia’s tech aims to overcome this problem by cutting images up into several parts, making it easier for the neural network to understand. The system determines how exactly to split up each image based on several factors: resolution of the object in the image, size required for processing by the neural network itself, “region of interest” within the image, and “user-defined parameters” (meaning, what the user wants the system to look for). 

The system would then localize and combine specific portions of the image for the neural network to get a clearer picture of what exactly it’s looking at, “resulting in better inference accuracy,” Nvidia said.  

To paint a picture, in the context of an autonomous vehicle, if it spots a hazard in the road from far away, a normal system may not detect it as a hazard at first, as it looks small on camera. But this system would be able to detect the hazard from farther away by separating it from the negative space around it. 

Nvidia’s wide-ranging patent puts this tech in the context of autonomous vehicles, as tech like this could help make self-driving cars’ decision-making skills more accurate. The company has a number of patent filings that aim to make the models in its vehicles sharper, including tech that automatically retrains image models from different perspectives, systems to perform fault detection, and ways to generate synthetic training data sets for visual AI. 

But similar to its previous patents in this realm, these all have far broader applications than simply driverless vehicles. All of these can be helpful tools for strengthening computer vision software generally to make object detection and task performance more accurate and efficient. This can be put to use in a host of different settings, from factory floors to retail environments to physical security. 

This also adds to the growing pile of AI software that Nvidia may be seeking to shore up as its AI chips growth continues at a breakneck pace.The company reported record quarterly revenue last week of $22.1 billion, up 265% year over year. Its data center revenue, which includes its AI chip sales, made up $18.4 billion of those earnings, up 409% from last year. The news shot Nvidia’s market cap up by $277 billion on Thursday, and it briefly hit $2 trillion on Friday.

Demand for its chips have reached such a fever pitch that the company has to carefully consider how to “fairly” dole them out, CEO Jensen Huang said on the company’s earnings call. “We do the best we can to allocate fairly, and to avoid allocating unnecessarily.” 

While the company’s chip sales are what continue to make headlines, its software ecosystem is nothing to sneeze at, especially as its patent filings continue to cover more ground. Nvidia’s AI software research could help it expand its horizons as the AI industry clambers to get a hold of its hardware — and potentially looks for cheaper alternatives.