Google May Bring AI to YouTube Shopping

The filing adds to its AI-powered ad tech, and fuels growing competition to make social media shoppable.

Photo of a Google patent
Photo via U.S. Patent and Trademark Office

Sign up to uncover the latest in emerging technology.

Google wants to make it even easier to shop from your favorite YouTubers. 

The company is seeking to patent “product identification in media items.” Google’s filing details a system that uses machine learning to detect the presence of specific products in image and video data. 

“It can take a significant amount of time and computing resources for a user to find information about products covered by a content item,” Google said in the filing. “There may be obstacles that cause it to be difficult, time-consuming, cumbersome, etc., for a user to purchase one or more items associated with a content item.” 

To make determinations, Google’s system analyzes several types of data, including text data from captions and metadata, audio data, and image data from frames of a video. The system relies on a fusion model — a deep learning model that can consider multiple types of data —  which calculates confidence values determining the likelihood that the text or images contain information related to a specific product. 

With image and video data, the system will use image searches to compare photos or frames of a video to a “data store” of product images. To decipher audio, the system employs a speech-to-text model for transcription, then searches that for product mentions. 

The system may also create an “alternative products panel” with pictures and information on alternate or associated products, such as color and size variations, bundles with other products or similar items from other brands. 

Google first started testing shoppable content on YouTube 2021 and launched the option to more users in 2022. Most recently, the company added the ability to timestamp when in a video each product is mentioned within longform content. 

The growth of ecommerce over the years has only led shoppable social media to become more popular. Instagram started experimenting with shopping capabilities in 2016, later adding a shopping tab to its interface in 2020 (which it removed in 2023 to make room for the Reels button). Pinterest has long had the goal of broad shopability across its platform, with the app’s CEO Bill Ready telling investors last year that the company aims to make “every pin shoppable.” TikTok was the latest to join the fold with the launch of in-app shopping capabilities in the US in September. 

These firms are following the lead of consumers by going where they spend their time — and who consumers trust. A survey of 2,000 US consumers from Deloitte found that 64% of Gen Z consumers and 57% of millennial consumers are more likely to purchase a product after seeing a review of it from an online content creator.

AI has made its way into this battle, too. The tech in this patent specifically seemingly aims to wrangle more creators with AI tools that take the work out of preparing their content with product tags. TikTok, meanwhile, launched its AI-powered Creative Assistant for businesses in September, and Meta introduced generative AI tools for advertisers in October. 

Integrating more AI tools could help YouTube continue its rebound: The division saw ad revenue jump more than 15.5% in the fourth quarter to roughly $9.2 billion. Referencing the company’s YouTube, Search and Cloud segments in the company’s earnings, CEO Sundar Pichai said that “Each of these is already benefiting from our AI investments and innovation. As we enter the Gemini era, the best is yet to come.”