eBay Vision Patent Tracks User Identity Over Time

eBay wants to use computer vision to keep you stylish. 

The e-commerce firm filed a patent application for an “intelligent online personal assistant” with an “offline visual search database.” eBay’s tech applies computer vision to the search process,  allowing users to search for items using photos of other items to find ones that are visually similar. 

“Comparison searching has become more important than ever, but current text-based solutions were not designed for this scale,” eBay said in its filing. “Irrelevant results are often shown and do not bring out the best results. Traditional forms of comparison searching (search+refinement+browse) are no longer useful.”

This system compares a user image to images from merchant listings to find a “visual similarity measure” between the two. The user image is compared to any number of merchant images to enable real-time visual searching “of a potentially vast catalog of items.” Along with taking in user images, this system pays attention to user text inputs and voice input when given. 

In addition, eBay holds onto this image analysis data to be made available for future “on-demand visual comparisons.”  This data helps feed a personalized digital shopping assistant that uses “scalable artificial intelligence” that can provide a user with personalized chatbot responses. This system relies on machine learning to understand user intents over time, “so that user identity and understanding is enhanced over time.” 

In one example, a user gives eBay’s system a photo of a casual dress, and asks the chatbot, “how about this for a formal dinner party?” Recognizing the user intent and that the photo provided doesn’t correlate with typical merchant listings for formal attire, the system recommends listing options that are less casual. 

“The user experience thus provided is inspiring, intuitive, unique, and may be focused on the usage and behavioral patterns of certain age groups,” eBay said. 

This isn’t the first time we’ve seen eBay take an interest in computer vision to understand its users at a deeper level. The company previously sought to patent a system that uses the vision-based AI to increase “efficiency in user interaction” by tracking interactions when online shopping. Both patents aim to help eBay hyper-personalize its user experience and better target its users with advertising. 

However, this tech isn’t exactly revolutionary, said Abhinai Srivastava, founder and CEO of computer vision firm Mashgin. eBay’s patent is quite broad, he noted, and exemplifies technology that AI firms everywhere are working on. The difference is that eBay is simply putting it into the context of its own shopping platform. 

But because this patent is broad, that means the end-product for users could manifest in a number of ways, said Srivastava. After nailing down the underlying tech, he said, eBay could experiment with how exactly to use it. For example, it could look like a chatbot, be embedded as part of search, or become a feature in eBay’s app, each of which would closely track user behavior to personalize the experience. 

The question becomes, Srivastava said, “Can they come to you before you even type that query? Or, in some cases, can they get to people before they even think they want something?  That may just be the future, (these companies) knowing you better in some ways.”  

While eBay’s tech may differentiate itself by being trained on its own database of millions of listings, developing this tech is an “arms race” among e-commerce companies, Srivastava said. Amazon, for example, is already the kingpin of e-commerce, and has quickly become a frontrunner for its AI work. “The next person might be offering a similar thing that you are,” he said.