Amazon Patent Highlights Double-Edged Sword of Personalized AI

Amazon wants Alexa to have an open mind. 

The tech giant filed a patent application for “machine learning model updating” corresponding to language and gesture input. Amazon’s tech essentially aims to help speech recognition modes better understand and adapt to a user’s behavior over time. 

When Amazon’s model first receives a new user input, whether it be a voice command or a gesture, this system categorizes that input based on how it could be used to train the device, depending on whether or not the data needs to be labeled.

Once the input is categorized, the system proceeds with training the model, whether it be a speech, gesture, or facial recognition model, based on its training type. These updates occur automatically and continuously, so whenever new data is available, the model is instantly training and gets better over time. 

After updating the model, the system may delete the data used to train it to preserve user privacy. If the data is kept for further training, it’s de-identified.

By updating this model continuously, this tech can understand a user’s habits, patterns, and trends. For example, if a user frequently requests the device play a certain song, it can more easily recognize that input for future requests.

This method “combine[s] automation and continual updating of machine learning models to enable increased user privacy while also providing a better user experience and [reducing] developer workload.” 

Amazon had high hopes for giving Alexa an AI-powered upgrade, announcing a generative AI revamp to the voice assistant last September and internally dubbing the project “Remarkable Alexa.” But a Fortune report from June found that Amazon has struggled to bring that vision to life: The project’s faced both technical and organizational mishaps. 

Now, Amazon has reportedly turned to Anthropic (in which it’s invested $4 billion), using its large language model Claude to power the next generation of Alexa devices, Reuters reported in late August. An Amazon spokesperson told Reuters that the company will use “many different technologies to power Alexa,” including its own and those from partners. 

Though the tech in this patent aims to make speech recognition devices sharper, more conversational, and more personalized, a system like this may have the opposite effect, said Bob Rogers, Ph.D., the co-founder of BeeKeeperAI and CEO of Oii.ai. To put it simply, training a model to be personalized bit by bit with small pieces of data could create a domino effect that impacts the way it was originally trained, he said. 

“If you don’t have a way to make sure that your updates don’t break the old stuff, your [model’s] behavior is going to get weird,” Rogers said.

This patent’s focus on privacy may also be a hindrance, said Rogers. Deleting the data right after it’s used for training creates an “instability risk” in which the model can drift from the original point of reference, unchecked. 

On the other side, holding on to that data creates a “honey pot,” he said. “It’s almost a security problem. How do you store that data without risking it, so that you may maintain that memory?” 

The question remains: Is a smarter version of Alexa worth it? Though having a device that can understand you and your context on a deeper level could achieve Amazon’s goal of creating the “world’s best personal assistant,” actually getting there clearly isn’t an easy feat, he said. “I’d rather have a dumb product that works really well than a smart product that only kind of works,” Rogers said.