Google May Watch Users Closely with Context-Aware Digital Assistants

Google wants its digital assistants to be there when you need them most. 

The company is seeking to patent what it calls “contextual triggering” of assistive functions. This would allow digital assistants to kick in and make suggestions without the user needing to use a wake word or press a button. 

Here’s how it works: This system monitors the “current state of the user” by tracking sensor data collected from different devices, such as a smartphone, tablet or smartwatch. These sensors can track a number of metrics, including motion data such as location, speed or direction, gaze or other image data, and audio data. These help the system determine if the user is “walking, driving, commuting, talking, or reading.” 

When certain states are detected that may signal a need for assistance, the system will present the user with intuitive options. For example, if a user is reading an article on their phone while on a walk, this might trigger a notification that says “Do you want this read out loud” before crossing a crosswalk to avoid getting into a collision. Or if a user is listening to something out loud in a noisy environment, it may ask them “do you want to switch to Bluetooth?” 

“As assistant interfaces become more integrated with these various applications and operating systems running on computing devices, digital assistants may be leveraged to influence how media content is presented to a user of a computing device to aid in the awareness of the user,” Google said in its filing.

Patents like this show how badly big consumer tech firms want to know their users intimately by offering tools for convenience, said Brian P. Green, director of technology ethics at the Markkula Center for Applied Ethics at Santa Clara University. This is also evident through Apple’s continued domination of health tech, or Meta’s interest in reading users eye movements and brain signals with its headsets.

“This is kind of a natural outgrowth of what’s already happening in the tech industry,” said Green. “I think there’s a general trend there: Collect more information in order to be more useful, in order (for the tech) to get used more, which of course then drives revenue.”

So what does Google stand to gain from this? Other than selling more devices by adding intuitive features, this tech also gives the company access to a lot of personal data, said Green. That data itself is highly valuable, and can be used to continually improve its systems as they get to know their users. 

What consumers sacrifice for more usefulness and convenience is privacy, said Green, as these tools need a constant stream of personal data to work optimally. While it depends on the company and the device or service itself, he noted, many consumers don’t realize how much data these companies have. “By collecting up a lot of this data, if there is a huge security breach … that’s going to be a breach of trust that these companies aren’t going to be able to heal very easily.” 

And as this data collection continues, so does the security risk, both to the consumer and the companies’ reputation. “Are they going to continue being like Icarus flying a little bit too close to the sun, or staying just out of having their wings melt?” Green said. 
Plus, let’s not forget that the majority of Google’s money comes from its behemoth of an advertising business, not its devices. The company brought in nearly $60 billion in ad revenue in the most recent quarter, up 9% year over year, and making up more than 77% of total revenue. Whether this data could be used for ad purposes, Green said, “There’s always going to be at least a temptation there.”