Amazon Listens to Your Voice (plus more from Disney)

Emotion tracking from Amazon, e-sports live streaming from Disney & more

1. Amazon – voice tracking for emotional data

Amazon filed a patent application for a wearable device that captures a user’s speech, analyses it for emotional data, and shows a user how their emotions are changing over time.

To extract emotional data from speech, the device will track the following:

Valence: how a user’s pitch of voice changes over time

Activation: how a user’s pace of speech changes over time

Dominance: patterns in the rise and fall of pitch

Words: specific words that a user speaks and their meaning

Using all of this data, Amazon will be able to map users’ speech to specific emotional states or a “sentiment score”.

Besides listening to a user’s words, Amazon also describes the wearable as measuring hours of sleep, heart rate, hours of sleep and number of steps taken.

In turn, the wearable could offer actionable steps for users to lift their emotional state. For instance, if a user displays more negative emotions if they sleep for less than 7 hours, they might receive a recommendation to get more rest.

Why might Amazon be exploring this?

With wearables, users are constantly streaming data that is being captured, analysed and presented in a way that can inform behavioural change. While we’ve seen the quantified-self movement being applied to our heart-rate, our foot steps and even our glucose levels, emotions have so far been out of reach. Mood trackers that require user-input do exist, but there’s too much friction and subjectiveness for the experience to be extremely effective.

There’s potentially a lot of value and demand for a device that continuously tracks our emotional state, especially in a world that is more actively managing their emotions.

More interestingly for Amazon is the fact that a “continuous mood-tracker” provides Amazon richer access to peoples’ lives. Alexa is limited to the home and only captures words that are intent-driven. This wearable captures a user’s life both inside and outside the home, as well as capturing thoughts that aren’t simply commands for Alexa.

If you express anger that your toaster is broken, Amazon could in theory listen out for this and start pushing ads for new toasters to you.

Or even more dystopic, Amazon could discover which emotional states are more conducive to specific purchasing decisions, and use this to inform product recommendation algorithms. For example, if you’re feeling stressed, you may be more likely to make an impulsive luxury purchase.

After Amazon announced their security drones for the home to much criticism from privacy campaigners, one can imagine a similar response from an Amazon emotion tracker that is actively listening and processing everything a user says.

2. Disney – interactive e-sports live streaming

The most interesting aspect to this patent is that Disney are considering live streaming broadcasts, especially in the e-sports arena.

In the patent, Disney describes the current state of e-sports live streaming as having limited interactivity . Usually, gaming live streams consist of the player’s face overlaid over the gaming environment, with a chat room to the side of the stream.

Disney want to enable viewers to pan, tilt, or zoom around the game environment during a live stream, thereby making the broadcasting experience more interactive. For example, imagine you’re watching someone play a game and you want to see what objects, items or characters are off-camera. According to Disney’s patent, viewers would be able to explore beyond what they are shown via the broadcaster’s live stream.

Besides e-sports, Disney also imagine a use case for live streams of real-world environments. In theory, a broadcaster could record their real-world environment with one or more cameras, and then it could be turned into a ‘VR’ environment for a live-stream, where viewers can then explore the environment to their own liking. So rather than just watching a live streamer, a viewer can interact with live streams in a more immersive way.

Is Disney looking to implement some sort of live-streaming content on Disney Plus? Could this be an interesting flywheel for Disney to promote its games (e.g. the new Avengers game) on Disney Plus, and convert gamers into Disney Plus subscribers? It’s unclear, but the fact it’s being thought about is fascinating.

3. Amazon – biometric input device

 

 

Amazon are thinking about building standalone biometric devices.

In a previous issue, I described how Amazon were incorporating biometric sensors in the Amazon Ring doorbells to capture data beyond just video and audio, such as fingerprints and DNA.

In this new application, Amazon describe a biometric input device that would capture a user’s palm using image sensors and no physical contact.

The device could be used in a number of contexts:

Controlling physical access to a facility: i.e. ensuring that only permitted people are allowed to get access into offices, warehouses, residences, transportation facilities etc.

Facilitating payments: for example, at the point-of-sale terminal in a store, the biometric data could be used to verify that a card is being used by the actual owner

Signing contracts: e.g. agreeing to a contract or signing for a delivery

According to the patent, Amazon describes these biometric devices as being superior to other devices, because the lack of physical contact avoids issues of sanitation (particularly relevant in the post-Covid world) and physical wear-and-tear. Moreover, Amazon claims its devices to capture higher resolution images, thereby ensuring higher confidence of verification. Finally, Amazon is able to ensure rapid verification, making it appropriate for places with a high traffic of people.

This patent hints at a future strategic direction for Amazon and its Ring subsidiary (the smart doorbell).

Firstly, Amazon might be looking beyond home surveillance with its Ring acquisition, and look to a make inroads in the ‘surveillance’ market for enterprises.

Secondly, introducing biometric verification at PoS terminals could give Amazon access to data on users’ purchasing behaviour outside of the Amazon ecosystem. Again, this could help Amazon build richer user profiles on their customers, and in turn inform product recommendation algorithms.

But, as is a recurring theme in this newsletter, there are undoubtedly huge concerns around privacy.