Microsoft May Use Emotion Detection for “Richer” Work Meetings

Microsoft wants to express how work meetings make you feel. 

The tech firm filed a patent application for “intentional virtual user expressiveness.” This essentially aims to detect a user’s emotional state and have it reflected via renderings of facial animations. 

“Facial emotions can be detected to enable richer meeting services for meeting and communication services,” Microsoft noted in the filing. 

Microsoft’s tech offers a way to virtually display the emotional states of users by monitoring visual cues from movements in facial landmarks, such as smile lines, nose features, and eyebrows. This tech may also monitor and analyze audio data to determine whether or not a user’s sound or tone matches any particular emotion (this may be separate from the image data, or used as a means to supplement the emotional display). 

Using a deep learning model, Microsoft takes in all of this data to determine how a user is feeling at any given moment, and the “magnitude” of their emotional state. Finally, a “facial animation model” creates an emotionally reflective rendering of the user, updating the animation in “real-time or near-real time.” 

In addition, users are allowed to replace or assign different emotions to different facial expressions. For example, if a user frowns naturally with their resting face, they can designate that look as “neutral” rather than sad. Or if they don’t want to express negative emotions at all via their animations, they can choose to hide negative emotions. 

It should be no surprise that Microsoft wants to get a feel for your feelings. The company has previously sought to patent systems to correct your tone in emails, an emotional support chatbot capable of psychological monitoring, and AI models that assess your well-being.   

And Microsoft is just one of several tech companies that seem very intent on reading your emotions. Nvidia, Google, Amazon, Snap, and even Disney have patent applications that take a run at multiple kinds of emotion recognition for different purposes. 

This type of personal data can be quite valuable to companies, whether it’s aimed at creating a better user experience by improving communications (as Microsoft’s tech seemingly intends), gauging reactions to different content or generally getting a read on user behaviors. 

The only issue: Emotion-reading AI models have been widely criticized. For one thing, AI simply isn’t good at reading human emotions and is often limited to basic expressions that fall short of what a person is feeling. And because everyone shows emotion differently, AI models can’t reliably comprehend and offer insight into every kind of person. Plus, privacy is always an issue depending on the way emotion detection is implemented. 

Microsoft has landed in hot water for this kind of tech in the past, retiring an emotion-detecting AI product in July of 2022 after it was criticized as being unscientific for equating external displays with internal feelings. 

However, given that the original filing date of this patent was around three months after Microsoft retired this product and that it would allow users to override preset emotions, the company may still be working on ways to get emotion detection right.