Meta doesn’t want to just hear your voice. It may want to feel it, too. 

The company filed a patent application for user authentication using a “combination of vocalization and skin vibration.” Meta’s tech essentially uses additional biometrics — namely the “vibration of tissue” caused by speaking — in tandem with a voice to authenticate users. 

“There is a security concern with voice-only activation authentication systems because one can easily hack another’s voice either through computer generation or by impersonating,” Meta said. 

Meta’s tech aims to solve that by using the biological signals that are created as a result of speaking to make sure that the person seeking access to its systems — which, in Meta’s examples, is a mixed reality headset or pair of smart glasses — is actually authorized to do so. 

As the title of this patent suggests, when a user says a wake word, skin vibrations are picked up via a “vibration measurement assembly” built into the headset. The system also detects “airborne acoustic waves” (a.k.a. the sound itself) that correspond specifically to the user’s voice. 

These measurements are combined to create an “authentication dataset” unique to the user; the vibrations act as a fingerprint to how they say a specific wake phrase. This process cuts out the need for additional authentication steps, such as passwords or fingerprint scanning, thereby improving user experience, Meta noted. “Additionally, a user’s voice can be authenticated periodically, improving device security.” 

The tech in this filing is meant specifically to protect its augmented reality headsets, which may be beneficial when considering the amount of personal data these devices can collect. But this isn’t the first time we’ve seen Meta seek to patent voice-authentication tech. 

The company previously filed an application for “user identification with voice prints” for social media, which adds user voiceprints as a part of the two-factor authentication process. These patents highlight the growing trend of seamless passwordless authentication, forgoing a typical password for something more difficult to breach, such as biometrics. 

But the rise of generative AI has given bad actors a far bigger toolset to breach cybersecurity defenses. Though biometrics are generally harder to breach than a traditional password, deepfakes and voice impersonation technology stand to make it much easier, said Rahul Sood, chief product officer at identity security company Pindrop. 

“Any authentication that relies on a human seeing an image/document or listening to a voice is at risk with deepfakes,” Sood said in an email. “Deepfakes break trust in all remote interactions.” 

Meta’s tech could overcome this, however, by taking into consideration the vibrations created through talking as a second factor. But offering up more in-depth personal biometrics may not be a cure-all to cybersecurity woes, said Theresa Payton, CEO and chief advisor at cybersecurity consultancy Fortalice Solutions. 

Biometric data is deeply personal, Payton said, and given the way that user data is treated as it stands, giving that data over for the sake of security presents its own risks. If that data is subject to a security breach, it’s not exactly easy to erase or replace, she said. 

“Once those are in the wrong hands, how do I prove I’m me and the [bad actor] is not me, if they’re able to present the biometrics?” Payton said. 

Apple may use AI to help the Vision Pro eat up less power. 

The company is seeking to patent a system that uses machine learning to determine user location in artificial reality based on “object interactions.” Apple’s filing claims that its method eats up less computational power than other artificial reality tracking processes by monitoring a user in relation to their environment.

“Existing systems may require utilization of a relatively large amount of processing and/or power resources to detect a user location,” Apple said. “Such systems may require constructing a map of an unknown environment using computationally-intensive techniques while simultaneously tracking a user location.” 

Instead, this tech uses a machine learning algorithm to create an “object relationship model,” which represents the “positional relationship” between objects in an environment. To create this object relationship model, the system collects sensor data to feed to the machine learning algorithm from audio, image and motion sensors embedded within the device. 

As the user interacts with the object in their environment, the device begins to understand their movements, such as orientation, object contact, or gaze direction. Finally, the machine learning algorithm determines a user’s location in relation to the objects in their surroundings. 

Understanding a user’s location and surroundings in this manner “can aid in the processing efficiency of displaying an XR scene,” Apple said.

After more than a year of remaining relatively quiet on its AI plans, Apple jumped headfirst into AI at its Worldwide Developers Conference in June, debuting “Apple Intelligence,” its personal AI, for iPhone, iPad, and Mac. It also announced a major partnership with OpenAI to integrate ChatGPT directly into Siri. 

AI integrations for the Vision Pro were not included in the slew of announcements this year, but the headset may not be left out of the company’s AI roadmap. Bloomberg reported last week that the company plans to bring Apple Intelligence to the Vision Pro, but is facing the challenge of ensuring that the user interface looks seamless in a mixed reality environment. Patents like these reveal how machine learning may be integrated into the inner workings of Apple’s mixed reality hardware, too. 

However, roughly six months after the company’s debut headset hit shelves, the market’s reaction has been expectedly “lukewarm,” said Tejas Dessai, research analyst at Global X ETFs. The current version is just a start, setting the stage for a “more comprehensive entry into the post-smartphone ambient computing landscape,” he said. “Overall, we are at least five years away from mixed reality being fully ready for mass-market adoption.” 

A lot needs to happen to get to that point, he said, including a broader application ecosystem, cheaper components, and better form factor. But adding AI into the mix could play a major role in getting consumers to adopt these devices. 
“Device-based intelligence will become increasingly important as consumers interact with devices in the coming years, driving permanent behavioral changes in how users engage with applications,” he said.

In AI development, data can be a double-edged sword. Microsoft may be looking at safety precautions. 

The company filed a patent application for a “privacy preserving generation” of synthetic training data — specifically “utterances,” or voice data. Microsoft’s tech aims to create robust datasets to train natural language processing and dialog models, such as speech recognition tools, without violating privacy.

This system first trains a generative model on a “differentially private” dataset of voice data. For reference, differential privacy is a technique in which noise is introduced to a dataset, essentially blurring out private information so that individual contributions aren’t revealed. 

After learning from already-private datasets, the trained generative model is in turn used to create synthetic datasets. This process effectively provides two layers of protection: one from the initial training on differentially private data, and another from the use of synthetic data itself.

The synthetic data generated by this model can be used by developers without fear that their creations may accidentally spill private data, Microsoft said, as “trained models can ‘memorize’ details of their training data, which can be exploited through different types of attacks.”

AI models’ hunger for data comes with a host of privacy and security risks, said Bob Rogers, Ph.D., co-founder of BeeKeeperAI and CEO of Oii.ai. Microsoft’s patent takes two known solutions — synthetic data and differential privacy — and stacks them on top of one another. 

While this could provide an extra barrier against an AI model leaking sensitive data, it also walks the line of the data losing its efficacy, said Rogers. The trade-off with privacy-forward synthetic data is less “expressiveness,” or the nuance and depth that help make an AI model more robust, said Rogers. In this case, balancing privacy and nuance may be tricky, he noted, but not impossible. 

Having a privacy-forward mindset could stand to benefit Microsoft as it continues to plug away at developing models. As tech firms rapidly rush to dish out new AI tools and chase the spotlight with ever-growing large language models, something’s bound to give, Rogers said. In the event of a massive data breach, “you don’t want to be the company responsible.” 

“You want to do what you can to avoid obvious things that would create a backlash against this thing that you’ve invested so much money in, and that you’re betting a lot of the future of the company on,” said Rogers.