Apple Cuts Though the Noise with AI Optical Detection Patent

Apple may want to use all its senses to understand its users. 

The company filed a patent application for “directional voice sensing” using “optical detection.” Basically, Apple’s tech uses optical sensors that pick up vibrations to improve voice detection in noisy environments. 

Along with using a microphone to pick up audio for speech recognition, Apple’s system uses an array of optical emitters and optical vibration sensors. The optical emitters give off synchronized light, and the optical sensors pick up vibrations from voice or movement corresponding with the light. 

These sensors pick up more than just vibrations of a person’s voice alone. Apple noted that they can decipher voices based on “vibrations of the people’s skin, mouths, teeth, glasses, throats, clothes, or other surfaces that may vibrate when the people use their voices.” 

After these sensors capture this data in “waveforms,” a processor that uses pattern recognition and AI analyzes it to decipher voices from one another. When the processor finds the voice of the user that owns the device, it can adjust the microphone to listen specifically for that voice. 

“A problem with typical audio microphones is that they often have poor directionality of pickup due to the wavelength of the baseband acoustic signal,” Apple said in the filing. “Using coherent optic sensing to detect audio vibrations (or voices) gives much better directionality.”

It’s no secret that Apple wants to make its voice recognition tech sharper. In June, the company debuted the capability to drop the “hey” from the “hey Siri” wake phase as part of iOS 17. The move marks a significant engineering feat, as dropping to just one wakeword gives speech recognition systems less time and less data to understand you. But tech like this patent lays out could aid in that understanding. 

Plus, the fact that this tech incorporates AI also adds up: Apple has been closely focused on AI work in recent months as it works to catch up with the rest of Big Tech. Bloomberg reported in October that the company is on track to spend $1 billion per year on generative AI efforts. 

Since then, reports have emerged that the company will announce major generative AI features with the debut of iOS 18 at its annual developer’s conference this summer. Along with updates to Apple Music, the Messages app, and iWork apps like Keynotes and Pages, the company is also reportedly working on a version of Siri that incorporates large language models. 

However, Apple’s not the only company looking to supercharge its smartphones with AI. The tech is already a major feature of Google’s Pixel 8 that hit the shelves in October, and Samsung announced last week that it’s working with Baidu on incorporating its large language models into its Galaxy S24 smartphones.