Apple Wants to Track Your Eyes Without Draining Power

Ahead of the release of its first headset, Apple may be hard at work on features to watch you while you use it. 

The company is seeking to patent what it calls “retinal reflection tracking” for gaze alignment. Apple’s tech tracks the light reflecting off a user’s eye, using that to monitor movement and initiate certain actions. 

Apple’s gaze detection system saves power by using low-power hardware in situations where advanced eye-tracking may not be needed, such as casual headset use, rather than advanced gaming. The company said in its filing that, while current gaze-tracking systems are highly accurate, they often suck up lots of power. 

With this system, Apple wants to offer eye-tracking that “does not need to be continually tracked across the entire possible gaze space in order to provide power savings for assessing an eye characteristic.”  

To break it down: This tech uses an illuminator to light up a user’s eyes. It then uses sensors to detect if the light reflecting off their eyes has certain properties, for example, “a spectral property indicative of a redeye-type reflection.” So, rather than continually tracking user gaze, this just determines if a user’s gaze is lined up on what Apple calls a “hot zone” or “hot corner.”

If the detector determines that a user’s glance lasts more than a certain amount of time, the system will launch certain actions, such as starting up a program or responding to a notification.  

Apple has spent years working on its upcoming mixed-reality headset. Its patent activity is littered with filings related to things like gaze monitoring and hand gesture tracking, spatial video capture and gathering “real-time social intelligence” on the people around you while wearing it.

The company’s first offering in the mixed reality space, called the Vision Pro, won’t hit the market until early 2024, but the company made waves with the device’s official debut at Apple’s Worldwide Developers Conference in June. Excitement surrounding the device has even gotten the attention of rivals: Samsung, Google, and Qualcomm are reportedly planning to manufacture 30,000 units of their own mixed reality headset, priced at $2,000, set to launch in late 2024. 

Whether or not the features that we see in Apple’s patents make it into the first iteration of the Vision Pro, rumors about the next version, called Project Alaska, have already begun to circulate. According to MacRumors, the company is working on a simpler and more streamlined design for its second-generation headset, potentially to drive down costs. Bloomberg reported that the company is working on a next-generation version that’s easier to wear for longer periods of time. 

While Apple’s Vision Pro debut caused a stir, it has some competition in the headset market, primarily with Meta. The company has been working on its headsets for years, and held 50% of the market share for the extended reality tech market in the second quarter, according to Counterpoint Research. Plus, Meta’s headset is significantly more affordable, coming in at just $250 for the cheapest models, compared to the steep $3,500 price tag that the Vision Pro will have.

One advantage that Apple may have compared to Meta — and most other tech companies — is that it has a strong reputation for user privacy protections, which is something that users may take into consideration when picking which device to wear on their face for long periods of time.