Eye tracking in VR/AR: The promise and the perils

Posted on May 26, 2017 in Connected Devices

CEO of Sensics

3D render of man wearing virtual reality glasses surrounded by virtual data

What if everyone knew exactly what you are looking at? Chat with a stranger and she will know if you focus on her eyes, chest or belt. Talk with a neighbor and he’ll know when your gaze is wandering away.

A couple of years ago, I met a salesperson demonstrating an eye tracker for a see-through goggle. He was wearing the goggles at a trade show booth and had a screen showing his gaze direction. His product had lots of possible applications. One was market research. You could learn which cereal box grabs a shopper’s attention. Another was user interface design – the layout of a car’s dashboard. But to showcase his technology, he had to train to avoid awkward social situations.

Fast forward and imagine surfing the Web or playing a game inside a VR goggle with eye tracking. On one hand, you have great benefits. Rendering is faster because it can match the quality to your gaze direction. Areas in the center of your vision render in high resolutions. Areas away from the center of vision show in lower resolution. A user interface is also easier because UI elements can respond to your gaze. Interaction with virtual characters also becomes more natural. You can wink at an avatar. You could stare down a virtual bad guy. You can even look in a virtual mirror and see the reflection of your eyes follow your actual gaze.

At the same time, the system monitors your gaze direction at all times. This is a marketer’s paradise. Guesswork is no longer required to understand your focus. Your eyes tell the story. What drew your attention in a 360-degree movie? What ad was most effective? What caused pupil dilation or contraction – a sign of arousal or fear.

What about privacy?

Eye tracking brings a lot of benefits but also carries significant privacy concerns. Will the benefits be so significant that users will allow tracking their gaze direction? Is there a way to limit the use of eye tracking information to a particular purpose?

One could say that this is not much different than Web browsing history. Google and others analyze your clickstream today. The Web sites you visit impact the advertisements that you see. Your phone location gets used in the same way. Is eye tracking different? Somehow, it feels that the answer is: yes.

Eye tracking companies such as Tobii or SMI are aware of this concern. Tobii, for instance, recommends explicit consent for use of eye tracking data. Unlike selecting a link with a mouse, gaze direction is semi-conscious. Thus, it could reveal more than the user wants to share.

Privacy and use agreements are sometimes handled through standards organizations. It is unclear how soon this could happen. One such VR/AR standards effort is Khronos OpenXR. It is still not public knowledge whether eye tracking will be part of the initial standard. Even if eye tracking will be part of the initial standard, privacy might not be.

Eye tracking has powerful and serious uses aside from gaming and surfing. For instance, people with disabilities can leverage eye tracking to improve quality of life. People with vision disabilities can see better. Those with limited mobility can use eye tracking for better computer interaction. The privacy concerns are not a reason to discard eye tracking technologies. We just think about them as VR goggles start using eye tracking and other sensors.

This article is part of our Virtual Reality series. You can download a high-resolution version of the landscape featuring 431 companies here.