This Apple patent may be the key to AR glasses we’d actually want to wear

Apple’s work on augmented-reality glasses are one of the worst-kept secrets in tech. Right now, AR experiences aimed at consumers ask us to experience the technology through phone cameras, and everybody knows that can’t last. Digital imagery projected into the world in front of us—via glasses or something like them—might be the next big user experience.

But getting the necessary componentry small enough to fit in glasses you’d actually want to wear has been a huge barrier. Apple may have found at least part of the answer. A patent published Thursday describes a pair of glasses that don’t project light onto a lens, but rather into the retina of the wearer. It’s Apple’s spin on a projection technology called foveated imaging.

Such systems focus light on just a tiny part of the retina called the fovea, the part of the eye that specializes in seeing small things (like text in a book) with high acuity. The fovea is packed with optical nerve endings; it’s less than 1% of the size of the retina, but it takes up more than half of the brain’s visual cortex. By focusing light on just that area of the eye, sharp images can be projected at different depths of field and at high detail.

The light engine in Apple’s design is positioned on the arm of the glasses, aiming toward the lens. The beams of light it sends deflect from a reflective surface on the inside front of the lenses and project back onto the eye. Wide beams of light are projected into the eye, through the lens, and to the fovea, while thinner beams are directed at the periphery of the fovea. The images created look like ghostly 3D holograms that seem to blend with the world. That differentiates Apple’s design from Google’s Glass, which projects flat, 2D images onto the world in front of the wearer.

But the fovea area of our eye is always moving around as we focus on things, so Apple’s glasses design also includes eye-tracking technology to make sure the beams are going to just the right place at all times. Microsoft’s HoloLens mixed-reality headset also uses eye tracking.

Projecting tiny beams of light into the retina may require far less power than systems that project light onto displays. If the technology works well and matures, it could change the whole calculus of powering and cooling the AR glasses. It may make that elusive comfortable pair of glasses—with great battery life—possible.

Nobody knows for sure when Apple might release AR glasses, and “never” remains a possibility. Apple didn’t invent foveated imaging— Nvidia, Mojo, and others have been developing the technology too—and there are no guarantees the company will actually use it in a shipping product. But even though a patent is just a patent, Apple’s research into the technology is important. There are only a handful of companies that have the resources and ecosystem needed to finally make AR glasses a mainstream tech product—and Apple is at the top of the list.