Apple’s latest ARKit preps augmented reality to become pervasive reality

This week, augmented reality took another step toward becoming much more than just an add-on.

On Monday, at its Worldwide Developers Conference in San Jose, California, Apple introduced the next gen of its ARKit augmented reality platform. Version 2 allows multiple players to interact in real time with the same objects in a single space, and it allows objects to permanently reside in a specific location.

There is also 3D object detection, improved face tracking, realistic rendering, and, in collaboration with animation studio Pixar, a new, open and zero-compression file format called USDZ that allows AR files to be exchanged across Apple and other applications. Plus, the multiperson interaction works between phone and phone, not with the cloud, because of privacy concerns.

These new capabilities begin to offer some tangible reasons why Apple CEO Tim Cook told the UK’s Independent last year that AR is “a big idea, like the smartphone.”

“I think AR is that big, it’s huge,” he added.

A multiplayer AR game (Image from Apple)

Shared experiences, if they take off, could offer marketers an entirely new way of presenting and experiencing products and services, beyond games. There’s also a spectator mode, so someone could watch others interacting with objects.

In a blog post, Apple suggested consumers might “collaborate on projects like home renovations,” but it’s not difficult to imagine other use cases: price tags and provenance audits that pop up from products in a showroom, for instance, seen at the same time through your and your shopping companion’s phones.

Persistent AR means that objects stay where you put them, setting up virtual products that actually reside in reality — puzzles, 3D games, furniture that you’re “testing out” in your living room. The newest ARKit also provides support for image detection and tracking, and automatically adds real-world reflections.

In other words, Apple is now making it possible for AR-based objects to possess the essential elements needed for real-world interaction. These objects know where physical surfaces are, they stay where they are put and they can be seen and acted upon by others at the same time.

At the announcement in San Jose, Apple showed a demo where real Lego blocks assembled into a cityscape had a digital overlay of characters, other buildings and cars, and bubbles overhead that indicated which overlaid items were interactive.

Another multiplayer game, with AR objects on top of real objects. Seen from Player Two’s tablet. (Image from Apple)

It also showed a new iOS2 AR application called Measure, which allows a user to visually measure dimensions, area and diagonals of objects like picture frames using an AR “tape measure.”

Parks Associates Research Analyst Kristen Hanich told me via email that “multiplayer experiences and persistent objects will be huge, and an important part of creating immersive experiences,” although she noted that the biggest impact — at least initially — will be in gaming.

In the world of product marketing, she envisioned that “entire families can work together to design their dream kitchen, and friends will be able to give each other digital makeovers.”

Forrester Vice President Jeffrey Hammond emailed that, while it may take time for multiplayer experiences to emerge, it could lead to such new marketing opportunities as “guided selling.”

But, he cautioned, the necessary tools to support this kind of ecosystem still need to emerge.

“Although Pixar has made [the USDZ file format] available under an open source license,” he wrote, “it’s not clear to me if there are many external contributors making pull requests to it yet. My advice to marketers: monitor the space, but be careful committing serious [resources] until the tool chain of developer AR tools supports these new features.”

Keep in mind that Apple is expected to release an AR headset by 2020. That device, and other kinds of eyewear, may offer the ability to see these shared, persistent and hidden layers of info and interactivity over anything, without carrying your phone or tablet in front of you.

Reality may be stretching far beyond augmentation, toward a fifth dimension where alternative objects, creatures and information permanently reside.

[Article on MarTech Today.]

About The Author

Barry Levine

Barry Levine covers marketing technology for Third Door Media. Previously, he covered this space as a Senior Writer for VentureBeat, and he has written about these and other tech subjects for such publications as CMSWire and NewsFactor. He founded and led the web site/unit at PBS station Thirteen/WNET; worked as an online Senior Producer/writer for Viacom; created a successful interactive game, PLAY IT BY EAR: The First CD Game; founded and led an independent film showcase, CENTER SCREEN, based at Harvard and M.I.T.; and served over five years as a consultant to the M.I.T. Media Lab. You can find him at LinkedIn, and on Twitter at xBarryLevine.