Google’s Assistant, Home announcements preview a new kind of ‘augmented reality’

Google CEO Sundar Pichai presenting the Google Assistant on Tuesday

Google’s announcements on Tuesday — describing how the new Pixel smartphone and Home device will be the first places where its Google Assistant intelligent agent will live — are steps toward a world turned inside out.

Instead of you interacting through display screens with a digital world next door to our physical world, you will now have more opportunities to live inside a physical world that has digital layers on top of it.

Of course, Google isn’t inventing this new kind of layered reality. Amazon’s Alexa intelligent agent and Echo device have pioneered that path, and further colonization of the real world can be expected in the near future by Apple’s Siri, Facebook Messenger M and its growing collection of intelligent chatbots, Microsoft’s Cortana and other intelligent agents and interactive layers.

So far, we’ve used the term “augmented reality” primarily to describe activities like Pokémon Go, whose creatures appear on top of real-world locations when seen through an app on a smartphone.

But that’s too narrow a definition of the term.

Engaging with disembodied computing systems that inhabit the ether around us, that summon any fact or media from the universe on our behalf, that are always listening for a hot-phrase like “Ok Google” so the hidden spirit can awaken to summon a car service or turn on the microwave — by any reasonable definition, this describes a reality augmented with a non-natural intelligence.

This layered reality has been explored so much in science fiction that it’s almost a cliché. But with it now becoming reality, it seems to stretch a veneer of fiction over reality. A fiction in which we’re not interacting with apps or touch-screens or mobile devices, but with sustained voices or textual conversations — which we will undoubtedly interpret as characters — who act on our behalf.

When interfaces really work — like a steering wheel and dashboard for a car or windows with point-and-click navigation for computers — they soon take over for that kind of interaction. When they work, they almost feel natural.

And, of course, what’s a more natural way for humans to interact than to talk?

Ubiquitous intelligent agents, whether text- or voice-based, could become our steering wheel for virtually everything. And when voices are used, we will inevitably relate to them as characters with personalities.

Google is moving into first gear for that ramp-up, announcing (October 17, 2016) the beginning of what will likely be an enormous and potentially transformative ecosystem. Home, for instance, is already integrated with home automation systems like Nest (owned by Google), Philips Hue, Samsung SmartThings and IFTTT (If This, Then That).

Direct and Conversation Actions

For Google Assistant, there are Actions that seem akin to Skills for Alexa. Starting in December, developers will be able to build either Direct or Conversation Actions.

Direct Actions integrate a company’s services with Google Assistant, like turning on living room lights or playing a playlist on Spotify.

Conversation Actions involve back-and-forth communications and employ Google’s conversational user experience platform, API.ai. For instance, if you ask Google Assistant to summon a Uber car, it first needs to get some details via conversation, such as: “When do you want it?”

At (October 17, 2016)’s announcement, Vice President of Engineering for Google Assistant Scott Huffman noted that interactions might be voice-only, text-only or combinations, and that a software development kit (SDK) will be available next year so that Assistant can be embedded into devices and software applications.

One can envision that Google Assistant, Alexa, Siri and their siblings will begin to inhabit anything that can house the required processor/WiFi/voice chip and speaker.

At one level, this means developers, publishers and businesses can expect that many of their products and services will be found and delivered not through a mute and faceless search engine, but through Google Assistant and its brethren.

Google showed how its Assistant will be able to find virtually any information you want, by utilizing snippets of info from “trusted websites” in its reply. In other words, the emerging generation of search engine is a genie.

Just last week, agent development platform PullString launched its platform for creating intelligent characters that can operate inside virtually anything, from Barbie dolls to Facebook Messenger to refrigerators. Appropriately, a quarter of the company’s staff are ex-employees of the world’s most innovative animation studio, Pixar.

The emerging world is rapidly being augmented by invisible, omniscient characters that can act as go-betweens for virtually any digital or real-world product or service. In other words, personality is now becoming the new interface.

About The Author

Barry Levine

Barry Levine covers marketing technology for Third Door Media. Previously, he covered this space as a Senior Writer for VentureBeat, and he has written about these and other tech subjects for such publications as CMSWire and NewsFactor. He founded and led the web site/unit at PBS station Thirteen/WNET; worked as an online Senior Producer/writer for Viacom; created a successful interactive game, PLAY IT BY EAR: The First CD Game; founded and led an independent film showcase, CENTER SCREEN, based at Harvard and M.I.T.; and served over five years as a consultant to the M.I.T. Media Lab. You can find him at LinkedIn, and on Twitter at xBarryLevine.