Think Different (Again): How Apple Plans To Compete In The AI Personal Assistant Wars

Think Different. That’s the simple and iconic slogan Apple used during the second coming of Jobs, during his reinvention and revitalization of the company in the late ’90s. Yes, there are second acts in American life. Jobs gave one to Apple, and it’s still living it.

But the world, of course, keeps changing, and computing changes, and Apple itself may need to think different. This year has been the year of artificial intelligence, natural language processing, machine learning, and computer vision. They’re the new technologies that big platform players like Microsoft, Amazon, Facebook, and Google are thinking about and talking about.

Some, like Microsoft CEO Satya Nadella, say we’re moving past the smartphone age and into the bot age. “Bots are the new apps,” he said during his company’s developer conference in March.

In this context, Apple, suddenly, seems to have its roots in an older paradigm. Its speciality is making beautiful (and high-margin) devices with clean, tightly integrated software. Apple invented our current understanding of the smartphone and the app, and continues to see the world through those two concepts.

Beyond The App

But apps aren’t the only way of looking at the digi-verse, and we will eventually evolve past them. Amazon’s Alexa voice assistant, for example, introduces a completely new construct for accessing and acting on data. As many disrupting technologies do, Alexa rides on top of existing devices, transcends them. The first product containing Alexa’s brain was Amazon’s own Echo device. There have been other Alexa vehicles since—from Amazon and others.

Alexa doesn’t deal in apps. It uses “skills.” Skills aren’t carefully designed bundles of content like apps are; they’re just actions. Amazon built some of these skills for Alexa, but the power of Alexa is that she learns new skills from third parties like Uber. “Alexa, call me an Uber.” As more third-party developers participate, the smarter Alexa becomes.

Google has been working with machine learning in the context of web search for the past decade. The company made it clear at its Google I/O developer conference that its AI personal assistant technology, like Alexa, does not belong to one physical product, but can have many different embodiments, ranging from a personal assistant app (Google Assistant and Google Now) to a messaging app (Allo) to a freestanding home assistant (Google Home). And like Alexa, the truth is in the name. Google chose not to brand its personal assistant technology at all; it’s just called “Google,” which suggests that the technology not only can go into any product, and that it’s core to Google’s being.

Microsoft’s Nadella openly criticized the app as the go-to way to organize information and tasks, before announcing a set of new tools to help developers build bots that understand human language. The bots we’ve seen so far are no match for human helpers, but it’s still early days.

Apple’s Announcements At WWDC

That’s the world in which Apple’s annual developer conference is happening this week in San Francisco.

Apple used the occasion to roll out a boatload of new features for its major operating systems, but large parts of the keynote presentation Monday were dedicated to improvements to its Siri personal assistant, to computer vision technology, and to machine learning.

iOS has started to consult user information from within app silos to provide more useful recommendations. In a simple example, when you type something like “I’m available at” using the OS’s native keyboard, you’ll see some suggestions for free time windows on the appropriate day.

Siri has gotten better about handling nested conversations. In the demo of Siri running on a Mac, Apple software chief Craig Federighi asked the assistant to gather a certain type of files, then asked it narrow the search results down to files from a certain person—without having to start a new search query.

It’s hugely important to Siri that Apple is now allowing third-party developers to use the personal assistant as an entry point to their apps. So Siri will be able to do things like call a Lyft car or compose a WhatsApp message. And that’s very good. It’s analogous to Amazon’s Alexa learning new “skills.” But Siri won’t be learning anything from these transactions. A Lyft developer told Fast Company that Siri is being used only to trigger the Lyft app; it’s not collecting data on when, how, or from where the user is requesting a ride.

Apple has also been working hard in the area of computer vision, or the science of teaching computer systems to analyze, understand, and act on the visual aspects of images. Like its competitor Google, Apple has applied its development in the area to photo management. The Photos app in both iOS 10 and macOS Sierra can now run billions of computations on user photos to determine their content, then group them accordingly. Apple is also catching up with Facebook by deploying facial recognition technology that recognizes the faces of the people in your pictures, so that photos containing specific people can be grouped and searched.

The new Moments feature in Photos looks crazy cool. It’s nothing short of a video production robot that does everything you do when you make a Movie Maker video from your video and photos library. It even picks the music and pace of the presentation based on styles you can preselect. That’s a lot of machine intelligence.

What Is A Personal Assistant?

Before discussing the relative value of Apple’s efforts, let’s define what an AI personal assistant should do.

It should:

• Know me intimately (but not too intimately)
• Know my stuff and where it is
• Know about the outside world (real and digital), and introduce useful or entertaining things to me

Scoring Siri

Going on the assumption that personal assistants are only as good as the data sets they can access, what are Siri’s chances of fulfilling the above requirements better than the competition? What are Siri’s chances of becoming the ideal artificially intelligent personal assistant?

For the first job qualification (“It should know me”), Apple reasonably well positioned. It’s sold me and millions of others on its devices, and I routinely put my personal data (communications, projects, schedules) in Apple apps like iMessage, Maps, Photos, and Calendar. (See the example above where Apple automatically fills available calendar times into my messages). Apple announced Monday that developers can make apps that run within iMessage and Maps apps, which could make those Apple apps even more sticky to iPhone users.

But much of my data still goes elsewhere. My social data goes to Facebook, which therefore knows a lot about my tastes, preferences, interests, and about my “intent” to buy things. That’s why it’s making billions putting ads in front of me. I use Gmail for email, so all that personal data lives on Google servers. Same thing with my writing files; they live on Google Docs. So those other platforms could use my data to more intelligently push assistive information at me.

On the second job qualification (“know my stuff”), Apple is in a strong position because it has control at the device level. I keep much of my stuff (music, project files, video, images) on my iPhone or Mac, and I keep some of my stuff in the Apple Cloud. So Siri should be able (if not now, then in the future) to reach into many different silos to search and retrieve my digital assets. Amazon and Google have no control over stuff I store on my iPhone or iPad.

On the third personal assistant job requirement (“know the outside world”), Apple could be at a real disadvantage to Google and Amazon. Google’s core search business is all about knowing where everything is on the web, and understanding just what part of it you might be looking for. The way you do that is with machine learning and natural language processing. Now Google is putting that data science to work in personal assistant technology. With its access to Google’s vast Knowledge Graph, Google’s assistant technology probably has the edge on Apple for suggesting things from the digital world that might be relevant and useful to me.

Amazon has control over a vast field of shopping data. Alexa might use my own shopping history and produce preference data to form suggestions that are relevant to me.

So while Siri can benefit from user data kept at the device level and in Apple native apps, competing assistants have access to some very rich data sets.

A Unified Siri

This might make it all the more important for Apple to approach AI in a more holistic, less siloed, way.

Based on what we saw during the WWDC keynote, there’s a glaring difference in Apple’s approach to AI and personal assistant technology compared to its peers. Apple is tying Siri closely to specific devices, at least for now. As Farhad Manjoo points out in his thoughful analysis, Siri on the Mac doesn’t necessarily know what Siri on the iPhone, or Siri on the Watch, is doing. Siri is different on each device. In a sense, Apple is pouring Siri’s brain into different silos, giving her multiple personalities, divided by OS. Meanwhile, Google’s assistant and Alexa are being built as single minds that float on top of device-level dividing lines.

Let’s hope Apple liberates Siri from those silos in the future. I’m betting it will. After all, the company has demonstrated that it’s thinking along those lines for other areas. The whole idea of the iCloud is to float over all iDevices and serve content to all of them. Apple’s “Continuity” concept is all about crossing OS lines and making different Apple products share data and work in concert with one another. One of the improvements in iOS and macOS is the ability for a Mac and an iPhone to share the same “Universal Clipboard.” I’m guessing that announcement of “Universal Siri” isn’t too many WWDC keynotes away.

Apple’s work in AI and personal assistant tech may lag behind its peers somewhat, but the new Siri shown at WWDC showed that the company is very much in the game. The company has some very big (human) brains thinking hard about this problem; It’ll be fascinating to see where they take Siri in the future and how she stacks up against Alexa and Google’s assistant in five years.