Pokémon Go May Be A Fad, But Augmented Reality For Kids Is Here To Stay

A shaggy-haired monster as lovable as a teddy bear offers a wave and an encouraging smile that is in no way diminished by the pair of incisors jutting up from his jaw. “Nice to meet you,” says the creature, named Mo, who lives in an iPad app called Osmo Monster. “Can you draw me a magic wand?”

So begins a surprisingly intuitive interaction between digital and physical worlds, the complex software operating in the background masked by the childhood delight of free-form play. Draw a magic wand for Mo on a real-life whiteboard placed in front of the iPad and suddenly the exact sketch appears on the screen. One after another, Mo requests drawings of coins, a magic banana, and a favorite toy, incorporating each into his digital home. As music plays, he conducts the “grand finale”: an act that features all the drawings as they spin and dance.

Pramod Sharma, cofounder and CEO of Osmo, the startup that designed the app, describes the Osmo Monster experience as a way to give “Walt Disney moments” to kids that have grown up around screens.

“You’re co-creating,” he says. “Kids love cartoons, they love characters. I think that if we can make them look into your world, then it’s going to be a much richer experience, much more exciting.”

Sharma, an engineer by training, has made a career out of building technology designed to bridge the digital and physical realms. After graduating from Stanford, he worked on developing the machines behind Google Books, the company’s controversial attempt to scan the pages of the world’s libraries. Osmo cofounder Jerome Scholler, also a former Googler, had previously worked in the game industry.

Together they imagined building an educational game company around what Sharma calls “reflective AI.” Each of the seven Osmo apps relies on image data from a mirror that slides over the iPad camera, allowing the software to capture and make sense of the objects or drawings placed in front of the device. Osmo Monster, which launched this month, is the first of the apps to revolve around an animated character.

“It’s very hard, iterating on the storytelling side, the computer vision side, and the animation side,” Sharma says. “Kids know how to draw, and we know how to do scripted. But no one knows how they’ll come together.”

For example, the app needs to understand which end of the magic wand Mo should hold in his paw, an orientation question that is easy for even young children but challenging for software.

Going forward, Sharma envisions updates that would allow the app to identify components of a drawing and animate those specific parts. But first he and his growing team of 40 need more training data, the raw material from which AI hones its knowledge. That means more drawings of the type that adorn fridges and classrooms, such as stick figures of Mom and Dad.

In the meantime, the runaway success of Pokémon Go has introduced many families to augmented reality (“AR”) for the first time, though Sharma draws a distinction between the ubiquitous game and Osmo’s approach.

With most AR, he says, “You hold the phone in your hand and you interact with the screen. The digital is interactive and the physical is static.” For Osmo, the core interactions involve physical objects, from Monster’s emphasis on old-fashioned drawing to Osmo Tangram’s focus on geometry and spatial relationships. “We think about the power of hands, everything is about the power of hands,” Sharma says. “People are starting to understand why this kind of interaction, physical-digital, is very powerful.”

When Osmo unveiled its first app in 2014, Sharma’s pitch was a harder sell. Now, thanks to Pokémon Go, AR has gone mainstream overnight. The Pokémon craze may fade, but Sharma says Osmo is here to stay. “Our focus has always been on kids and creativity, learning,” he says. The goal is “to keep on innovating.”