New AI tech could make video-game NPCs a lot more interesting

Non-player characters (NPCs) in video games generally have a pretty limited range. They might have a restricted number of canned responses to your questions but generally act as cannon fodder or story exposition.

Convai, however, believes they have a lot more potential.

The AI firm turned heads at CES earlier this month at the Nvidia press conference, showcasing a technology that blended generative AI with NPCs and letting players have conversations with the in-game characters that weren’t scripted and could go anywhere.

“What we are doing is giving these NPCs a mind,” Purnendu Mukherjee, founder and CEO of the San Jose-based Convai tells Fast Company. “So much so that they’re able to have a natural conversation with us. You can talk to them about anything. Not only can you do a voice-to-voice conversation, they show emotion. They have the right facial expressions. And in a 3D environment, the next level of cognition is they can perceive the environment and interact with it.”

The conversations aren’t exactly humanlike. There’s a minor delay before the NPCs answer your question and the facial animations have a way to go, but it’s a big leap from the blank stare you get when trying to get information in, say, Skyrim.

The AI runs on the cloud. Characters “see” and “hear” your character in the game (as well as your voice—you actually speak to these characters, rather than choose from a limited set of text questions). Their responses are sent back to your device and rendered into the game locally.

The NPC you interact with can be anything from a generic citizen of a village to an in-game companion that accompanies you on quests. And while some people will certainly try to induce AI hallucinations or have townsfolk in a period-based game like Assassin’s Creed discuss modern events, Mukherjee says there are safeguards against that.

“They will stay in character,” he says. “They will not break character. If you’re interacting with a professor or medieval merchant and start talking about something outside of that world, they will say, ‘What are you talking about? I have no idea what that is.’ . . . Developers have the complete freedom to say how far [characters] can go from their backstory and how restrictive they want to keep them.”

You will be able to do more with these characters than you can with current NPCs, though. One example Mukherjee gives is asking an NPC to sacrifice themselves so you can advance in the game, which could raise some very interesting morality narratives in games.

And there’s obviously use cases for Convai technology beyond the world of gaming. Mukherjee mentioned AI-generated characters could potentially act as virtual concierges in hotels or airports, giving directions via location-based knowledge, or act as agents for brands.  

While Convai says it can’t name the companies it’s working with at present for contractual reasons, Mukherjee says the technology will roll out into the real world soon. Initially, it will be from small to mid-size developers, as those groups tend to roll out games at a quicker pace. But Convai says it is working with larger studios to incorporate the technology into games.

“They’re being more conservative about announcing this,” Mukherjee says. “That will probably be another two or three years. . . . We have a bunch of partners, but it’s not our information to share.”