Google’s generative AI tool might give wildly inaccurate summaries

The generative AI revolution has already consumed chatbots and search engines. And now Google plans to summarize whatever you’re reading on the web using the same basic technology behind Bard and ChatGPT.

On Tuesday, Google rolled out an early version of an experimental tool called “SGE while browsing” for users signed up to its Search Labs beta testing tool. The technology will summarize content you’re reading as you browse online, providing AI-generated key points to allow you to more quickly consume and understand the content.

“Our aim is to test how generative AI can help you navigate information online and get to the core of what you’re looking for even faster,” Rany Ng, VP of product management for search, wrote in a blog post.

It’s a major moment in the spotlight for generative AI—and one that gives those who analyze the technology pause for thought.

“Summarization models trained using supervised learning have existed for years and achieve high performance on a variety of tasks,” says Sasha Luccioni of Hugging Face, an AI company. “Using generative AI instead is fraught with all sorts of risks that aren’t worth it.”

AI summarization has long been a useful technology, but those summaries were generated with models trained using supervised learning. It’s where the dataset on which an AI model is trained is pre-labeled and the output the model creates is checked against known, labeled outcomes to ensure it’s working correctly.

The previous generation of AI summarization generally uses only the content fed into it to create the summary, meaning it’s less likely to hallucinate new content that is not included in the original.

Generative AI does not do that. Instead, its underlying technical basis is to try and discern patterns in a training dataset to be able to create new data similar to that on which it has been trained. It means that generative AI is guessing the probable next likely word in a sentence at any time, rather than innately checking it against a ground truth.

That’s fine when generating creative works, such as florid prose or fun bits of writing. But when layered into summarizing text accurately, it can misfire. Hallucination—sometimes called confabulation—is a major issue in generative AI.

Google has already experienced backlash resulting from high-profile mistakes in generative AI results in search and summarization. When its Bard chatbot was unveiled to counter the rise of ChatGPT in February 2023, the chatbot made a significant error in summarizing the role of the James Webb Space Telescope that wiped $100 billion off the value of Alphabet, Google’s parent company.

The fear is that something of a similar magnitude will happen again when the tool summarizes text on a web page. Worries that generative AI isn’t ready quite yet to be thrust into the spotlight plague Luccioni. “Just because you have a generative AI hammer,” she says, “doesn’t make everything a nail.”