Salesforce Einstein Makes Bold Claims With First AI Technologies

It’s been a busy few months for the new crew of data scientists at Salesforce’s artificial intelligence division, Salesforce Einstein, who have already published four research papers. They claim to have remade the way that computers read and understand language in the process. Sounds a bit arrogant to claim so much, but they do have some results to back it up.

When it introduced Einstein in September, Salesforce promised to embed AI across its cloud services, such as building predictive models for how to manage each client in its customer relationship management (CRM) database. Salesforce Einstein is starting with tools to dig through the innumerable records of interactions between companies and their customers. Keyword search only gets you so far with so many records, so Salesforce focused first on natural language processing (NLP) tools to understand the documents.

“Generally, AI is very open, and it’s moving at a very rapid pace,” says Richard Socher, the chief scientist at Salesforce.

The science gets complex pretty quickly, but the gist of Salesforce’s claims is that it has found more ways to compare documents to each other, and common questions that users might ask, so that the AI can better understand what people are looking for and where to find it. The company also says it has found a way to speed up all this work by making better use of parallel processing—a growing trend in AI that has so far been used most in image recognition. Salesforce submitted several of its papers to ICLR 2017, a machine-learning conference in April next year. Submissions won’t be reviewed and approved (or rejected) until February, so there’s no outside expert opinion yet on if Salesforce has really delivered on its promises.

The biggest promise may come from a mouthful of a method called a Dynamic Coattention Network for Question Answering (or DCN), which focuses on the interplay between the documents people are searching for and the questions they are most likely to ask. Salesforce’s system learns about documents not only by scanning them but also by taking hints about what’s in the documents based on the questions people ask of them. “This model can really reinterpret a document through the lens of specific questions,” says Socher. “But also sometimes [it] can understand the question in a new light given that this is the most likely document that might include the answer.”

The best evidence that this works would be the latest results of the Stanford Question Answering Dataset (SQuAD)—a reading comprehension test for computers based on articles from Wikipedia. Salesforce’s team currently has the top spot on the leaderboard, but SQuAD is a pretty new test, just launched in June.

Another claimed advance is called the Pointer Sentinel Mixture Model. It’s basically a computerized version of the advice we give children to “use context clues” when they are reading and come across a word they don’t know. The system can point to unknown words, such as foreign terms or unusual names, and figure out what they are based on how they are discussed in a document. Socher offers the example of Srini Tallapragada, Salesforce’s president of technology. The system doesn’t know ahead of time what a “Srini” is, but can figure it out because “Srini” shows up near words like “president of technology” and “Salesforce.”

The other two claims have to do with efficiency. Salesforce says it has developed a single AI system that can do the work of several systems handling different aspects of language analysis. “Imagine you have a search bar, and you also have a live chat messenger where someone can ask a question, and you also have an FAQ, and now you also want to have a translation system for expanding your website to different countries,” says Socher. Their new system can do all that at once, he says.

Finally, Salesforce claims to have merged traditional natural language processing with methods used in visual recognition systems. The latter make heavy use of parallel processing on graphics cards to speed up operations. (Graphics card maker Nvidia, for instance, now has a nice side business building what it calls AI supercomputers.)

Salesforce’s bold claims in language AI are yet to be reviewed by experts, so the (academic) jury is still out. But Salesforce is a business, not a university. So the real test of the technologies will come from its users.