Baidu Says Its New Face Recognition Tech Is Better Than Humans At Checking IDs

“When I go to a Broadway show, I’m not very happy that scalpers have bought up all the tickets,” says Andrew Ng, chief scientist of Baidu Research, the Silicon Valley branch of the tech company often described as the Google of China. Baidu has proposed a new artificial intelligence solution not only to thwart scalpers but to speed up the wait at any place where checking tickets or IDs leads to long lines. Today the tech giant announced new face recognition technology that it says is up to 99.77% accurate—able to distinguish people even better than a human can.

Baidu published two peer-reviewed research papers and tested the technology using popular methodologies, so it’s making more than empty claims. But beyond questions over how and how well these technologies work are questions of how they will be used. What are the implications of an extremely accurate face recognition database, owned by a massive tech and advertising company, that could be subpoenaed by a government?

Of course, Baidu is not alone in evangelizing the benefits of new face-recognition technologies. In recent months, both HSBC Bank and MasterCard have introduced selfie ID verification on their mobile apps (at least in pilot programs) as an alternative to passwords.

But Baidu’s program is potentially more ambitious. It aims not just to match one photo with the image already associated with the card. Instead it will scan everyone in long lines to get into venues, trying to match each scan against potentially huge facial databases—all in one second, vs. 10 seconds for fingerprint scanners that are currently being used in China (which is still faster than manually checking paper or on-screen tickets, says Ng). And that can cut backups. “The difference between one second and 10 seconds is not nine seconds,” says Ng. “It’s the difference between a five-minute wait and a half-hour wait.”

Baidu is starting with a program at the Wuzhen hotel along the waterways of the Xizha Scenic Area near Shanghai. The resort gets 8 million visitors per year, according to Baidu; and tickets are required to enter certain parts of the resort. (Face recognition won’t be used for access to guest rooms.) Baidu intends to market the cloud-based service widely. Ng says that its already working with a company (that he couldn’t name) to make face-recognition door locks for houses. And Ng would love to see it used to thwart scalpers: Someone buying a ticket would have to look into a webcam or smartphone camera so the system could match a face to a ticket (or a limited number of tickets). That would prevent bots from buying up huge blocks of tickets as soon as they go on sale, he believes. Ng also sees the tech being used in security settings like passport checks at airports.

Baidu is using a few tricks to improve accuracy and speed. For starters, it shoots video, not photos, so the system can pick the frame with the best focus and lighting, and without obstructions like another person or a raised hand, to do the analysis on. Ng’s team developed a technique called DenseBox, which he says can use a single neural network to identify a bunch of “bounding boxes” that mark parts of a scene, such as faces (like the yellow boxes that appear around someone’s face when you take a cellphone photo).

The next step, called Face Recognition via Deep Embedding, uses a pair of AI tools called convolutional neural networks (CNNs). CNNs are a staple of image recognition: They mimic the multi-step process in the brain that identifies ever-finer details—from lines, to shapes, to textures, etc. The innovation here, according to Baidu, is the two stages of recognition. The first CNN recognizes the details of a face in high resolution. The second extracts just the key features that are enough to make a comparison. “It takes as input the raw image and distills it down to a vector that captures the essence, the essential shape,” says Ng.

Could it be fooled by holding up a photo? There are some ways for the system to tell if it’s looking at a real human. Does the face move, for instance? Does the person blink? Videos might try to mimic this, but Ng says that Baidu has built other, subtler mechanisms to prevent fraud. He didn’t want to provide all the details, for fear of tipping off scammers.

Despite all the thought Ng put into the technology and all the places it could be used, he didn’t seem to have thought much about the privacy aspect. At least, he seemed a bit surprised when I asked about it, then explained that the project at Wuzhen is pretty small in a country of about 1.36 billion inhabitants. “A lot of people visit Wuzhen, but when you look at it as a fraction of all of China, only a small fraction of China visits,” he says. But Baidu didn’t develop all this technology just to win one client, I say. (Indeed, he already mentioned the lock company.) “I do think this is a growing area,” says Ng.

Baidu will offer facial recognition both as an online service to customers, in which the data is handled on Baidu’s computers, and as software that companies can use in their own systems. The tech can even run on a PC or a high-end cellphone if an internet connection isn’t available of the connection goes down. (It won’t be quite as accurate though, he says, because of the limited computing power.) But by offering a cloud service (and dealing with all the technical issues of the system), Baidu will likely end up hosting a lot of personal data—faces matched to names.

What if the government asks for this data? “I’m not personally involved in that,” says Ng. “Obviously we fully comply with the law in every jurisdiction we operate in.”

That’s the standard answer for many tech companies (though some, like Apple, take a harder stand). Baidu certainly isn’t the only company with a lot of images of users. Think of Facebook, and now financial firms like HSBC and MasterCard with their selfie verification. Giving up our faces might just be another step in how modern technology eats away at privacy.

UPDATED: Baidu contacted us with a higher figure for the number of annual visitors to Wuzhen.