ACLU: Amazon’s controversial Rekognition tech mistakes members of Congress for mugshot subjects
In a new experiment conducted by the American Civil Liberties Union, Amazon’s image recognition tool falsely matched 28 members of Congress to mugshots of other people who had been arrested for a crime, according to the ACLU.
The civil liberties group built a database using 25,000 publicly available mugshot photos–it’s not disclosing the exact source of the images–and used Amazon’s Rekognition tool to search that database for matches for each member of the Senate and House of Representatives.
The process cost just $12.33 but turned up 28 false matches, of which 39% of them were for people of color. Only 20% of the members of Congress are people of color, according to the ACLU, and previous research has shown facial recognition technology often has a harder time identifying people of color and women than it does matching white male faces. The experiment used Rekognition’s default settings, including reporting a match on a similarity score of 80 out of a possible 100, says Jacob Snow, a technology and civil liberties attorney with the ACLU of Northern California.
“I think these results emphasize that there are applications of face surveillance that are not safe,” Snow tells Fast Company. “You could imagine a police officer getting a match that somebody has, for example, a concealed weapons arrest and that’s going to bias the police officer.”
The ACLU is calling for Congress to impose a moratorium on law enforcement use of facial recognition.
“Every match that we got was incorrect,” Snow says. “I think that’s far too high of an error rate to be used by law enforcement.”
Members of the Congressional Black Caucus wrote in May to Amazon CEO Jeff Bezos to express concern about facial recognition and Amazon’s tool in particular.
“It is quite clear that communities of color are more heavily and aggressively policed than white communities,” wrote Cedric Richmond, a Louisiana Democratic in the House and the chair of the Black Caucus. “This status quo results in an oversampling of data, which, once used as inputs to an analytical framework leveraging artificial intelligence, could negatively impact outcomes in those oversampled communities.”
Amazon’s Rekognition, part of the Amazon Web Services suite of cloud services, is far from the only commercially available image recognition tool: Microsoft offers a Face API to match faces and give similarity ratings, IBM offers facial recognition tools, and a variety of open-source artificial intelligence tools exist designed specifically for image recognition or trainable for that purpose. In a statement released to Fast Company, an Amazon Web Services wrote:
We have seen customers use the image and video analysis capabilities of Amazon Rekognition in ways that materially benefit both society (e.g. preventing human trafficking, inhibiting child exploitation, reuniting missing children with their families, and building educational apps for children), and organizations (enhancing security through multi-factor authentication, finding images more easily, or preventing package theft). We remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement.
With regard to this recent test of Amazon Rekognition by the ACLU, we think that the results could probably be improved by following best practices around setting the confidence thresholds (this is the percentage likelihood that Rekognition found a match) used in the test. While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty. When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95% or higher.
Finally, it is worth noting that in real world scenarios, Amazon Rekognition is almost exclusively used to help narrow the field and allow humans to expeditiously review and consider options using their judgement (and not to make fully autonomous decisions), where it can help find lost children, restrict human trafficking, or prevent crimes.
Snow says Rekognition was selected for the test in part because of evidence that it’s been marketed to law enforcement, and the ACLU hasn’t tested how it performs compared to competing products. The group has previously called upon Amazon to stop offering the tool to law enforcement agencies. Some Amazon shareholders have done the same, citing the potential for bias as well as privacy risks. Law enforcement groups have already reported using Amazon’s tech to locate and identify suspects.
Washington County, Oregon, for instance, has used the system to index about 300,000 mugshots from its “jail management system,” enabling Sheriff’s Office officials to identify suspects from surveillance video and even locate a “person of interest” who was posting material under a pseudonym on Facebook, according to a post last year on an Amazon machine learning blog.
“Amazon Rekognition has become a powerful tool for identifying suspects for my agency,” wrote Chris Adzima, a senior information systems analyst at the Washington County Sheriff’s Office. “As the service improves, I hope to make it the standard for facial recognition in law enforcement.”
Facial recognition technology, not necessarily from Amazon, has been increasingly used in other law enforcement contexts as well: New York Governor Andrew Cuomo recently said the state is testing searching for criminal suspects at the state’s bridges and tunnels by scanning motorists’ faces. Customs and Border Protection officials have also embraced (though not without some controversy) facial recognition as an alternative to paper boarding passes and passports at airports and border crossings.
“I think this is an issue that is really in the front of the public’s mind,” Snow says.
This story has been updated.
(12)
