In the hands of police, facial recognition software risks violating civil liberties

California has long been at the forefront of strong, progressive public policy.

Former President Barack Obama once hailed it as a model for the rest of the country, calling on Congress “to catch up to California,” after the state passed the nation’s most aggressive expansion of paid family leave.    

The Golden State has taken another bold step toward protecting its residents. 

Last week, California temporarily banned the use of facial-recognition technology in body cameras used by law enforcement — one of the first states to do so. And once again, it’s time for the rest of the country to follow suit and implement safeguards to regulate this technology before it can be used to compromise public safety and violate our civil rights.

Facial recognition uses artificial intelligence to track objects and faces; anticipates what’s important to the user; and scans images against databases with millions of faces. It’s touted as one of the most exciting advancements in tech.

But limited information about how it’s being used by law enforcement, and a disturbing lack of transparency and accountability from the very companies that develop and sell it, mean we need to consider its dangers, including its proven racial and gender bias. In some circumstances, it's easy to see how the technology could potentially cost a life.

POLICING THE USA:A look at race, justice, media

Amazon is one of the nation's largest distributors of the software and is already selling and pushing its product, Rekognition, to police departments in cities across the country. 

But an American Civil Liberties Union (ACLU) test of Rekognition netted scary results.  Headshots of 28 members of Congress were mistaken for people who had been arrested for crimes. While men and women of every race were misidentified, the level of inaccuracy was significantly higher for people of color.

About 40% of the false matches Rekognition generated were of people of color, according to the ACLU test.

If the software is improperly used by police, that stat could mean even more trouble for black men and boys, who are already more than twice as likely to die during an encounter with police than their white counterparts.   

In another study released this year by MIT's Media Lab, researchers demonstrated Rekognition’s inherent gender and racial bias.

COLUMN:For black shooting victims, sometimes anger (not forgiveness) is the best response

The software disproportionately misidentified dark-skinned women as men 31% of the time, compared with 7% for lighter-skinned women. And while facial recognition tools created by IBM and Microsoft performed better, they also misidentified people of color at a significantly higher rate than whites.

Amazon and other technology companies are selling facial recognition systems to governments without any public transparency or accountability about the potential pitfalls. And none of these companies nor their government clients has put proper protections in place for those who may be harmed.

Rekognition has been piloted by the FBI. Amazon has marketed Rekognition to Immigration and Customs Enforcement. With its real-time capabilities and extensive database of faces, this technology has the potential to create a level of government surveillance that neither Congress nor the American people have consented to.

In a report released during Sunshine Week — a national observance of the importance of open access to information — Open the Government spotlighted the dearth of accountability and transparency for private technology companies, even as they execute government contracts related to policing, immigration enforcement and national security with facial recognition technologies. Until we can ensure that national security and our most vulnerable populations are not threatened by this technology, we need to put the brakes on these local law enforcement agency contracts.

In addition to the ban in California, San Francisco had its own ban and put in place policies to review related surveillance tech already in use. Somerville, Massachusetts, followed suit with a ban soon after.

LIFERS:Nation's failed weed war turned many into prisoners and others into moguls

And in Detroit, protesters and state representatives alike have demanded a moratorium on facial recognition, claiming it enables racial biases in the police force due to its inaccuracies. The use of facial recognition has even unified some Democrats and Republicans on the House Committee on Oversight and Reform.   

The potentially serious risks of facial recognition technology warrant much needed checks and balances in both the private and public sectors.

Companies should put people’s lives above their bottom lines by limiting the sale of the technology until they can proactively ensure safeguards that prevent abuse, as well as hold themselves to higher ethics and transparency standards.

Local, state and federal governments must exercise oversight to combat the limitations of the technology by requiring third parties to conduct public accuracy and bias tests. 

Cutting-edge technology can transform societies for the better, but only if it does so without threatening our civil rights, our liberties and our lives.

Rep. Jimmy Gomez represents California’s 34th congressional district. He's a member of the Congressional Hispanic Caucus and serves on the House Committee on Oversight and Reform. Follow him on Twitter: @RepJimmyGomez.

Lisa Rosenberg is the executive director of Open the Government, a nonpartisan coalition focused on government transparency and accountability. Prior, she launched many of the Sunlight Foundation’s advocacy initiatives. Follow her on Twitter: @LisaRosenbrg.