This algorithm is quickly clearing old marijuana convictions in San Francisco

When Proposition 64 legalized marijuana in California, it also meant that people with old marijuana convictions could petition to have those offenses taken off their criminal record or downgraded to lower-level crimes. That process takes time and money, and sets up a lot of obstacles for people who may not be able to hire an attorney or take time off work. In San Francisco, the district attorney’s office is helping by going through records itself, and doing it very quickly, with the help of an algorithm.

A new machine learning algorithm developed by the nonprofit Code for America can read through charging documents, identify codes for various crimes, and then automatically determine which felony convictions can be downgraded to a misdemeanor (those who also committed violent crimes, for example, can’t have their records downgraded). Then the tool automatically fills out required forms that the district attorney can file with the court.

For those with a criminal record, the changes could have meaningful impacts on their lives. “If you have a felony conviction, or in many cases, if you have a misdemeanor conviction, there are many employers who will not hire you,” says George Gascón, San Francisco district attorney. “There are many landlords that will not allow you to rent or lease a place for them. There are certain types of student loans that you would not qualify because of the felony conviction. So we know that having a felony conviction–for offenses that have been legalized–still holds back a lot of people. Mostly, quite frankly, poor people, and people in communities of color.”

Though rates of marijuana use are similar in black and white populations, nationally, someone who is black is nearly four times as likely to be arrested for possession. In California, until marijuana use was legalized, black people were more than twice as likely to be arrested.

Clearing records can also improve public safety, Gascón says. “There are two major components to reducing the likelihood that people will commit crimes. One is employment and the other one is housing. If you have a steady job and you have housing, you’re less likely to engage in other criminal activity.”

The D.A.’s office can fairly quickly process misdemeanor records, which don’t involve any discretion on the part of the prosecutors–if you have a misdemeanor for marijuana, it can be expunged. But felonies are more complex. It might take a well-trained paralegal 15 minutes to review someone’s criminal history, and that process would need to be repeated across thousands of records. The algorithm can process hundreds of records in a matter of minutes.

Code for America started looking at the issue of expunging criminal records because of another California law, Proposition 47. The law intended to get some low-level offenses reclassified as misdemeanors, but few people were actually getting their records cleared. When the nonprofit started working on the issue in 2016, through a project called Clear My Record, it helped digitize the process, and made changes like sending people notifications by text instead of through the mail. It helped–7,000 people signed up. But “we could see that it really would not scale,” says Code for America founder and executive director Jennifer Pahlka. The team began working on an algorithm to automatically move from rap sheets to completed applications, and that has now been adapted for the San Francisco D.A.’s office.

It’s important, she says, that the technology came out of a lengthy process of understanding what challenges existed in the system. “I think if you look at certain attempts to use technology within complex bureaucratic systems, you’ll very often have people writing a beautiful algorithm, but for a problem that’s the wrong problem,” Pahlka says. “What I’m proud of our team doing is the work to figure out where the real problem is.”

After testing the tool, San Francisco now plans to begin using it, and the team also plans to share it with other district attorneys in the state and in other parts of the country. “This entire system will be put in the public domain so anybody can use it,” says Gascón.