In August 2017 in a courtroom in Cleveland, Ohio, Hercules Shepherd Jr was waiting to be arraigned after the 18-year-old was arrested for possession of a small bag of cocaine.

The judge presiding over the case had a new tool that day to help guide his decision: risk-assessment software that analyses case files to predict future behaviour. Shepherd had been marked as a low-risk defendant and therefore suitable for pretrial release. He was released the same night and back at school the next day.

© iStock/PhonlamaiPhoto
© iStock/PhonlamaiPhoto

In a growing number of courtrooms around the world, judges are being guided by similar automated decision-making systems, but not every algorithm is producing fair results.

A 2016 ProPublica investigation found that a popular risk assessment tool called Compas was twice as likely to misclassify black defendants as future criminals than white defendants.

The shortcomings of automated judges are causing justified alarm, but their human counterparts can have far more serious defects.

Professor Elliott Ash, an assistant professor of economics at the University of Warwick, believes that human judges can be even more biased and far more impulsive. Ash's analysis of contemporary judicial systems reveals the arbitrary nature of many legal decisions.

In the US, the judge to which an asylum seeker is randomly assigned can double the chances of their claim being granted. In Newark, the chances can go from 10 percent to 90 percent of being granted - depending on the judge you get.

There is also endless evidence of discriminatory outcomes against racial minorities, from their chances of getting stopped to the severity of their sentence.

Even breakfast can be decisive in a judge's decision.  A peer-reviewed study of more than 1,000 rulings in Israel found that judges gave far more lenient decisions at the start of the day and immediately after their lunch break.

"You are anywhere between two and six times as likely to be released if you're one of the first three prisoners considered versus the last three prisoners considered," Jonathan Levav, one of the Columbia University research team, told the Guardian.

A robot judge could cut out a temporary mood or a deeper personal prejudice to make an objective decision based purely on the evidence.

"So," asked Ash in a talk at the headquarters of think tank the Social Market Foundation."Do you want a judge, a jury or an EXEcute file?"

Automated judging

The basic methodology behind automated judging is fairly simple. Machine learning algorithms are trained on legal rules and previous case decisions to understand the factors that best predict guilt or innocence.

When a new case arrives, the evidence is converted into machine readable data and analysed by the system to make a decision.

There is now significant evidence that one can accurately predict a judge's decision using only the publicly available data collected for administrative purposes.

Ash points to studies that have predicted 67 percent of US bankruptcy court decisions, 70 percent of Supreme Court decisions, and up to 88 percent of routine prosecution decisions.

Those percentages may be too low to justify automated decision-making, but they could be a lot higher if they included data collected for predictive purposes. They may never achieve perfect accuracy, but they would be more consistent than a human judge.

"Even if it's not 100 percent, we have this advantage that it won't depend on what judge you get, or if they've had lunch," said Ash.

The robot clerk

Legal automation can net out personal prejudice, but a systematic bias will be reproduced and possibly amplified. Racial discrimination has already been encoded in risk assessment software, as the ProPublica study showed. 

"It's not as simple as leaving these measures out," explained Ash. "You can't just tell it to ignore race or income or zip code. Whatever variables that you use, they're going to be correlated with other factors. Any prejudice in the system is likely to be reproduced in automated decisions, and having a lot of human involvement in the system is important for that reason."

There are further limitations. Black boxes lack transparency as they don't explain their decisions. Algorithms can also be gamed by people with inside knowledge about the systems, and if there are different models available, they could pick the one that will produce their preferred outcome.

Making the systems open source would make the decisions easier to explain but also easier to predict.

"One compromise solution might be the actual weights on the evidence could be private, but the actual code that's used to construct those parameters could be open source," said Ash. "That's what I would advocate."

The systems would also struggle to assess uncommon cases. Unusual circumstances will be harder to analyse and algorithms will not easily adapt to new laws and moral norms.

There is also a risk that judges rely on the technology to such an extent that they neglect their own critical thinking. This faith in machines could endanger everyone involved. Ash has heard of a court in China that gains trust in its decisions by claiming that they're automated, when in fact they're made by a human.

Ash believes a robot clerk could be a good short-term option for legal automation. It could tell judges what colleagues have done in similar cases and help them quickly identify patterns in cases with clear outcomes. The robot clerk could also include an assessment of its level of certainty to help the judge decide whether to use the advice. 

"It might not change their decisions but it will make them think again," said Ash.

The AI judge

"There's also a hard way," added Ash. "Legal artificial intelligence that can understand and apply the law, explain its reasoning and then to have this optimal social policy development process to improve social outcomes."

This would amplify the dangers of the robot clerk, as a true form of artificial intelligence could independently learn to make its own judgements, which may not match human morality or policy objectives. 

"This is unproven technology," admitted Ash. "The AI is controversial. One of the reasons it's controversial is because we think of justice as this important factor or value that separates man from machine and if we have the robot judge trying to make value judgements this is quite delicate for our sensibilities. But this is happening at some point. The trend is moving in this direction."

"What we're looking at now is a call to action: a way of democratic non-profit and open source solutions and we should develop a community between private sector, public sector and academia to make sure those solutions are implemented."