Algorithms identify repeat offenders better than judges

Can AI make better predictions about future crimes?

Algorithms identify repeat offenders better than judges
Image source: Andrey_Popov/Shutterstock
  • A new study finds algorithmic predictions of recidivism more accurate than human authorities.
  • Researchers are trying to construct tests of such AI that accurately mirror real-world deliberations.
  • What level of reliability should we demand of AI in sentencing?

It's pre-crime time again. (See Minority Report.)

When judges, correctional authorities, and parole boards are making sentencing, supervision, and release decisions, they're essentially trying to peer into an offender's future to assess the person's potential for recidivism. To help guide these determinations — and no doubt influenced by our contemporary infatuation with artificial intelligence — authorities are increasingly turning to risk assessment instruments (RAIs) on the assumption that their AI can more accurately identify those likely to be repeat offenders.

A new study in Science Advances more rigorously confirms that algorithmic judgements may in fact be more accurate than humans. Of concern, though, is that given the stakes involved — future crimes, a defendant's freedom or continued incarceration — they're still not reliable enough to ensure that justice is truly done and that tragic mistakes can be avoided.


Image source: Andrey Suslov/Shutterstock

The new study, led by computational social scientist Sharad Goel of Stanford University, is in a sense a reply to a recent work by programming expert Julia Dressel and digital image specialist Hany Farid. In that earlier research, participants attempted to predict whether or not any of 50 individuals would commit new crimes of any kind within the next two years based on short descriptions of their case histories. (No images or racial/ethnic information were provided to participants to avoid a skewing of results due to related biases.) The average accuracy rate participants achieved was 62%.

The same criminals and case histories cases were also processed through a widely used RAI called COMPAS, for "Correctional Offender Management Profiling for Alternative Sanctions." The accuracy of its predictions was about the same: 65%, leading Dressel and Farid to conclude that COMPAS "is no more accurate … than predictions made by people with little or no criminal justice expertise."

Taking a second look

Goel felt that two aspects of the testing method used by Dressel and Farid didn't reproduce closely enough the circumstances in which humans are called upon to predict recidivism during sentencing:

  1. Participants in that study learned how to improve their predictions, much as an algorithm might, as they were provided feedback as to the accuracy of each prognostication. However, as Goel points out, "In justice settings, this feedback is exceedingly rare. Judges may never find out what happens to individuals that they sentence or for whom they set bail."
  2. Judges, etc. also often have a great deal of information in hand as they make their predictions, not short summaries in which only the most salient information is presented. In the real world, it can be hard to ascertain which information is the most relevant when there's arguably too much of it at hand.

Both of these factors put participants on a more equal footing with an RAI than they would be in real life, perhaps accounting for the similar levels of accuracy encountered.

To that end, Goel and his colleagues performed several of their own, slightly different, trials.

The first experiment closely mirrored Dressel's and Farid's — with feedback and short case descriptions — and indeed found that humans and COMPAS performed pretty much equally well. Another experiment asked participants to predict the future occurrence of violent crime, not just any crime, and again the accuracy rates were comparable, though much higher. Humans scored 83% as COMPAS achieved 89% accuracy.

When participant feedback was removed, however, humans fell far behind COMPAS in accuracy, down to around 60% as opposed to COMPAS's 89%, as Goel hypothesized they might.

Finally, humans were tested against a different RAI tool called LSI-R. In this case, both had to try and predict an individual's future using on a large amount of case information similar to what a judge may have to wade through. Again, the RAI outperformed humans in predicting future crimes, 62% to 57%. When asked to predict who would wind up going back to prison for their future misdeeds, the results were even worse for participants, who got it right just 58% of the time as opposed to 74% for LSI-R.

Good enough?

Image source: klss/Shutterstock

Goel concludes, "our results support the claim that algorithmic risk assessments can often outperform human predictions of reoffending." Of course, this isn't the only important question. There's also this: Is AI yet reliable enough to make its prediction count for more than that of a judge, correctional authority, or parole board member?

Science News asked Farid, and he said no. When asked how he'd feel about an RAI that could be counted on to be right 80% of the time, he responded, "you've got to ask yourself, if you're wrong 20 percent of the time, are you willing to tolerate that?"

As AI technology improves, we may one day reach a state in which RAIs are reliably accurate, but no one is claiming we're there yet. For now, then, the use of such technologies in an advisory role for authorities tasked with making sentencing decisions may make sense, but only as one more "voice" to consider.

A landslide is imminent and so is its tsunami

An open letter predicts that a massive wall of rock is about to plunge into Barry Arm Fjord in Alaska.

Image source: Christian Zimmerman/USGS/Big Think
Surprising Science
  • A remote area visited by tourists and cruises, and home to fishing villages, is about to be visited by a devastating tsunami.
  • A wall of rock exposed by a receding glacier is about crash into the waters below.
  • Glaciers hold such areas together — and when they're gone, bad stuff can be left behind.

The Barry Glacier gives its name to Alaska's Barry Arm Fjord, and a new open letter forecasts trouble ahead.

Thanks to global warming, the glacier has been retreating, so far removing two-thirds of its support for a steep mile-long slope, or scarp, containing perhaps 500 million cubic meters of material. (Think the Hoover Dam times several hundred.) The slope has been moving slowly since 1957, but scientists say it's become an avalanche waiting to happen, maybe within the next year, and likely within 20. When it does come crashing down into the fjord, it could set in motion a frightening tsunami overwhelming the fjord's normally peaceful waters .

"It could happen anytime, but the risk just goes way up as this glacier recedes," says hydrologist Anna Liljedahl of Woods Hole, one of the signatories to the letter.

The Barry Arm Fjord

Camping on the fjord's Black Sand Beach

Image source: Matt Zimmerman

The Barry Arm Fjord is a stretch of water between the Harriman Fjord and the Port Wills Fjord, located at the northwest corner of the well-known Prince William Sound. It's a beautiful area, home to a few hundred people supporting the local fishing industry, and it's also a popular destination for tourists — its Black Sand Beach is one of Alaska's most scenic — and cruise ships.

Not Alaska’s first watery rodeo, but likely the biggest

Image source:

There have been at least two similar events in the state's recent history, though not on such a massive scale. On July 9, 1958, an earthquake nearby caused 40 million cubic yards of rock to suddenly slide 2,000 feet down into Lituya Bay, producing a tsunami whose peak waves reportedly reached 1,720 feet in height. By the time the wall of water reached the mouth of the bay, it was still 75 feet high. At Taan Fjord in 2015, a landslide caused a tsunami that crested at 600 feet. Both of these events thankfully occurred in sparsely populated areas, so few fatalities occurred.

The Barry Arm event will be larger than either of these by far.

"This is an enormous slope — the mass that could fail weighs over a billion tonnes," said geologist Dave Petley, speaking to Earther. "The internal structure of that rock mass, which will determine whether it collapses, is very complex. At the moment we don't know enough about it to be able to forecast its future behavior."

Outside of Alaska, on the west coast of Greenland, a landslide-produced tsunami towered 300 feet high, obliterating a fishing village in its path.

What the letter predicts for Barry Arm Fjord

Moving slowly at first...

Image source:

"The effects would be especially severe near where the landslide enters the water at the head of Barry Arm. Additionally, areas of shallow water, or low-lying land near the shore, would be in danger even further from the source. A minor failure may not produce significant impacts beyond the inner parts of the fiord, while a complete failure could be destructive throughout Barry Arm, Harriman Fiord, and parts of Port Wells. Our initial results show complex impacts further from the landslide than Barry Arm, with over 30 foot waves in some distant bays, including Whittier."

The discovery of the impeding landslide began with an observation by the sister of geologist Hig Higman of Ground Truth, an organization in Seldovia, Alaska. Artist Valisa Higman was vacationing in the area and sent her brother some photos of worrying fractures she noticed in the slope, taken while she was on a boat cruising the fjord.

Higman confirmed his sister's hunch via available satellite imagery and, digging deeper, found that between 2009 and 2015 the slope had moved 600 feet downhill, leaving a prominent scar.

Ohio State's Chunli Dai unearthed a connection between the movement and the receding of the Barry Glacier. Comparison of the Barry Arm slope with other similar areas, combined with computer modeling of the possible resulting tsunamis, led to the publication of the group's letter.

While the full group of signatories from 14 organizations and institutions has only been working on the situation for a month, the implications were immediately clear. The signers include experts from Ohio State University, the University of Southern California, and the Anchorage and Fairbanks campuses of the University of Alaska.

Once informed of the open letter's contents, the Alaska's Department of Natural Resources immediately released a warning that "an increasingly likely landslide could generate a wave with devastating effects on fishermen and recreationalists."

How do you prepare for something like this?

Image source:

The obvious question is what can be done to prepare for the landslide and tsunami? For one thing, there's more to understand about the upcoming event, and the researchers lay out their plan in the letter:

"To inform and refine hazard mitigation efforts, we would like to pursue several lines of investigation: Detect changes in the slope that might forewarn of a landslide, better understand what could trigger a landslide, and refine tsunami model projections. By mapping the landslide and nearby terrain, both above and below sea level, we can more accurately determine the basic physical dimensions of the landslide. This can be paired with GPS and seismic measurements made over time to see how the slope responds to changes in the glacier and to events like rainstorms and earthquakes. Field and satellite data can support near-real time hazard monitoring, while computer models of landslide and tsunami scenarios can help identify specific places that are most at risk."

In the letter, the authors reached out to those living in and visiting the area, asking, "What specific questions are most important to you?" and "What could be done to reduce the danger to people who want to visit or work in Barry Arm?" They also invited locals to let them know about any changes, including even small rock-falls and landslides.

Your genetics influence how resilient you are to the cold

What makes some people more likely to shiver than others?

Surprising Science

Some people just aren't bothered by the cold, no matter how low the temperature dips. And the reason for this may be in a person's genes.

Keep reading Show less

Harvard study finds perfect blend of fruits and vegetables to lower risk of death

Eating veggies is good for you. Now we can stop debating how much we should eat.

Credit: Pixabay
Surprising Science
  • A massive new study confirms that five servings of fruit and veggies a day can lower the risk of death.
  • The maximum benefit is found at two servings of fruit and three of veggies—anything more offers no extra benefit according to the researchers.
  • Not all fruits and veggies are equal. Leafy greens are better for you than starchy corn and potatoes.
Keep reading Show less