Most New Psychology Findings Can’t Be Replicated. So Now What?

A massive, groundbreaking study has found that the majority of new psychology findings in the top three flagship journals can't be replicated. Where do we go from here?

Most New Psychology Findings Can’t Be Replicated. So Now What?

The field of psychology has been shaken by a massive replication effort, which has found that out of 98 papers published in the top three psychology journals only 39 could be replicated.


From much of the coverage you’d be forgiven for thinking that this means we should discard psychology as a field. I certainly don’t believe this is the case for the simple reason that well-studied areas of psychology are full of findings that have been repeatedly tested in various different ways. Instead of resorting to cynicism, we should take this opportunity to embrace a culture of skepticism and fundamentally change the way we interpret new research findings.

Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.

... taking a scientifically informed view of the world requires a great deal more than looking at any one study.

In that last sentence I was careful not to restrict that claim to psychology, for the simple reason that we have every indication to believe that new research is similarly difficult to replicate across many areas of science. For example, 30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials — a number that is in fact worse than the rate found by the psychology reproducibility project.

So what does this mean to you, the reader, who is simply trying to distinguish fact from fiction? Obviously, it is the responsibility of researchers to make their research considerably more reliable, but this may take a very long time, and may never happen.

The solution is to take a skeptical approach to the world around us, to treat every new claim not as a problem solved, but as an open question. Questions that are best answered by looking with a critical eye at all the evidence we have. In a perfect world, I’d wholly agree with psychologist Steven Pinker that much of this responsibility lies with science journalists:

I've long argued that sci journalists should stop reporting single newsworthy studies = recipe for error. Only meta-analyses, lit reviews.

— Steven Pinker (@sapinker) August 30, 2015

No, the Yellowstone supervolcano is not ‘overdue’

Why mega-eruptions like the ones that covered North America in ash are the least of your worries.

Ash deposits of some of North America's largest volcanic eruptions.

Image: USGS - public domain
Strange Maps
  • The supervolcano under Yellowstone produced three massive eruptions over the past few million years.
  • Each eruption covered much of what is now the western United States in an ash layer several feet deep.
  • The last eruption was 640,000 years ago, but that doesn't mean the next eruption is overdue.
Keep reading Show less

Smartly dressed: Researchers develop clothes that sense movement via touch

Measuring a person's movements and poses, smart clothes could be used for athletic training, rehabilitation, or health-monitoring.

Technology & Innovation

In recent years there have been exciting breakthroughs in wearable technologies, like smartwatches that can monitor your breathing and blood oxygen levels.

Keep reading Show less

Do you worry too much? Stoicism can help

How imagining the worst case scenario can help calm anxiety.

Stoicism can help overcome anxiety

Credit: OLIVIER DOULIERY via Getty Images
Personal Growth
  • Stoicism is the philosophy that nothing about the world is good or bad in itself, and that we have control over both our judgments and our reactions to things.
  • It is hardest to control our reactions to the things that come unexpectedly.
  • By meditating every day on the "worst case scenario," we can take the sting out of the worst that life can throw our way.
Keep reading Show less
Quantcast