Most New Psychology Findings Can’t Be Replicated. So Now What?

A massive, groundbreaking study has found that the majority of new psychology findings in the top three flagship journals can't be replicated. Where do we go from here?

The field of psychology has been shaken by a massive replication effort, which has found that out of 98 papers published in the top three psychology journals only 39 could be replicated.


From much of the coverage you’d be forgiven for thinking that this means we should discard psychology as a field. I certainly don’t believe this is the case for the simple reason that well-studied areas of psychology are full of findings that have been repeatedly tested in various different ways. Instead of resorting to cynicism, we should take this opportunity to embrace a culture of skepticism and fundamentally change the way we interpret new research findings.

Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.

... taking a scientifically informed view of the world requires a great deal more than looking at any one study.

In that last sentence I was careful not to restrict that claim to psychology, for the simple reason that we have every indication to believe that new research is similarly difficult to replicate across many areas of science. For example, 30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials — a number that is in fact worse than the rate found by the psychology reproducibility project.

So what does this mean to you, the reader, who is simply trying to distinguish fact from fiction? Obviously, it is the responsibility of researchers to make their research considerably more reliable, but this may take a very long time, and may never happen.

The solution is to take a skeptical approach to the world around us, to treat every new claim not as a problem solved, but as an open question. Questions that are best answered by looking with a critical eye at all the evidence we have. In a perfect world, I’d wholly agree with psychologist Steven Pinker that much of this responsibility lies with science journalists:

I've long argued that sci journalists should stop reporting single newsworthy studies = recipe for error. Only meta-analyses, lit reviews.

— Steven Pinker (@sapinker) August 30, 2015

​There are two kinds of failure – but only one is honorable

Malcolm Gladwell teaches "Get over yourself and get to work" for Big Think Edge.

Big Think Edge
  • Learn to recognize failure and know the big difference between panicking and choking.
  • At Big Think Edge, Malcolm Gladwell teaches how to check your inner critic and get clear on what failure is.
  • Subscribe to Big Think Edge before we launch on March 30 to get 20% off monthly and annual memberships.
Keep reading Show less

Is this why time speeds up as we age?

We take fewer mental pictures per second.

(MPH Photos/giphy/yShutterstock/Big Think)
Mind & Brain
  • Recent memories run in our brains like sped-up old movies.
  • In childhood, we capture images in our memory much more quickly.
  • The complexities of grownup neural pathways are no match for the direct routes of young brains.
Keep reading Show less

New alternative to Trump's wall would create jobs, renewable energy, and increase border security

A consortium of scientists and engineers have proposed that the U.S. and Mexico build a series of guarded solar, wind, natural gas and desalination facilities along the entirety of the border.

Credit: Purdue University photo/Jorge Castillo Quiñones
Politics & Current Affairs
  • The proposal was recently presented to several U.S. members of Congress.
  • The plan still calls for border security, considering all of the facilities along the border would be guarded and connected by physical barriers.
  • It's undoubtedly an expensive and complicated proposal, but the team argues that border regions are ideal spots for wind and solar energy, and that they could use the jobs and fresh water the energy park would create.
Keep reading Show less

Why are so many objects in space shaped like discs?

It's one of the most consistent patterns in the unviverse. What causes it?

Videos
  • Spinning discs are everywhere – just look at our solar system, the rings of Saturn, and all the spiral galaxies in the universe.
  • Spinning discs are the result of two things: The force of gravity and a phenomenon in physics called the conservation of angular momentum.
  • Gravity brings matter together; the closer the matter gets, the more it accelerates – much like an ice skater who spins faster and faster the closer their arms get to their body. Then, this spinning cloud collapses due to up and down and diagonal collisions that cancel each other out until the only motion they have in common is the spin – and voila: A flat disc.