Most New Psychology Findings Can’t Be Replicated. So Now What?

A massive, groundbreaking study has found that the majority of new psychology findings in the top three flagship journals can't be replicated. Where do we go from here?

The field of psychology has been shaken by a massive replication effort, which has found that out of 98 papers published in the top three psychology journals only 39 could be replicated.


From much of the coverage you’d be forgiven for thinking that this means we should discard psychology as a field. I certainly don’t believe this is the case for the simple reason that well-studied areas of psychology are full of findings that have been repeatedly tested in various different ways. Instead of resorting to cynicism, we should take this opportunity to embrace a culture of skepticism and fundamentally change the way we interpret new research findings.

Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.

... taking a scientifically informed view of the world requires a great deal more than looking at any one study.

In that last sentence I was careful not to restrict that claim to psychology, for the simple reason that we have every indication to believe that new research is similarly difficult to replicate across many areas of science. For example, 30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials — a number that is in fact worse than the rate found by the psychology reproducibility project.

So what does this mean to you, the reader, who is simply trying to distinguish fact from fiction? Obviously, it is the responsibility of researchers to make their research considerably more reliable, but this may take a very long time, and may never happen.

The solution is to take a skeptical approach to the world around us, to treat every new claim not as a problem solved, but as an open question. Questions that are best answered by looking with a critical eye at all the evidence we have. In a perfect world, I’d wholly agree with psychologist Steven Pinker that much of this responsibility lies with science journalists:

I've long argued that sci journalists should stop reporting single newsworthy studies = recipe for error. Only meta-analyses, lit reviews.

— Steven Pinker (@sapinker) August 30, 2015

Befriend your ideological opposite. It’s fun.

Step inside the unlikely friendship of a former ACLU president and an ultra-conservative Supreme Court Justice.

Sponsored by Charles Koch Foundation
  • Former president of the ACLU Nadine Strossen and Supreme Court Justice Antonin Scalia were unlikely friends. They debated each other at events all over the world, and because of that developed a deep and rewarding friendship – despite their immense differences.
  • Scalia, a famous conservative, was invited to circles that were not his "home territory", such as the ACLU, to debate his views. Here, Strossen expresses her gratitude and respect for his commitment to the exchange of ideas.
  • "It's really sad that people seem to think that if you disagree with somebody on some issues you can't be mutually respectful, you can't enjoy each other's company, you can't learn from each other and grow in yourself," says Strossen.
  • The opinions expressed in this video do not necessarily reflect the views of the Charles Koch Foundation, which encourages the expression of diverse viewpoints within a culture of civil discourse and mutual respect.
Keep reading Show less

3 ways to find a meaningful job, or find purpose in the job you already have

Learn how to redesign your job for maximum reward.

Videos
  • Broaching the question "What is my purpose?" is daunting – it's a grandiose idea, but research can make it a little more approachable if work is where you find your meaning. It turns out you can redesign your job to have maximum purpose.
  • There are 3 ways people find meaning at work, what Aaron Hurst calls the three elevations of impact. About a third of the population finds meaning at an individual level, from seeing the direct impact of their work on other people. Another third of people find their purpose at an organizational level. And the last third of people find meaning at a social level.
  • "What's interesting about these three elevations of impact is they enable us to find meaning in any job if we approach it the right way. And it shows how accessible purpose can be when we take responsibility for it in our work," says Hurst.
Keep reading Show less

Physicist advances a radical theory of gravity

Erik Verlinde has been compared to Einstein for completely rethinking the nature of gravity.

Photo by Willeke Duijvekam
Surprising Science
  • The Dutch physicist Erik Verlinde's hypothesis describes gravity as an "emergent" force not fundamental.
  • The scientist thinks his ideas describe the universe better than existing models, without resorting to "dark matter".
  • While some question his previous papers, Verlinde is reworking his ideas as a full-fledged theory.
Keep reading Show less

UPS has been discreetly using self-driving trucks to deliver cargo

TuSimple, an autonomous trucking company, has also engaged in test programs with the United States Postal Service and Amazon.


PAUL RATJE / Contributor
Technology & Innovation
  • This week, UPS announced that it's working with autonomous trucking startup TuSimple on a pilot project to deliver cargo in Arizona using self-driving trucks.
  • UPS has also acquired a minority stake in TuSimple.
  • TuSimple hopes its trucks will be fully autonomous — without a human driver — by late 2020, though regulatory questions remain.
Keep reading Show less