'The first crack in the wall of significance testing'

A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?

If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:


"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"

Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:

"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."

Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:

Follow Neurobonkers on TwitterFacebookGoogle+RSS, or join the mailing list. Image Credit: Adapted from original artwork by Shutterstock.

Reference:

Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991

3D printing might save your life one day. It's transforming medicine and health care.

What can 3D printing do for medicine? The "sky is the limit," says Northwell Health researcher Dr. Todd Goldstein.

Northwell Health
Sponsored by Northwell Health
  • Medical professionals are currently using 3D printers to create prosthetics and patient-specific organ models that doctors can use to prepare for surgery.
  • Eventually, scientists hope to print patient-specific organs that can be transplanted safely into the human body.
  • Northwell Health, New York State's largest health care provider, is pioneering 3D printing in medicine in three key ways.
Keep reading Show less

Adam Gopnik on the rhinoceros of liberalism vs. the unicorns of everything else

Torn between absolutism on the left and the right, classical liberalism—with its core values of compassion and incremental progress whereby the once-radical becomes the mainstream—is in need of a good defense. And Adam Gopnik is its lawyer.

Think Again Podcasts
  • Liberalism as "radical pragmatism"
  • Intersectionality and civic discourse
  • How "a thousand small sanities" tackled drunk driving, normalized gay marriage, and could control gun violence
Keep reading Show less

Why the south of Westeros is the north of Ireland

As Game of Thrones ends, a revealing resolution to its perplexing geography.

Image: YouTube / Doosh
Strange Maps
  • The fantasy world of Game of Thrones was inspired by real places and events.
  • But the map of Westeros is a good example of the perplexing relation between fantasy and reality.
  • Like Britain, it has a Wall in the North, but the map only really clicks into place if you add Ireland.
Keep reading Show less

Fascism and conspiracy theories: The symptoms of broken communication

The lost practice of face-to-face communication has made the world a more extreme place.

Videos
  • The world was saner when we spoke face-to-face, argues John Cameron Mitchell. Not looking someone in the eye when you talk to them raises the potential for miscommunication and conflict.
  • Social media has been an incredible force for activism and human rights, but it's also negatively affected our relationship with the media. We are now bombarded 24/7 with news that either drives us to anger or apathy.
  • Sitting behind a screen makes polarization worse, and polarization is fertile ground for conspiracy theories and fascism, which Cameron describes as irrationally blaming someone else for your problems.
Keep reading Show less