'The first crack in the wall of significance testing'

A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?

If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:


"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"

Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:

"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."

Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:

Follow Neurobonkers on TwitterFacebookGoogle+RSS, or join the mailing list. Image Credit: Adapted from original artwork by Shutterstock.

Reference:

Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991

Got a question for a real NASA astronomer? Ask it here!

NASA astronomer Michelle Thaller is coming back to Big Think to answer YOUR questions! Here's all you need to know to submit your science-related inquiries.

Surprising Science

Big Think's amazing audience has responded so well to our videos from NASA astronomer and Assistant Director for Science Communication Michelle Thaller that we couldn't wait to bring her back for more!

And this time, she's ready to tackle any questions you're willing to throw at her, like, "How big is the Universe?", "Am I really made of stars?" or, "How long until Elon Musk starts a colony on Mars?"

All you have to do is submit your questions to the form below, and we'll use them for an upcoming Q+A session with Michelle. You know what to do, Big Thinkers!

Keep reading Show less

The value of owning more books than you can read

Or how I learned to stop worrying and love my tsundoku.

(Photo from Wikimedia)
Personal Growth
  • Many readers buy books with every intention of reading them only to let them linger on the shelf.
  • Statistician Nassim Nicholas Taleb believes surrounding ourselves with unread books enriches our lives as they remind us of all we don't know.
  • The Japanese call this practice tsundoku, and it may provide lasting benefits.
Keep reading Show less

Take the Big Think survey for a chance to win :)

Calling all big thinkers!

  • Tell us a little bit about where you find Big Think's videos, articles, and podcasts.
  • Be entered for a chance to win 1 of 3 Amazon gift cards each worth $100.
  • All survey information is anonymous and will be used only for this survey.
Keep reading Show less