'The first crack in the wall of significance testing'

A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?

If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:


"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"

Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:

"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."

Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:

Follow Neurobonkers on TwitterFacebookGoogle+RSS, or join the mailing list. Image Credit: Adapted from original artwork by Shutterstock.

Reference:

Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991

Change.org
Politics & Current Affairs
  • The tongue-in-cheek petition, whose stated aim is to reduce the national debt, has been signed more than 8,600 times as of Tuesday.
  • Selling Montana, the fourth largest state in the country, would constitute the largest land deal since the Louisiana Purchase.
  • The national debt is often a source of concern for individuals, but the chances of the U.S. defaulting on its debts are relatively low — in part because the bulk of the national debt is owned by the American public.
Keep reading Show less

Study: Memories of music cannot be lost to Alzheimer's and dementia

The part of your brain responsible for ASMR catalogs music, and appears to be a stronghold against Alzheimer's and dementia.

The parts of the brain highlighted in red and yellow are thought to control your sense of attention and memory. (image c/o Brain Network Lab)
popular

Some music inspires you to move your feet, some inspires you to get out there and change the world. In any case, and to move hurriedly on to the point of this article, it's fair to say that music moves people in special ways. 

Keep reading Show less

What makes someone gay? Science is trying to get it straight.

Evolutionarily speaking, being gay is still something of an enigma

Videos
  • Heterosexual people have been less interesting to scientists than gay people, in terms of where they come from, because, evolutionarily speaking, being gay doesn't lead to a higher "higher reproductive fitness" — meaning, it doesn't lead to more babies.
  • Across cultures, gay boys tend to be more interested in spending time with their mothers.
  • We still don't really know why gay people are attracted to each other.