'The first crack in the wall of significance testing'

A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?

If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:


"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"

Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:

"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."

Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:

Follow Neurobonkers on TwitterFacebookGoogle+RSS, or join the mailing list. Image Credit: Adapted from original artwork by Shutterstock.

Reference:

Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991

Opioids not much better than placebos at treating pain, study says

Alternative treatments are often better for noncancer pain, the study found.

Surprising Science
  • The study examined more than 26,000 people experiencing chronic pain.
  • Opioids were only marginally better than placebos at treating pain and improving physical functioning.
  • It's estimated that at least 2 million Americans have opioid use problems.
Keep reading Show less

Iceland is officially worshiping Norse Gods again

For the first time since the Vikings sailed, the Icelandic public will soon be able to worship classical Norse gods like Odin, Thor, and Frigg at a public temple built in their honor.

popular

For the first time since the Vikings sailed, the Icelandic public are worshiping classical Norse gods like Odin, Thor, and Frigg at a public temple built in their honor. "The worship of Odin, Thor, Freya and the other gods of the old Norse pantheon became an officially recognized religion exactly 973 years after Iceland’s official conversion to Christianity."

Keep reading Show less

Sound could replace lasers in surgery

Moving from HOT to HAT, a dazzling new acoustic technology.

(Proceedings of the National Academy of Sciences)
Technology & Innovation
  • Scientists announce the ability to simultaneously manipulate individual levitated objects.
  • Using high-frequency sound waves may provide a safer alternative to laser microsurgery.
  • Video of the research looks like a cartoon, but it's all real.
Keep reading Show less