'The first crack in the wall of significance testing'

A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?

'The first crack in the wall of significance testing'

If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:


"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"

Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:

"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."

Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:

Follow Neurobonkers on TwitterFacebookGoogle+RSS, or join the mailing list. Image Credit: Adapted from original artwork by Shutterstock.

Reference:

Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991

The mystery of the Bermuda Triangle may finally be solved

Meteorologists propose a stunning new explanation for the mysterious events in the Bermuda Triangle.

Surprising Science

One of life's great mysteries, the Bermuda Triangle might have finally found an explanation. This strange region, that lies in the North Atlantic Ocean between Bermuda, Miami and San Juan, Puerto Rico, has been the presumed cause of dozens and dozens of mind-boggling disappearances of ships and planes.

Keep reading Show less

America of the 1930s saw thousands of people become Nazi

Nazi supporters held huge rallies and summer camps for kids throughout the United States in the 1930s.

League of the Friends of the New Germany rally at Madison Square Garden. 1934.

Credit: Bettman / Getty Images
Politics & Current Affairs
  • During the 1930s, thousands of Americans sympathized with the Nazis, holding huge rallies.
  • The rallies were organized by the American German Bund, which wanted to spread Nazi ideology.
  • Nazi supporters also organized summer camps for kids to teach them their values.
Keep reading Show less

Coffee and green tea may lower death risk for some adults

Tea and coffee have known health benefits, but now we know they can work together.


Credit: NIKOLAY OSMACHKO from Pexels
Surprising Science
  • A new study finds drinking large amounts of coffee and tea lowers the risk of death in some adults by nearly two thirds.
  • This is the first study to suggest the known benefits of these drinks are additive.
  • The findings are great, but only directly apply to certain people.
Keep reading Show less
Surprising Science

Why San Francisco felt like the set of a sci-fi flick

But most city dwellers weren't seeing the science — they were seeing something out of Blade Runner.

Scroll down to load more…
Quantcast