'The first crack in the wall of significance testing'
A major psychology journal has banned the use of the near-universally adopted practice of significance testing, citing recent evidence of the technique's unreliability. What will be the fallout for psychology as a field?
Simon Oxenham covers the best and the worst from the world of psychology and neuroscience. Formerly writing with the pseudonym "Neurobonkers", Simon has a history of debunking dodgy scientific research and tearing apart questionable science journalism in an irreverent style. Simon has written and blogged for publishers including: The Psychologist, Nature, Scientific American and The Guardian. His work has been praised in the New York Times and The Guardian and described in Pearson's Textbook of Psychology as "excoriating reviews of bad science/studies”.
Follow Simon on Twitter
Like Simon on Facebook
Follow Simon on Google+
Subscribe via Email
Subscribe via RSS
Contact Simon directly by Email
If you have ever read a piece of psychology research cover to cover, you will almost certainly have witnessed the p-value, the controversial statistical measure that we have discussed at length on this blog previously. Last month, a scientific journal made perhaps the boldest move since journals began opening their doors to open access. The journal, Basic and Applied Social Psychology (BASP), has banned the use of null hypothesis significance testing, a technique used almost universally in psychology research and much scientific research across the board. Some, however, might call this the nuclear option, as when used properly, the p-value can be a useful indicator. Instead of significance testing, the journal will rely on arguably more reliable measures that are often left out of modern psychology research:
"BASP will require strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem"
Significance testing is one of the most important, yet most widely misunderstood definitions in science. Over at the excellent Science-Based Medicine blog, Yale clinical neurologist Steven Novella sums up the problem well:
"The p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously."
Novella's account refers to an absolutely beautiful Guitar-Hero-meets-Space-Invaders-meets-Tetris-meets-roulette statistical demonstration of the problem by Geoff Cumming, dubbed "The dance of the p-values." If it is not the most inspired stats lesson you've ever had, then I'll eat my hat:
Trafimow D. (2014). Editorial, Basic and Applied Social Psychology, 37 (1) 1-2. DOI: http://dx.doi.org/10.1080/01973533.2015.1012991
NASA astronomer Michelle Thaller is coming back to Big Think to answer YOUR questions! Here's all you need to know to submit your science-related inquiries.
Big Think's amazing audience has responded so well to our videos from NASA astronomer and Assistant Director for Science Communication Michelle Thaller that we couldn't wait to bring her back for more!
And this time, she's ready to tackle any questions you're willing to throw at her, like, "How big is the Universe?", "Am I really made of stars?" or, "How long until Elon Musk starts a colony on Mars?"
All you have to do is submit your questions to the form below, and we'll use them for an upcoming Q+A session with Michelle. You know what to do, Big Thinkers!
Or how I learned to stop worrying and love my tsundoku.
- Many readers buy books with every intention of reading them only to let them linger on the shelf.
- Statistician Nassim Nicholas Taleb believes surrounding ourselves with unread books enriches our lives as they remind us of all we don't know.
- The Japanese call this practice tsundoku, and it may provide lasting benefits.
Calling all big thinkers!
SMARTER FASTER trademarks owned by The Big Think, Inc. All rights reserved.