Why the number 2.9013 will go down in the history of bad science
Simon Oxenham covers the best and the worst from the world of psychology and neuroscience. Formerly writing with the pseudonym "Neurobonkers", Simon has a history of debunking dodgy scientific research and tearing apart questionable science journalism in an irreverent style. Simon has written and blogged for publishers including: The Psychologist, Nature, Scientific American and The Guardian. His work has been praised in the New York Times and The Guardian and described in Pearson's Textbook of Psychology as "excoriating reviews of bad science/studies”.
Follow Simon on Twitter
Like Simon on Facebook
Follow Simon on Google+
Subscribe via Email
Subscribe via RSS
Contact Simon directly by Email
If you’ve been reading this blog for any amount of time you’ll probably be familiar with the name Sokal from the Sokal affair, the scandal in 1996 in which physicist Alan Sokal intentionally submitted a paper that was a steaming pile of horse manure in a leading journal of cultural studies, designed as a test to see if it would get published (spolier, it did).
At the time, Sokal came under harsh criticism from some quarters for creating a "tempest in a tea cup" and for making "tendentious misrepresentations", according to some he had "insufficiently grasped the philosophy (he) criticized". Well the tables have turned and now Sokal along with Nicholas Brown and Harris Friedman have published a paper that mercilessly destroys a paper on positive psychology that has been cited close to a thousand times on Google Scholar and in countless self help books, somehow with no one in the field realizing the paper was not worth the paper it was written on.
The original paper by Fredrickson & Losada makes the case that the “positivity ratio” - the relationship between an individual's positive and negative emotions reaches a critical tipping point when it reaches the number 2.9013.
But as Sokal et al make painfully clear, the paper is utterly brimming with “fundamental conceptual and mathematical errors" and "the total absence of any justiﬁcation for the applicability of the Lorenz equations to modeling the time evolution of human emotions”. With me until that last bit? Well the original paper stated that the “positivity ratio” followed the following, completely irrelevant complex mathematic pattern:
Sokal et al point out that “the idea that any aspect of human behavior or experience should be universally and reproducibly constant to ﬁve signiﬁcant digits would, if proven, constitute a unique moment in the history of the social sciences”. This before demonstrating the fatal mathematical and logical flaws at every level, before concluding “that Fredrickson and Losada’s (2005) claims concerning the alleged critical values of the positivity ratio are entirely unfounded… One can only marvel at the astonishing coincidence that human emotions should turn out to be governed by exactly the same set of equations that were derived in a celebrated article several decades ago as a deliberately simpliﬁed model of convection in ﬂuids, and whose solutions happen to have visually appealing properties”.
The final verdict is damning and enlightening:
“An alternative explanation—and, frankly, the one that appears most plausible to us—is that the entire process of “derivation” of the Lorenz equations has been contrived to demonstrate an imagined ﬁt between some rather limited empirical data and the scientiﬁcally impressive world of nonlinear dynamics.”
The crack team point out the mind bending absurdness of the conclusions Fredrickson and Losada abstract from their unbelievable (in the truest sense of the word) findings:
"They appear to assert that the predictive use of differential equations abstracted from a domain of the natural sciences to describe human interactions can be justiﬁed on the basis of the linguistic similarity between elements of the technical vocabulary of that scientiﬁc domain and the adjectives used metaphorically by a particular observer to describe those human interactions. If true, this would have remarkable implications for the social sciences. One could describe a team’s interactions as “sparky” and conﬁdently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the inﬂuence of an electric ﬁeld. Alternatively, the interactions of a team of researchers whose journal articles are characterized by “smoke and mirrors” could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics."
Sokal et al don't postulate with regard to whether the cause of the obviously unlikely finding is the result of "excessive enthusiasm, sincere self-deception, or other motivations".
In a response from Fredrickson (her coauthor has gone mysteriously AWOL), she concedes that “Losada’s mathematical work may have been ﬂawed” - and endeavors to scrub this part of the theory from the record, leaving the original psychological theory intact (which in fairness it may well be, this mumbo jumbo aside).
This all begs the question, how much more of the literature is steeped in fluffy feathered nonsense? “Filigrees of rhetorical precision atop unsteady pillars of conceptual bluff”. The answer is potentially a hefty chunk.
I stumbled on one particularly priceless gem myself only last week, disasters this big are thankfully pretty rare in science, but it just goes to show how important it is to always read critically yourself, even if a thousand other researchers in your field have given something the green light, it doesn’t mean they are right. The whole fiasco reminds me of the bogus TedX talk which received a standing ovation, months went by before it was discovered that the lecture on how “vortel based maths” would save the world was complete and utter unmitigated nonsense.
If anything, my guess would be that this is an example of groupthink, did the researchers that cited this paper really believe it, did they even read it? Or did they see that other esteemed psychologists had cited it and think… “X cited it… must be legit”. It’s something we’re all guilty of doing on some level at some point or another, if we insisted on checking every single “fact” we came across in life ourselves, we’d be both hopelessly ill equipt and have a hard time functioning in society. But when making a scientific argument, or analyzing the particularly earth shattering ones, it’s a practice clearly worth taking up.
Another thing worth pondering is whether this study was so widely cited simply because at first glance it appears more “sciencey” than traditional positive psychology research. In the aftermath of The Seductive Allure of the “Seductive Allure” of Neuroscience Explanations debate, this couldn’t have been more timely.
I’m going to stop here to avoid simply rehashing what has already been said more eloquently elsewhere. For further analysis, I’ll pass you over to the venerable Neuroskeptic over at Discover and Will Willkinson over at the Daily Beast.
Fredrickson, B. L., & Losada, M. F. (2005). Positive affect and the complex dynamics of human ﬂourishing. American Psychologist, 60, 678–686. doi:10.1037/0003-066X.60.7.678 (PDF)
Image Credit: Shutterstock/Wikimedia Commons
Update: This article was amended on 23/08/13 to include the line "At the time, Sokal came under harsh criticism from some quarters for creating a "tempest in a tea cup" and for making "tendentious misrepresentations", according to some he had "insufficiently grasped the philosophy (he) criticized" - so that the following remark makes sense in context.
New research links urban planning and political polarization.
- Canadian researchers find that excessive reliance on cars changes political views.
- Decades of car-centric urban planning normalized unsustainable lifestyles.
- People who prefer personal comfort elect politicians who represent such views.
Progressive America would be half as big, but twice as populated as its conservative twin.
- America's two political tribes have consolidated into 'red' and 'blue' nations, with seemingly irreconcilable differences.
- Perhaps the best way to stop the infighting is to go for a divorce and give the two nations a country each
- Based on the UN's partition plan for Israel/Palestine, this proposal provides territorial contiguity and sea access to both 'red' and 'blue' America
Science and the squishiness of the human mind. The joys of wearing whatever the hell you want, and so much more.
- Why can't we have a human-sized cat tree?
- What would happen if you got a spoonful of a neutron star?
- Why do we insist on dividing our wonderfully complex selves into boring little boxes
SMARTER FASTER trademarks owned by The Big Think, Inc. All rights reserved.