Why the number 2.9013 will go down in the history of bad science

Why the number 2.9013 will go down in the history of bad science

If you’ve been reading this blog for any amount of time you’ll probably be familiar with the name Sokal from the Sokal affair, the scandal in 1996 in which physicist Alan Sokal intentionally submitted a paper that was a steaming pile of horse manure in a leading journal of cultural studies, designed as a test to see if it would get published (spolier, it did).


At the time, Sokal came under harsh criticism from some quarters for creating a "tempest in a tea cup" and for making "tendentious misrepresentations", according to some he had "insufficiently grasped the philosophy (he) criticized".  Well the tables have turned and now Sokal along with Nicholas Brown and Harris Friedman have published a paper that mercilessly destroys a paper on positive psychology that has been cited close to a thousand times on Google Scholar and in countless self help books, somehow with no one in the field realizing the paper was not worth the paper it was written on.

The original paper by Fredrickson & Losada makes the case that the “positivity ratio” - the relationship between an individual's positive and negative emotions reaches a critical tipping point when it reaches the number 2.9013.

But as Sokal et al make painfully clear, the paper is utterly brimming with “fundamental conceptual and mathematical errors" and "the total absence of any justification for the applicability of the Lorenz equations to modeling the time evolution of human emotions”. With me until that last bit? Well the original paper stated that the “positivity ratio” followed the following, completely irrelevant complex mathematic pattern:

Sokal et al point out that “the idea that any aspect of human behavior or experience should be universally and reproducibly constant to five significant digits would, if proven, constitute a unique moment in the history of the social sciences”. This before demonstrating the fatal mathematical and logical flaws at every level, before concluding “that Fredrickson and Losada’s (2005) claims concerning the alleged critical values of the positivity ratio are entirely unfounded… One can only marvel at the astonishing coincidence that human emotions should turn out to be governed by exactly the same set of equations that were derived in a celebrated article several decades ago as a deliberately simplified model of convection in fluids, and whose solutions happen to have visually appealing properties”.

The final verdict is damning and enlightening:

“An alternative explanation—and, frankly, the one that appears most plausible to us—is that the entire process of “derivation” of the Lorenz equations has been contrived to demonstrate an imagined fit between some rather limited empirical data and the scientifically impressive world of nonlinear dynamics.”

The crack team point out the mind bending absurdness of the conclusions Fredrickson and Losada abstract from their unbelievable (in the truest sense of the word) findings:

"They appear to assert that the predictive use of differential equations abstracted from a domain of the natural sciences to describe human interactions can be justified on the basis of the linguistic similarity between elements of the technical vocabulary of that scientific domain and the adjectives used metaphorically by a particular observer to describe those human interactions. If true, this would have remarkable implications for the social sciences. One could describe a team’s interactions as “sparky” and confidently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the influence of an electric field. Alternatively, the interactions of a team of researchers whose journal articles are characterized by “smoke and mirrors” could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics."

Sokal et al don't postulate with regard to whether the cause of the obviously unlikely finding is the result of "excessive enthusiasm, sincere self-deception, or other motivations".

In a response from Fredrickson (her coauthor has gone mysteriously AWOL), she concedes that “Losada’s mathematical work may have been flawed” - and endeavors to scrub this part of the theory from the record, leaving the original psychological theory intact (which in fairness it may well be, this mumbo jumbo aside).

This all begs the question, how much more of the literature is steeped in fluffy feathered nonsense? “Filigrees of rhetorical precision atop unsteady pillars of conceptual bluff”. The answer is potentially a hefty chunk.

I stumbled on one particularly priceless gem myself only last week, disasters this big are thankfully pretty rare in science, but it just goes to show how important it is to always read critically yourself, even if a thousand other researchers in your field have given something the green light, it doesn’t mean they are right. The whole fiasco reminds me of the bogus TedX talk which received a standing ovation, months went by before it was discovered that the lecture on how “vortel based maths” would save the world was complete and utter unmitigated nonsense.

If anything, my guess would be that this is an example of groupthink, did the researchers that cited this paper really believe it, did they even read it? Or did they see that other esteemed psychologists had cited it and think… “X cited it… must be legit”. It’s something we’re all guilty of doing on some level at some point or another, if we insisted on checking every single “fact” we came across in life ourselves, we’d be both hopelessly ill equipt and have a hard time functioning in society. But when making a scientific argument, or analyzing the particularly earth shattering ones, it’s a practice clearly worth taking up.

Another thing worth pondering is whether this study was so widely cited simply because at first glance it appears more “sciencey” than traditional positive psychology research. In the aftermath of The Seductive Allure of the “Seductive Allure” of Neuroscience Explanations debate, this couldn’t have been more timely.

I’m going to stop here to avoid simply rehashing what has already been said more eloquently elsewhere. For further analysis, I’ll pass you over to the venerable Neuroskeptic over at Discover and Will Willkinson over at the Daily Beast.

References

Brown N.J.L., Sokal A.D. & Friedman H.L. (2013). The Complex Dynamics of Wishful Thinking: The Critical Positivity Ratio., The American psychologist, PMID:  (PDF)

PDF)

Fredrickson B.L. (2013). Updated Thinking on Positivity Ratios., American Psychologist, DOI:  (PDF)

To keep up to date with this blog you can follow Neurobonkers on TwitterFacebookRSS or join the mailing list

 

Image Credit: Shutterstock/Wikimedia Commons

Update: This article was amended on 23/08/13 to include the line "At the time, Sokal came under harsh criticism from some quarters for creating a "tempest in a tea cup" and for making "tendentious misrepresentations", according to some he had "insufficiently grasped the philosophy (he) criticized" - so that the following remark makes sense in context.

Is the universe a graveyard? This theory suggests humanity may be alone.

Ever since we've had the technology, we've looked to the stars in search of alien life. It's assumed that we're looking because we want to find other life in the universe, but what if we're looking to make sure there isn't any?

According to the Great Filter theory, Earth might be one of the only planets with intelligent life. And that's a good thing (NASA, ESA, and the Hubble Heritage Team [STScI/AURA]).
Surprising Science

Here's an equation, and a rather distressing one at that: N = R* × fP × ne × f1 × fi × fc × L. It's the Drake equation, and it describes the number of alien civilizations in our galaxy with whom we might be able to communicate. Its terms correspond to values such as the fraction of stars with planets, the fraction of planets on which life could emerge, the fraction of planets that can support intelligent life, and so on. Using conservative estimates, the minimum result of this equation is 20. There ought to be 20 intelligent alien civilizations in the Milky Way that we can contact and who can contact us. But there aren't any.

Keep reading Show less

Beyond the two cultures: rethinking science and the humanities

Cross-disciplinary cooperation is needed to save civilization.

Credit: Public domain
13-8
  • There is a great disconnect between the sciences and the humanities.
  • Solutions to most of our real-world problems need both ways of knowing.
  • Moving beyond the two-culture divide is an essential step to ensure our project of civilization.
Keep reading Show less

Stephen Hawking's black hole theory proved right

New study analyzes gravitational waves to confirm the late Stephen Hawking's black hole area theorem.

Model of spiraling black holes that are merging with each other.

Credit: NASA's Goddard Space Flight Center
Surprising Science
  • A new paper confirms Stephen Hawking's black hole area theorem.
  • The researchers used gravitational wave data to prove the theorem.
  • The data came from Caltech and MIT's Advanced Laser Interferometer Gravitational-Wave Observatory.
Keep reading Show less
Quantcast