When The New Yorker Probes the "Decline Effect," An Opportunity Emerges to Rethink Science Education

A weekly newsletter featuring the biggest ideas from the smartest people

At the New Yorker last week, science journalist Jonah Lehrer penned a conversation-starting feature on the so-called “decline effect,” the tendency across scientific fields for a new and exciting finding to eventually lose its dramatic qualities and excitement. Why? Because subsequent replications and studies poke holes in the certainty of the original observations and conclusions.

Lehrer’s article is a notable piece of science journalism, in the tradition of explaining complex realities about the social nature of doing science and how scientific findings are communicated and perceived by the public. Lehrer’s focus and the reaction it has generated offers yet another useful example of the opportunity and need to embed media literacy curricula within science education at the highschool and college levels. It’s a topic I will be exploring over the next several months, and I am hoping to draw upon the expertise, insights, and suggestions of readers who leave their comments.

Since the article is behind a paywall, perhaps the best explainer of Lehrer’s decline effect comes from Scientific American’s John Horgan, author of The End of Science:

Lehrer reports on the “decline effect,” the tendency of scientific claims to receive decreasing support over time. The term was coined by the parapsychologist J. B. Rhine in the 1930s to describe the apparent drop-off in extrasensory perception (ESP) of psychic subjects tested by Rhine for extended periods.

The likely explanation, of course, is that Rhine’s subjects were never psychic; the initial finding of ESP was illusory, vanishing as Rhine’s methods became more rigorous and his data more statistically significant. The decline effect is really a “decline of illusion,” Lehrer explains, which “reminds us how difficult it is to prove anything.” The decline effect has turned up in a host of fields, including not only squishy sciences like psychology, psychiatry, psychopharmacology and medical genetics, but even physics; measurements of the charge of electrons and a constant governing the decay of neutrons have exhibited the decline effect, according to Lehrer.

Lehrer attributes the effect to several common factors. First, a researcher stumbles on a dramatic correlation—say, between a new pharmaceutical and amelioration of schizophrenia—that actually stems not from causation but just from coincidence. The more dramatic the researcher’s claim, the more likely he may be to get published and to obtain funding for more research. Other researchers jump on the bandwagon, doing follow-up studies that—because of the unconscious bias in favor of the initial claim—often corroborate it. Only gradually does counterevidence emerge, showing that the initial correlation stemmed not from causation but coincidence.

Consequently, science yields not truth but what comedy talk-show host Stephen Colbert calls “truthiness”. None of this should surprise veteran science watchers—or anyone who’s taken a course in the history or philosophy of science. I’ve whacked fields such as clinical psychology and behavioral genetics for churning out claims—A new and improved treatment for depression! A gene for liberalism!—that don’t stand up to scrutiny. But Lehrer does a good job pulling together multiple strands into a unifying narrative of doubt. He cites the remarkable recent work of the epidemiologist John Ioaniddis, who has presented evidence that “most published research findings are false.”

Lehrer also presents examples of the decline effect that were new to me. One is the link between physiological symmetry and sexual attraction in humans and other animals, which evolutionary psychologists have been touting for almost 20 years. Finally, the neo-Darwinian theory of human nature produced a not-completely obvious result! Except it didn’t. A flurry of positive findings in the 1990s—Women have more orgasms when they couple with symmetrical guys!—gradually gave way to negative reports.

Anyone who works in the sciences or social sciences, analyzes quantitative data, tests and develops theories, puts them forward for peer-review, and tracks how conclusions and findings evolve in the literature, knows that there is a strong ring of truth to Lehrer’s narrative of doubt.

Yet the reaction that the article has stirred in some cases does not match the nuance of Lehrer’s arguments. The article has been unfairly critiqued by some for giving ammunition to those already committed to extreme doubt about subjects such as climate change or evolution. As Lehrer notes at his blog, he’s also been accused of being a post-modernist, arguing that there is no such thing as truth or reality.

On the first count that the article gives fodder to skeptics of science, I think this is another example of perceiving public opinion in terms of the extremes and to similarly evaluate the media as hostile, while overlooking the vast middle of public opinion and the favorable qualities of coverage. (This occurs commonly on heated issues such as climate change. See a column I did this year for Slate.)

Instead of fearing Lehrer’s article and other science journalists who probe the social and uncertain world of science, as I explain below, scientists and educators should be embracing the article and asking for more. As for Lehrer being a “post-modernist,” this is another example of some science advocates having a false, positivist outlook and using a catch-all term to describe an argument they don’t like. (See the Science Wars.)

Journalism That Opens the Black Box of Science

If in the news media and in education we present science as a black box, only allowing the public or students to witness a staged presentation of overly-certain simplified claims about findings and benefits, then when true controversies or incidents of fraud occur, we only make the public vulnerable to dismissing the entire institution of peer-review and science.

Instead if we use examples such as Lehrer’s article to engage in a conversation about the realities of science, students and the public will be much more likely to correctly attribute a controversy or incidence of fraud to individual failings and/or the type of more innocent social factors that Lehrer discusses in his article. Moreover, the public will be far less vulnerable to the cycles of hype that are now endemic to commercially-driven claims about science and technology. (See article last year at Nature Biotechnology and post yesterday.)

Consider the learning opportunity as describe by my fellow Big Think blogger David Berreby in a post titled “It’s Not Anti-Science To Think In Terms of Evidence“:

Lehrer’s piece described an anxious sense, noticeable in many different branches of science, that today the evidence section of a lot of research papers isn’t as reliable as assumed. The reasons could be psychological, mathematical (statistics packages are powerful and sophisticated, and the sheer amount of data now is huge), and cultural. The question is open. Gracefully, he avoids claiming to know more than he does.

None of this is a claim that about any theories, one way or the other. It’s not about whether I’m right when I describe what I think you see in my telescope. It’s about whether the telescope is working as well as it should.

So, here is what the piece is not: It’s not a claim that there’s no such thing as evidence. It’s not a claim that we should all believe whatever the hell we want. It doesn’t privilege ignorance over science. It does what we science journalists should be doing for our readers: Tell them how research is actually done, so that they can better judge for themselves what “science says.” It is a lucid explanation of a real phenomenon, whose point, I think, is that scientists have taken notice because they want science to do better.

Embedding Media Literacy In Science Education

In reflecting on Lehrer’s article, SciAm’s John Horgan does note a flaw in his narrative of doubt, suggesting perhaps correctly that Lehrer’s concluding framing of this complex side of science might lead readers to extreme attributions rather than informed skepticism.

And here an added layer is offered relative to public education. As journalist Lehrer probes the social side of science, Horgan in reaction links public attention back to the social side of journalism. From Horgan’s post:

My main complaint is that Lehrer makes science as a whole sound much “truthier” than it really is. His article was first pointed out to me by my friend Valerie, who believes in homeopathy and tarot cards. The article confirmed her suspicions that mainstream science and medicine may not be based on evidence any more solid than her supposedly (and IMHO, actually—sorry, Valerie) pseudoscientific beliefs. Lehrer’s broad-brush critique will no doubt also cheer global-warming deniers, creationists, postmodernists and other pesky challengers of scientific orthodoxy.

Lehrer himself seems to have realized that he went too far. On his blog The Frontal Cortex, he dismisses the notion that “The Truth Wears Off” implicitly undermines the status of the theory of evolution by natural selection and global warming, which are “two of the most robust and widely tested theories of modern science.” He also denies that he is “some sort of Derridean postmodernist, trying to turn publication bias into an excuse to not believe in anything.”

But here is how Lehrer ends his article: “Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.” This assertion is absurd. We may choose to believe in psychoanalysis rather than behaviorism, because both are equally flimsy. But the evidence is rock-solid for quantum mechanics, general relativity, the germ theory of infectious disease, the genetic code and many other building blocks of scientific knowledge, which have yielded applications that have transformed our world. There’s nothing truthy about a hydrogen bomb.

If Lehrer didn’t really mean that belief in a given scientific claim is always a matter of choice, why did he say it? He apparently decided, like many scientists, that truthiness would make a bigger splash than truth.

Are you starting to see where I am going? Lehrer’s article and the exchanges it has catalyzed are wonderful teaching tools for science students. No doubt, a few teachers and professors across the country have forwarded the articles and discussed them in class.

Yet these opportunities are too rich to only have a few classrooms engage with. Funding, formative research, development, evaluation, and resources are needed to widely diffuse a media literacy curriculum.

Science media literacy and other strategies for increasing civic science education are topics I will be focusing on over the next several months, as I prepare to give a lecture on the subject at a meeting in March convened by the American Association of Colleges and Universities on STEM education.

What do readers think, especially those who might be educators? What are the opportunities for articles like Lehrer’s, Horgan’s reply, and other responses to be woven systematically into science coursework and courses across other disciplines?

Reconsidering Climate Change Literacy and Education

Nisbet, M.C. & Scheufele, D.A. (2009). What’s Next for Science Communication? Promising Directions and Lingering Distractions. American Journal of Botany, 96 (10), 1767-1778. (PDF).

A weekly newsletter featuring the biggest ideas from the smartest people