Is It Tough Love Time For Science?

Is "science broken" or self-correcting? And who is going to do the grown-up thing and fix the game (instead of scoring points within it)?


1. Science needs some tough love (fields vary, but some enable and encourage unhealthy habits). And “good cop” approaches aren't fixing “phantom patterns” and “noise mining” (explained below).

2. Although everyone’s doing what seems “scientifically reasonable” the result is a “machine for producing and publicizing random patterns,” statistician Andrew Gelman says.

3. Gelman is too kind; the “reproducibility crisis” is really a producibility problem—professional practices reward production and publication of unsound studies.

4. Gelman calls such studies “dead on arrival,” but they’re actually dead on departure, doomed at conception by “flaws inherent in [their] original design” (+much that’s “poorly designed” gets published).

5. Optimists say relax, “science is self-correcting.” For instance, Christie Aschwanden says the “replication crisis is a sign that science is working,” it’s not “untrustworthy,” it’s just messy and hard (it’s “in the long run… dependable,” says Tom Siegfried).

6. “Science Is Broken” folks like Dan Engber ask, “how quickly does science self-correct? Are bad ideas and wrong results stamped out [quickly]... or do they last for generations?” And at what (avoidable) cost?

7. We mustn’t overgeneralize—physics isn’t implicated, instructively it’s intrinsically less variable, (all electrons behave consistently). Biology and social science aren’t so lucky: People ≠ biological billiard balls.

8. Richard Harris’s book Rigor Mortis argues “sloppy science… wastes billions” (~50% of US taxpayer-funded biomedical research budget, ~$15 billion squandered).

9. Harris blames ultra-competitive “publish first, correct later games, and heartbreakingly abysmal experimental design, that can threaten lives (Gelman concurs, “Clinical trials are broken.”).

10. Harris sees “no easy” fix. But a science-is-hard defense doesn’t excuse known-to-be-bad practices.

11. Engber’s “bad ideas and wrong results” are dwarfed by systemic generation-spanning method-level ills. For instance, Gelman calls traditional statistics “counterproductive”—badly misnamed “statistical significance” tests aren’t arbiters “of scientific truth," though they’re widely used that way.

12. Psychology brought “statistical significance” misuse to light recently (e.g.,the TED chart-topping “power pose”), but Deirdre McCloskey declared "statistical significance has ruined empirical… economics" in 1998, and traced concerns to 1920s. Gelman wants us to “abandon statistical significance.”

13. Yet “noise mining” abounds. Fields with inherent variability, small effects, and noisy measurements drown in datasets with phantom patterns, unrelated to stable causes (see Cornell’s “world-renowned eating... expert”)

14. No “statistical alchemy” (Keynes, 1939) can diagnose phantom patterns. Only further reality-checking can. “Correlation doesn’t even imply correlation” beyond your data. Always ask: Why would this pattern generalize? By what causal process(es)?

15. Basic retraining must emphasize representativeness and causal stability. Neither bigger samples, nor randomization necessarily ensure representativeness (see, mixed-type stats woes, pattern types).

16. Journalism that showcases every sensational-seeming study ill-serves us. Most unconfirmed science should go unreported—media exaggerations damage public trust.

17. Beyond avoidable deaths, and burned dollars, there’s a substantial “social cost of junk science” (e.g., enabling the science is bogus” deniers).

18. Great science is occurring, but the “free play of free intellects” game, fun though it is, is far from free of unforced errors.

19. “Saving science” (Daniel Sarewitz) means fixing the game—not scoring points within it.

 

Illustration by Julia SuitsThe New Yorker cartoonist & author of The Extraordinary Catalog of Peculiar Inventions

Big Think Edge
  • The meaning of the word 'confidence' seems obvious. But it's not the same as self-esteem.
  • Confidence isn't just a feeling on your inside. It comes from taking action in the world.
  • Join Big Think Edge today and learn how to achieve more confidence when and where it really matters.

To boost your self-esteem, write about chapters of your life

If you're lacking confidence and feel like you could benefit from an ego boost, try writing your life story.

Personal Growth

In truth, so much of what happens to us in life is random – we are pawns at the mercy of Lady Luck. To take ownership of our experiences and exert a feeling of control over our future, we tell stories about ourselves that weave meaning and continuity into our personal identity.

Keep reading Show less

Yale scientists restore brain function to 32 clinically dead pigs

Researchers hope the technology will further our understanding of the brain, but lawmakers may not be ready for the ethical challenges.

Still from John Stephenson's 1999 rendition of Animal Farm.
Surprising Science
  • Researchers at the Yale School of Medicine successfully restored some functions to pig brains that had been dead for hours.
  • They hope the technology will advance our understanding of the brain, potentially developing new treatments for debilitating diseases and disorders.
  • The research raises many ethical questions and puts to the test our current understanding of death.

The image of an undead brain coming back to live again is the stuff of science fiction. Not just any science fiction, specifically B-grade sci fi. What instantly springs to mind is the black-and-white horrors of films like Fiend Without a Face. Bad acting. Plastic monstrosities. Visible strings. And a spinal cord that, for some reason, is also a tentacle?

But like any good science fiction, it's only a matter of time before some manner of it seeps into our reality. This week's Nature published the findings of researchers who managed to restore function to pigs' brains that were clinically dead. At least, what we once thought of as dead.

What's dead may never die, it seems

The researchers did not hail from House Greyjoy — "What is dead may never die" — but came largely from the Yale School of Medicine. They connected 32 pig brains to a system called BrainEx. BrainEx is an artificial perfusion system — that is, a system that takes over the functions normally regulated by the organ. The pigs had been killed four hours earlier at a U.S. Department of Agriculture slaughterhouse; their brains completely removed from the skulls.

BrainEx pumped an experiment solution into the brain that essentially mimic blood flow. It brought oxygen and nutrients to the tissues, giving brain cells the resources to begin many normal functions. The cells began consuming and metabolizing sugars. The brains' immune systems kicked in. Neuron samples could carry an electrical signal. Some brain cells even responded to drugs.

The researchers have managed to keep some brains alive for up to 36 hours, and currently do not know if BrainEx can have sustained the brains longer. "It is conceivable we are just preventing the inevitable, and the brain won't be able to recover," said Nenad Sestan, Yale neuroscientist and the lead researcher.

As a control, other brains received either a fake solution or no solution at all. None revived brain activity and deteriorated as normal.

The researchers hope the technology can enhance our ability to study the brain and its cellular functions. One of the main avenues of such studies would be brain disorders and diseases. This could point the way to developing new of treatments for the likes of brain injuries, Alzheimer's, Huntington's, and neurodegenerative conditions.

"This is an extraordinary and very promising breakthrough for neuroscience. It immediately offers a much better model for studying the human brain, which is extraordinarily important, given the vast amount of human suffering from diseases of the mind [and] brain," Nita Farahany, the bioethicists at the Duke University School of Law who wrote the study's commentary, told National Geographic.

An ethical gray matter

Before anyone gets an Island of Dr. Moreau vibe, it's worth noting that the brains did not approach neural activity anywhere near consciousness.

The BrainEx solution contained chemicals that prevented neurons from firing. To be extra cautious, the researchers also monitored the brains for any such activity and were prepared to administer an anesthetic should they have seen signs of consciousness.

Even so, the research signals a massive debate to come regarding medical ethics and our definition of death.

Most countries define death, clinically speaking, as the irreversible loss of brain or circulatory function. This definition was already at odds with some folk- and value-centric understandings, but where do we go if it becomes possible to reverse clinical death with artificial perfusion?

"This is wild," Jonathan Moreno, a bioethicist at the University of Pennsylvania, told the New York Times. "If ever there was an issue that merited big public deliberation on the ethics of science and medicine, this is one."

One possible consequence involves organ donations. Some European countries require emergency responders to use a process that preserves organs when they cannot resuscitate a person. They continue to pump blood throughout the body, but use a "thoracic aortic occlusion balloon" to prevent that blood from reaching the brain.

The system is already controversial because it raises concerns about what caused the patient's death. But what happens when brain death becomes readily reversible? Stuart Younger, a bioethicist at Case Western Reserve University, told Nature that if BrainEx were to become widely available, it could shrink the pool of eligible donors.

"There's a potential conflict here between the interests of potential donors — who might not even be donors — and people who are waiting for organs," he said.

It will be a while before such experiments go anywhere near human subjects. A more immediate ethical question relates to how such experiments harm animal subjects.

Ethical review boards evaluate research protocols and can reject any that causes undue pain, suffering, or distress. Since dead animals feel no pain, suffer no trauma, they are typically approved as subjects. But how do such boards make a judgement regarding the suffering of a "cellularly active" brain? The distress of a partially alive brain?

The dilemma is unprecedented.

Setting new boundaries

Another science fiction story that comes to mind when discussing this story is, of course, Frankenstein. As Farahany told National Geographic: "It is definitely has [sic] a good science-fiction element to it, and it is restoring cellular function where we previously thought impossible. But to have Frankenstein, you need some degree of consciousness, some 'there' there. [The researchers] did not recover any form of consciousness in this study, and it is still unclear if we ever could. But we are one step closer to that possibility."

She's right. The researchers undertook their research for the betterment of humanity, and we may one day reap some unimaginable medical benefits from it. The ethical questions, however, remain as unsettling as the stories they remind us of.

Videos
  • Prejudice is typically perpetrated against 'the other', i.e. a group outside our own.
  • But ageism is prejudice against ourselves — at least, the people we will (hopefully!) become.
  • Different generations needs to cooperate now more than ever to solve global problems.