The most important boring idea in the universe

We live in a world dominated by science, but most people don't understand its most essential characteristic: establishing standards of evidence to keep us from getting fooled by our own biases and opinions.

Credit: David Matos via Unsplash
  • Maintaining standards of evidence is the most important and least appreciated idea in science.
  • Modern science was established in the late Renaissance when networks of researchers began working out best practices for linking evidence with conclusions.
  • In the face of science denial and attempts to create a post-truth society, we have to protect the primacy of standards of evidence in science and society.

I talk a lot about science to people who are not scientists. It's generally a lot of fun because most folks are science-curious even if they don't think about it a lot on their own time. But whether I'm talking about alien life, black holes, or the weirdnesses of quantum mechanics, there is always one really important idea that I try to get across that generally no one is interested in:

Standards of evidence. It's the most important boring idea in the universe.

Networks of scientists led to scientific societies

The development of modern science was a long, slow process that required input from most of the world's cultures ranging from ancient Greece and medieval Islam to India and China and eventually Renaissance Europe.

One of the most critical elements in Europe was the gradual build-up of international communities of scholars. While we usually think of science as being driven forward through the inspiration of one singular genius after another, that's only part of the story. For every Galileo and Newton there were hundreds of people you never heard of. They formed a network of thinkers and tinkerers writing letters to each other and making visits across the continent. In this way, they exchanged notes on things like the best way to carry out an experiment on boiling liquids or a new way to consider the mathematics of problems in celestial mechanics.

Unless you are a scientist, you probably have very little idea of how science knows what it knows, or even more important, how it knows what it doesn't know.

While they might not have known it at the time, what these scholars were also doing was setting up the foundations for an international order of scientific knowledge that would rest upon mutually agreed standards of evidence.

Eventually these networks became formalized. Scientific academies started popping up in places like Italy where the Academy of the Mysteries of Nature was founded in Naples in 1560. Later the Royal Society in England, formally known as the The Royal Society of London for Improving Natural Knowledge, was established in 1660. The French Academy of Sciences was formed just six years later. Over the years, these institutions and others would lead the way in establishing "best practices" for how to carry out scientific research and how to make sure that the conclusions a scientist drew from that research were supported by the evidence.

Scientific societies led to standards of evidence

Credit: Karlis Reimanis via Unsplash

I'm telling you this not because I think the history is so cool (though it is). Instead, what matters is seeing how the idea of standards of evidence was born in its scientific form. It came from people arguing in public over what should count as public facts or better yet public knowledge. Science didn't drop out of the sky fully formed. It was, and is, the fruit of a very human, very collective effort. The goal of that effort was to determine the best way to ask nature questions and ensure that you're getting correct answers.

This was not, by the way, a smooth process. There were lots of wrong turns in figuring out what counted as meaningful evidence and what was just another way of getting fooled. But over time, people figured out that there were standards for how to set up an experiment, how to collect data from it, and how to interpret that data. These standards now include things like isolating the experimental apparatus from spurious environmental effects, understanding how data collection devices respond to inputs, and accounting for systematic errors in analyzing the data. There are, of course, many more.

In this way, scientists figured out which standards were useful in linking evidence to conclusions.

Why standards matter

Science is now the most powerful force shaping human life. Without it, there could never be seven billion of us living on the planet at the same time. It has shaped and reshaped how we eat, how we travel, how we deal with sickness, how we communicate, and how we go to war. It is also how we are pushing Earth into new and dangerous (for us) climate states. But despite all this ubiquity and power, unless you are a scientist, you probably have very little idea of how science knows what it knows, or even more important, how it knows what it doesn't know.

Most of us don't understand what it means to have standards of evidence or how these standards get applied. That means that we can't see how the same methods that gave us our cell phones also gave us our understanding of climate change. When a pandemic hits, we can't see how the science is going to be an evolving process as those standards of evidence get used to sort through the firehose of real-time data. And when it comes to things like UFOs or "Ancient Aliens," we won't see that holding fast to those standards is the only thing that can keep us from being fooled by a conclusion that we may want to be true as opposed to accepting the one that actually is true.

    Admittedly, standards of evidence is not the most thrilling topic in the world. But it very well may be the most important.

    More From Adam Frank
    Related Articles

    COVID and "gain of function" research: should we create monsters to prevent them?

    Gain-of-function mutation research may help predict the next pandemic — or, critics argue, cause one.

    Credit: Guillermo Legaria via Getty Images
    Coronavirus

    This article was originally published on our sister site, Freethink.

    "I was intrigued," says Ron Fouchier, in his rich, Dutch-accented English, "in how little things could kill large animals and humans."

    It's late evening in Rotterdam as darkness slowly drapes our Skype conversation.

    This fascination led the silver-haired virologist to venture into controversial gain-of-function mutation research — work by scientists that adds abilities to pathogens, including experiments that focus on SARS and MERS, the coronavirus cousins of the COVID-19 agent.

    If we are to avoid another influenza pandemic, we will need to understand the kinds of flu viruses that could cause it. Gain-of-function mutation research can help us with that, says Fouchier, by telling us what kind of mutations might allow a virus to jump across species or evolve into more virulent strains. It could help us prepare and, in doing so, save lives.

    Many of his scientific peers, however, disagree; they say his experiments are not worth the risks they pose to society.

    A virus and a firestorm

    The Dutch virologist, based at Erasmus Medical Center in Rotterdam, caused a firestorm of controversy about a decade ago, when he and Yoshihiro Kawaoka at the University of Wisconsin-Madison announced that they had successfully mutated H5N1, a strain of bird flu, to pass through the air between ferrets, in two separate experiments. Ferrets are considered the best flu models because their respiratory systems react to the flu much like humans.

    The mutations that gave the virus its ability to be airborne transmissible are gain-of-function (GOF) mutations. GOF research is when scientists purposefully cause mutations that give viruses new abilities in an attempt to better understand the pathogen. In Fouchier's experiments, they wanted to see if it could be made airborne transmissible so that they could catch potentially dangerous strains early and develop new treatments and vaccines ahead of time.

    The problem is: their mutated H5N1 could also cause a pandemic if it ever left the lab. In Science magazine, Fouchier himself called it "probably one of the most dangerous viruses you can make."

    Just three special traits

    Recreated 1918 influenza virionsCredit: Cynthia Goldsmith / CDC / Dr. Terrence Tumpey / Public domain via Wikipedia

    For H5N1, Fouchier identified five mutations that could cause three special traits needed to trigger an avian flu to become airborne in mammals. Those traits are (1) the ability to attach to cells of the throat and nose, (2) the ability to survive the colder temperatures found in those places, and (3) the ability to survive in adverse environments.

    A minimum of three mutations may be all that's needed for a virus in the wild to make the leap through the air in mammals. If it does, it could spread. Fast.

    Fouchier calculates the odds of this happening to be fairly low, for any given virus. Each mutation has the potential to cripple the virus on its own. They need to be perfectly aligned for the flu to jump. But these mutations can — and do — happen.

    "In 2013, a new virus popped up in China," says Fouchier. "H7N9."

    H7N9 is another kind of avian flu, like H5N1. The CDC considers it the most likely flu strain to cause a pandemic. In the human outbreaks that occurred between 2013 and 2015, it killed a staggering 39% of known cases; if H7N9 were to have all five of the gain-of-function mutations Fouchier had identified in his work with H5N1, it could make COVID-19 look like a kitten in comparison.

    H7N9 had three of those mutations in 2013.

    Gain-of-function mutation: creating our fears to (possibly) prevent them

    Flu viruses are basically eight pieces of RNA wrapped up in a ball. To create the gain-of-function mutations, the research used a DNA template for each piece, called a plasmid. Making a single mutation in the plasmid is easy, Fouchier says, and it's commonly done in genetics labs.

    If you insert all eight plasmids into a mammalian cell, they hijack the cell's machinery to create flu virus RNA.

    "Now you can start to assemble a new virus particle in that cell," Fouchier says.

    One infected cell is enough to grow many new virus particles — from one to a thousand to a million; viruses are replication machines. And because they mutate so readily during their replication, the new viruses have to be checked to make sure it only has the mutations the lab caused.

    The virus then goes into the ferrets, passing through them to generate new viruses until, on the 10th generation, it infected ferrets through the air. By analyzing the virus's genes in each generation, they can figure out what exact five mutations lead to H5N1 bird flu being airborne between ferrets.

    And, potentially, people.

    "This work should never have been done"

    The potential for the modified H5N1 strain to cause a human pandemic if it ever slipped out of containment has sparked sharp criticism and no shortage of controversy. Rutgers molecular biologist Richard Ebright summed up the far end of the opposition when he told Science that the research "should never have been done."

    "When I first heard about the experiments that make highly pathogenic avian influenza transmissible," says Philip Dormitzer, vice president and chief scientific officer of viral vaccines at Pfizer, "I was interested in the science but concerned about the risks of both the viruses themselves and of the consequences of the reaction to the experiments."

    In 2014, in response to researchers' fears and some lab incidents, the federal government imposed a moratorium on all GOF research, freezing the work.

    Some scientists believe gain-of-function mutation experiments could be extremely valuable in understanding the potential risks we face from wild influenza strains, but only if they are done right. Dormitzer says that a careful and thoughtful examination of the issue could lead to processes that make gain-of-function mutation research with viruses safer.

    But in the meantime, the moratorium stifled some research into influenzas — and coronaviruses.

    The National Academy of Science whipped up some new guidelines, and in December of 2017, the call went out: GOF studies could apply to be funded again. A panel formed by Health and Human Services (HHS) would review applications and make the decision of which studies to fund.

    As of right now, only Kawaoka and Fouchier's studies have been approved, getting the green light last winter. They are resuming where they left off.

    Pandora's locks: how to contain gain-of-function flu

    Here's the thing: the work is indeed potentially dangerous. But there are layers upon layers of safety measures at both Fouchier's and Kawaoka's labs.

    "You really need to think about it like an onion," says Rebecca Moritz of the University of Wisconsin-Madison. Moritz is the select agent responsible for Kawaoka's lab. Her job is to ensure that all safety standards are met and that protocols are created and drilled; basically, she's there to prevent viruses from escaping. And this virus has some extra-special considerations.

    The specific H5N1 strain Kawaoka's lab uses is on a list called the Federal Select Agent Program. Pathogens on this list need to meet special safety considerations. The GOF experiments have even more stringent guidelines because the research is deemed "dual-use research of concern."

    There was debate over whether Fouchier and Kawaoka's work should even be published.

    "Dual-use research of concern is legitimate research that could potentially be used for nefarious purposes," Moritz says. At one time, there was debate over whether Fouchier and Kawaoka's work should even be published.

    While the insights they found would help scientists, they could also be used to create bioweapons. The papers had to pass through a review by the U.S. National Science Board for Biosecurity, but they were eventually published.

    Intentional biowarfare and terrorism aside, the gain-of-function mutation flu must be contained even from accidents. At Wisconsin, that begins with the building itself. The labs are specially designed to be able to contain pathogens (BSL-3 agricultural, for you Inside Baseball types).

    They are essentially an airtight cement bunker, negatively pressurized so that air will only flow into the lab in case of any breach — keeping the viruses pushed in. And all air in and out of the lap passes through multiple HEPA filters.

    Inside the lab, researchers wear special protective equipment, including respirators. Anyone coming or going into the lab must go through an intricate dance involving stripping and putting on various articles of clothing and passing through showers and decontamination.

    And the most dangerous parts of the experiment are performed inside primary containment. For example, a biocontainment cabinet, which acts like an extra high-security box, inside the already highly-secure lab (kind of like the radiation glove box Homer Simpson is working in during the opening credits).

    "Many people behind the institution are working to make sure this research can be done safely and securely." — REBECCA MORITZ

    The Federal Select Agent program can come and inspect you at any time with no warning, Moritz says. At the bare minimum, the whole thing gets shaken down every three years.

    There are numerous potential dangers — a vial of virus gets dropped; a needle prick; a ferret bite — but Moritz is confident that the safety measures and guidelines will prevent any catastrophe.

    "The institution and many people behind the institution are working to make sure this research can be done safely and securely," Moritz says.

    No human harm has come of the work yet, but the potential for it is real.

    "Nature will continue to do this"

    They were dead on the beaches.

    In the spring of 2014, another type of bird flu, H10N7, swept through the harbor seal population of northern Europe. Starting in Sweden, the virus moved south and west, across Denmark, Germany, and the Netherlands. It is estimated that 10% of the entire seal population was killed.

    The virus's evolution could be tracked through time and space, Fouchier says, as it progressed down the coast. Natural selection pushed through gain-of-function mutations in the seals, similarly to how H5N1 evolved to better jump between ferrets in his lab — his lab which, at the time, was shuttered.

    "We did our work in the lab," Fouchier says, with a high level of safety and security. "But the same thing was happening on the beach here in the Netherlands. And so you can tell me to stop doing this research, but nature will continue to do this day in, day out."

    Critics argue that the knowledge gained from the experiments is either non-existent or not worth the risk; Fouchier argues that GOF experiments are the only way to learn crucial information on what makes a flu virus a pandemic candidate.

    "If these three traits could be caused by hundreds of combinations of five mutations, then that increases the risk of these things happening in nature immensely," Fouchier says.

    "With something as crucial as flu, we need to investigate everything that we can," Fouchier says, hoping to find "a new Achilles' heel of the flu that we can use to stop the impact of it."

    The misguided history of female anatomy

    From "mutilated males" to "wandering wombs," dodgy science affects how we view the female body still today.

    Credit: Hà Nguyễn via Unsplash
    Sex & Relationships
    • The history of medicine and biology often has been embarrassingly wrong when it comes to female anatomy and was surprisingly resistant to progress.
    • Aristotle and the ancient Greeks are much to blame for the mistaken notion of women as cold, passive, and little more than a "mutilated man."
    • Thanks to this dubious science, and the likes of Sigmund Freud, we live today with a legacy that judges women according to antiquated biology and psychology.
    Keep reading Show less

    Why do holidays feel like they're over before they even start?

    People tend to reflexively assume that fun events – like vacations – will go by really quickly.

    Mind & Brain

    For many people, summer vacation can't come soon enough – especially for the half of Americans who canceled their summer plans last year due to the pandemic.

    Keep reading Show less
    Quantcast