Scientific Theories Never Die, Not Unless Scientists Choose To Let Them

As wonderful as the evidence that supports or invalidates a theory is, it can never truly kill the ones that don’t work out.

When it comes to science, we like to think that we formulate hypotheses, test them, throw away the ones that fail to match, and continue testing the successful one until only the best ideas are left. But the truth is a lot muddier than that. The actual process of science involves tweaking your initial hypothesis over and over, trying to pull it in line with what we already know. It involves a leap-of-faith that when you formulate your theory correctly, the predictions it makes will be even more successful, across-the-board, than any other alternatives. And when things don’t work out, it doesn’t always necessitate abandoning your original hypothesis. In fact, most scientists don’t. In a very real way, scientific theories can never truly be killed. The only way they ever go away is if people stop working on them.

When distant supernovae were first discovered to be fainter than they otherwise should have been based on their redshift, it brought about a revolution in cosmology. The way the Universe expands is inextricably linked to the matter and energy present within it, and so the goal of cosmology, for a long time, was to measure the expansion rate and how it changes over time. The expectation was that it would either recollapse or expand forever, or remain in an in-between state right on the border between those two. Instead, these supernovae showed that a fourth option was most likely: the most distant galaxies of all were speeding up as they moved away from us. There must be some new form of energy in the Universe — dark energy — different from all other forms of energy, permeating all of space.

But for many years, most physicists and astronomers approached this idea with skepticism, wondering if there weren’t another explanation. Perhaps, one alternative theory posited, space wasn’t expanding with an extra value due to some form of dark energy, but rather there was something occurring at large distances to block the light. So that became a proposition: there was some additional dust in the distant Universe, and the reason the supernovae appeared fainter wasn’t because they were farther away due to an extra expansion of space, but because dust was blocking the light.

Dust grains, however, come in particular sizes, and the size of the dust grains determines which wavelengths of light are preferentially blocked, with most dust better at blocking blue than red light. Measurements of different wavelengths of light, however, showed that both red and blue light were reduced by equal amounts.

Was that sufficient to rule out the “dust” theory? In that incarnation, yes. But what if the dust in the distant Universe was of a new type, that blocked all the wavelengths of light equally? This undiscovered type of dust, dubbed “grey dust,” could block all wavelengths equally. So we needed some way to put that to the test, and that involved looking at supernovae at a variety of distances, to see whether dust would continue to block more and more light at greater distances, as more and more “grey dust” would tend to do.

It didn’t. So does that mean dark energy must be real? Not necessarily, because you can modify your “grey dust” explanation to include dust that changes in density and location over time: “replenishing grey dust.” By the addition of enough extra free parameters, caveats, behaviors, or modifications to your theory, you can literally salvage any idea. As long as you’re willing to tweak what you’ve come up with sufficiently, you can never rule anything out.

There have been many ideas in this vein that have the same problem (or feature) inherent to them: so long as you’re willing to make the theory more complicated, you can fit any data that comes back. The discovery of the CMB ruled out the Steady-State theory, but they added reflected starlight to explain that leftover glow. When the spectrum of the CMB was measured, ruling out reflected starlight, they added a series of bursts and “mini-bangs” in the past, creating a Quasi-Steady-State theory. When the fluctuations in the CMB’s temperature were discovered, ruling that out, its proponents tweaked it still further.

This behavior isn’t unique to scientists, but has been a feature (or bug) of science for centuries. It led Max Planck, more than 100 years ago, to make the following now-famous statement:

A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.

This is often paraphrased as “physics advances one funeral at a time,” owing to the fact that ideas cannot be proven wrong as we commonly think. Rather, they need to be tweaked so thoroughly and so frequently that they lose their predictive power, instead always playing catch-up as new observations come in.

It’s why theories like quantum field theory and general relativity are so powerful: even after all these decades, they’re still making new predictions that are being successfully borne out by experiment. It’s why dark matter is here to stay, as its successful predictions include the speeds of galaxy pairs, the large-scale cosmic web, the fluctuations in the CMB, baryon acoustic oscillations, gravitational lensing and more. It’s why cosmic inflation — with its successful predictions including superhorizon fluctuations, the acoustic peaks in the Big Bang’s leftover glow, the departure from scale invariance, etc. — is the leading theory for the origin of the Big Bang. And it’s why their alternatives are so thoroughly fringe.

You can always add another loophole, parameter, or epicycle to your own pet theory to make it be “not ruled out.” I, along with most physicists, feel this way about a great many non-standard alternatives, including MOND, f(R) gravity, the Quasi-Steady-State model, tired-light cosmology, the plasma Universe, and so on. At some point, you just have to say “enough.” You have to recognize that the level of contortions you need to perform are absurd, and that these theories don’t have any useful predictive power. They’re simply an example of special pleading.

Of course, their adherents don’t think so. They think they’re being marginalized, oppressed, ignored, or not taken seriously. On very rare occasion, they’re actually correct, and that’s when a scientific revolution occurs. It’s important to keep your mind open to those possibilities, to explore them, and to consider what it would look like if these alternatives were correct after all. But for the overwhelming majority of scientists working on these alternative ideas, their life’s work will turn out to be a blind alley, and their ideas will die out when they (and possibly their students) die. It’s both sad and tragic to look back at history and realize that the last decades of the scientific careers of Einstein, Hoyle, Burbidge, Schrodinger, and many more were a total waste. But whether even the most brilliant scientist accepts a new scientific truth or not is irrelevant. Our knowledge and understanding march forward.