Why Are Scientists So Cruel To New Ideas?

Demanding appropriate levels of skepticism and scrutiny isn’t cruelty, but rather demonstrates scientific integrity and intellectual honesty.

Every few months, a novel headline will fly across the world, claiming to revolutionize one or more of our most deeply held scientific ideas. The declarations are always sweeping and revolutionary, ranging from “the Big Bang never happened” to “this idea does away with dark matter and dark energy” to “black holes aren’t real” to “maybe this unexpected astronomical phenomenon is due to aliens.” And yet, despite the glowing coverage of the novel proposal, it most frequently languishes in obscurity, attracting little mainstream attention other than a myriad of dismissals.

Commonly, it’s portrayed that scientists in this particular field are dogmatic, wedded to old ideas, and close-minded. This narrative might be popular among contrarian scientists or those who themselves hold fringe beliefs, but it paints a disingenuous picture of the scientific truth. In reality, the evidence supporting the prevailing theories are overwhelming, and the new headline-grabbing proposals are no more compelling than the scientist’s equivalent of playing in the sandbox. Here are the four big flaws that commonly occur with new ideas, and why you’ll never hear about most of them again after they’re first put forth.

1.) When you work, every day, with “the real McCoy,” you can immediately spot an impostor’s shortcomings. In science, we have accumulated an enormous body of knowledge — a set of experimental and observational data — and a set of theories that provides a framework to accurately describe the governing rules of our reality. Many of the results that we obtained were initially bizarre and counterintuitive, with multiple theoretical possibilities proposed to explain them. Over time, further experiments and observations winnowed them down, and the most successful theories with the greatest degrees of validity were the ones that survived.

Proposals that attempt to revolutionize one (or more) of our accepted theories have a large suite of hurdles to overcome. In particular, they must:

• reproduce all the successes of the prevailing theory,
• explain a phenomenon more successfully than the current theory can,
• and make novel predictions that can be tested that differ from the theory it’s attempting to supersede.

It’s very rare that all three of these criteria are met. In fact, the overwhelming majority of these grand proposals fail on even the first point.

Attempts to explain the Universe without a hot Big Bang fail to explain the existence and properties of the Cosmic Microwave Background: an omnidirectional pattern of radiation that’s been known for some 55+ years. Contentions that gravitational wave detectors are seeing noise, rather than signals, ignore the large suite of evidence that links electromagnetically observed events with their gravitational wave counterparts. And the idea that gravity might emerge from another entity, like entropy, yield absurd results for the problem of dark matter, failing to maintain the necessarily constant ratio of dark matter to normal matter.

It’s not enough, by scientific standards, to simply propose a wild idea that explains one property that the prevailing, presently accepted theory has difficulty with. One new observation can always be explained by one new “free parameter,” which is a kindhearted way to say “invoking something brand new.” If that new theoretical addition lacks the power to explain other phenomena as well, however, it’s not likely to gain serious traction of any type.

2.) Many “new ideas” are unoriginal repackagings of old, discredited ideas that don’t merit reconsideration. Most of us, if we have any sort of imagination at all, have played the “what if” game about some aspect of reality at some point. Perhaps you’ve wondered about this yourself, and have had ideas such as:

• What if you traveled in a straight line through the Universe for a long enough distance; would you ever come back to your starting point?
• What if the particles that we think of as fundamental today — quarks, electrons, photons, etc. — are actually composite particles made up of more fundamental components?
• What if there’s some sort of extra, new field in the Universe that permeates all of space, and that’s the explanation behind what we’re currently calling “dark matter” and “dark energy?”

All of these ideas are good ideas. There are many papers that have been written about them, and explored them in great detail.

But each and every one of them has difficulties that led to them being abandoned, and no new evidence has come in to favor them over the prevailing theories. For example, the idea that the Universe might have a non-trivial topology continues to be interesting, but if it does, the evidence demonstrates that whatever the “size” of the Universe is, it must be significantly larger than the entire observable Universe. If any of our fundamental particles are composite particles, they don’t exhibit that behavior under any of the experimental conditions we’ve ever probed.

And if there is no dark matter or dark energy, but rather a field explanation instead, then that explanation requires at least two new free parameters: a “clumpy” one that behaves like dark matter, and a “smooth” one that behaves like dark energy. You don’t gain anything by these reformulations, and in many cases, you’ve just added more complexity to explain a puzzle in an inferior fashion. There’s no reason you can’t explore these avenues, but unless you can either explain something the prevailing theory cannot or you can reduce the number of free parameters required by your theory, you haven’t done anything more than playing in the sandbox.

3.) It’s fundamentally unscientific to begin with an ideologically motivated conclusion. This is one of the most dangerous pitfalls that scientists — particularly young and inexperienced scientists — can fall into. If you have a puzzle or problem that vexes or fascinates you, you might have a thought along the lines of, “wouldn’t it be fascinating if ____________ explained what we were seeing?” There’s absolutely nothing wrong with having that thought, and there isn’t even anything wrong with exploring the theoretical consequences of what your idea would imply for the Universe we have the capacity to observe.

But there’s a line that, once you cross it, pushes you over the line from legitimate scientist into crackpot territory: when you become convinced that your idea must be correct. As soon as you take that leap, you’ve decided that “I know what the conclusion is,” and that means you’ll fiddle with your theory until it gives you the conclusion you know you need to reach. This type of model-building-by-working-backwards might give you the result you want, but it won’t be a scientific result.

Many scientists have fallen prey to this pitfall. Fred Hoyle became convinced that the Universe must be in a steady state and could not have a hot, dense origin, despite the overwhelming evidence supporting the Big Bang. Arthur Eddington was convinced that the stars in the Universe could never achieve properties beyond certain limits, despite the observational evidence that those limits were frequently exceeded. Even Einstein himself became convinced that quantum “randomness” must have a deterministic explanation and that gravity and classical electromagnetism would lead to a unified force; these avenues produced no consequential results over the last 20+ years of Einstein’s scientific life.

In many ways, these influential scientists held back progress in their field substantially until their deaths, with the lesson being that your physical intuition — regardless of who you are or what you’ve accomplished — is no substitute for the legitimate information that we gain by asking the Universe questions about itself. It’s why Johannes Kepler, who threw away his “beautiful” theory of nested spheres and perfect solids for the “ugly” theory of elliptical orbits that fit the data better than any other, remains such a spectacular role model for how to do science right.

4.) A scientist’s job is to rigorously attack their own hypothesis, and “new idea proponents” often fail to do that very job. Did you have an idea, and did you fall in love with it? Many of us do, and this is a tremendous problem for us. In science, it falls to us to be the harshest critics of our own ideas, as we will be the first to explore them in-depth, before we present our findings to the world where they will be evaluated by others. If you fail in the attempt to knock down your own ideas — to find its weak points, to expose where its range of validity ends, to identify where it compares unfavorably with the theory it seeks to replace — others will do that work for you.

That is not cruelty. That is not close-mindedness. And that is certainly not adherence to dogma. That is a necessary part of science: to subject any new hypothesis to rigorous scrutiny and evaluation. Though it may be unfortunate, most “new ideas” will fall apart under the weight of the evidence that has already been collected, just as most of the ideas that are originally proposed to explain a novel phenomenon turn out to fail spectacularly at describing the full suite of evidence that the Universe provides.

It’s easy to understand why, if you have an idea that you love, you’d want others to love it, too. But it’s very difficult to convince other scientists — particularly the scientists who embrace the notion of having appropriate levels of skepticism for ideas — that your idea is worth loving if you haven’t subjected it to the necessary scrutiny. If you want to propose a theory where the speed of light is different for different wavelengths of light, it had better not disagree with any of the multiwavelength observations we’ve already collected about light from distant objects, for example.

If you have an idea that falls outside of the mainstream, there are some questions you’ll definitely want to ask.

• What is the problem you’re considering that motivated this idea?
• How does this idea compare to the prevailing theory when applied to this specific phenomenon?
• How does this idea compare to the prevailing theory when applied to the other major successes of the prevailing theory?
• And what are some critical tests that you can legitimately perform (with current or near-future technology) to further discern your idea versus the prevailing theory?

As Richard Feynman once put it so eloquently, “The first principle is that you must not fool yourself — and you are the easiest person to fool.”

It is not an act of cruelty, dogmatism, or close-mindedness to demand scientific rigor. Instead, it is a mark of integrity and a commitment to finding the scientific truth surrounding whatever issue or phenomenon you’re investigating. There are many great, brilliant ideas that have been relegated to the historical dustbin of failed theories for the best reason of all: because they did not successfully line up with our observed reality. No matter how fanciful or compelling an idea may be, if it disagrees with experiment, measurement, and observation, it’s wrong.

There are plenty of compelling, interesting, and viable ideas that are out there, and there will always be plenty of room for speculation about the unknown. But whenever we consider a novel, alternative idea, we have to do it through the lens of scientific rigor. We cannot simply pick-and-choose the phenomena we wish to pay attention to while ignoring the aspects of reality that are inconvenient for our pet ideas.

In the end, the Universe will always be the ultimate arbiter of what is real and what theories best describe our reality. But it’s up to us — the intelligent beings that conduct the enterprise of science — to rigorously uncover those truths. Unless we do it responsibly, we run the risk of fooling ourselves into believing what we want to be true. In science, integrity and intellectual honesty are the ideals to which we must aspire.

Starts With A Bang is written by Ethan Siegel, Ph.D., author of Beyond The Galaxy, and Treknology: The Science of Star Trek from Tricorders to Warp Drive.