Skip to content
Starts With A Bang

This Is The One Key Difference Separating Good Science From Junk Science

A device designed to simulate a working cold fusion reaction, but that was in fact a deliberate deception. Not every piece of junk science is, in fact, a deliberate deception; sometimes we simply fool ourselves despite doing our very best. (JUAN-LOUIS NAUDIN, 2003)

The line between science and pseudoscience is imperceptible to most. Here’s how you can catch it in yourself.

When it comes to science, the first principle is that you must not fool yourself, and you are the easiest person to fool. This is true in all fields of science: from social sciences to physical sciences; from theory to experiment to observation. Whenever you’re considering a scientific problem, any well-meaning inquiry will attempt to formulate and determine a decisive solution to it, uncovering the secrets of the natural workings of the Universe.

For the scientist working towards that solution, though, there’s an extraordinary danger afoot. No matter how dispassionate we attempt to be, we cannot help but be colored by the biases of our preferred outcome, no matter how subtle or overt it is. If we perform our scientific inquiry with a view towards the results supporting the conclusion we desire, we run the risk of doing nothing but junk science, rather than sound science.

From 1955, Jack Rogers, estate manager at Piltdown manor, Sussex, points to the monolithic tombstone of ‘Piltdown Man’, who allegedly died 500,000 years ago. Cranium fragments were found on the site from 1908 to 1912, but in 1953 they were formally denounced as fakes. The entire endeavor was an example of scientific fraud. (Peter Purdy/BIPs/Getty Images)

This can manifest itself in any number of ways. The most obvious is through outright scientific fraud, where the investigators begin with the conclusion they wish to reach and work backwards from there. If you’re someone who’s convinced of what the conclusion must be before you even begin, such as:

  • that human beings were created in their current form by an all-powerful, all-knowing being,
  • that the current changes Earth’s climate are experiencing must be due to the Sun,
  • that the extinction of the dinosaurs could not be due to an asteroid strike,
  • or that chemicals found in vaccines are the cause of autism in humans,

you’re going to find a way to validate that conclusion regardless of what the data says. Countless scientific frauds — involving everything from selection bias to the post hoc fallacy to simply faking your results — have been perpetrated in this manner upon both the scientific communities and general public.

A banner is shown during a 2017 rally against the obligation to vaccinate children. While there is significant public perception that vaccines are unsafe and unnecessary, the overwhelming suite of scientific evidence demonstrates that they save hundreds of thousands of lives annually and pose a negligible risk to public health and safety. (Nicolò Campo/LightRocket via Getty Images)

But fraud is a strong word, and implies that there’s an intent to deceive. Oftentimes that isn’t the case at all; the researchers aren’t engaging in an elaborate scheme to deceive an external observer. Instead, they’re victims of what Richard Feynman was alluding to when he said, “and you are the easiest person to fool.”

Travel the Universe with astrophysicist Ethan Siegel. Subscribers will get the newsletter every Saturday. All aboard!

Researchers of all types can succumb to self-deception, where you over-emphasize the portions of the evidence that support your preferred conclusion, while simultaneously discounting any evidence that points away from it. This is a classic way that motivated reasoning can manifest itself, which, as human beings, we do all the time in our daily lives.

Light, whether passed through two thick slits (top), two thin slits (middle) or one thick slit (bottom), displays evidence of interference, pointing to a wave-like nature. (BENJAMIN CROWELL)

When we disagree and argue and debate, the victor is usually the side with the more persuasive argument. In science, however, persuasion often has very little to do with scientific truths. In the early 1800s, there was a debate over whether light was a particle or wave, with a compelling argument demonstrating the absurdity of the wave theory put forth by Simeon Poisson, winning over many scientists. However, when the apparently absurd prediction was actually put to the test by François Arago, it was borne out, demonstrating that reductio ad absurdum does not work in science. Only experiments and observations can determine truths about the Universe.

The results of an experiment, showcased using laser light around a spherical object, with the actual optical data. Note the extraordinary validation of Fresnel’s theory’s prediction: that a bright, central spot would appear in the shadow cast by the sphere, verifying the absurd prediction of the wave theory of light. (THOMAS BAUER AT WELLESLEY)

This occurs again and again throughout the history of science, whenever conventional wisdom is overturned.

  • Ludwig Boltzmann faced massive ridicule for his work on thermodynamics, eventually committing suicide in 1906. His work was largely carried on and extended by Paul Ehrenfest, who faced similar ridicule, committing suicide in 1933. Their work laid the foundation for modern statistical mechanics.
  • In 1927, Georges Lemaître put together data about the redshift and distance measurements of galaxies to infer the expanding Universe, writing to Einstein about his findings. Einstein responded, “Your calculations are correct, but your physics are abominable.” Yet Lemaître was correct, with his conclusions predating Hubble’s identical ones by two years.
  • Fritz Zwicky, who first inferred the existence of dark matter in the 1930s, had his results dismissed based on the absurdity that such a significant fraction of the Universe could be hitherto undetected. The work of Vera Rubin and Kent Ford in the 1970s led to dark matter being seriously considered, but the work of Zwicky could have given us a 40 year head-start on the puzzle.
The two bright, large galaxies at the center of the Coma Cluster, NGC 4889 (left) and the slightly smaller NGC 4874 (right), each exceed a million light years in size. But the galaxies on the outskirts, zipping around so rapidly, point to the existence of a large halo of dark matter throughout the entire cluster. The mass of the normal matter alone is insufficient to explain this bound structure. (ADAM BLOCK/MOUNT LEMMON SKYCENTER/UNIVERSITY OF ARIZONA)

The thing is, each of these scientists had a great, compelling, interesting idea, and went to work on it. But they also were guided by data and evidence that pushed them in that direction, rather than ideology alone. New ideas that fly in the face of the current scientific consensus may be the way that many scientific theories arise and the way that scientific revolutions occur.

But this is also how bad ideas arise, and more dangerously, how they persist. When you formulate a scientific theory that attempts to replace the existing one, you absolutely must come up with a set of tests that allow you to discern its predictions, robustly, from the currently dominant paradigm.

The results of the 1919 Eddington expedition showed, conclusively, that the General Theory of Relativity described the bending of starlight around massive objects, overthrowing the Newtonian picture. This was the first observational confirmation of Einstein’s General Relativity, and appears to align with the ‘bent-fabric-of-space’ visualization. (THE ILLUSTRATED LONDON NEWS, 1919)

The way new scientific theories emerge and replace old ones isn’t set-in-stone, but there are three features that they all share. They are:

  1. able to reproduce all of the successful explanations that previously-dominant theory explained,
  2. able to successfully explain at least one already-observed or already-measured phenomenon that the prior theory couldn’t explain, and
  3. able to make new and measurable predictions, for hitherto unobserved phenomena, that will be distinct from the predictions of the current theory.

This is how Einstein’s General Relativity came to replace Newton’s gravity. This is how plate tectonics rose to geological prominence. This is how DNA added an additional layer to genetics, which itself added an additional layer to Darwinian evolution. And this is how cosmic inflation advanced the Big Bang, which itself advanced our picture of the expanding Universe.

Fluctuations in spacetime itself at the quantum scale get stretched across the Universe during inflation, giving rise to imperfections in both density and gravitational waves. Whether inflation arose from an eventual singularity or not is unknown, but the signatures of whether it occurred are accessible in our observable Universe. (E. SIEGEL, WITH IMAGES DERIVED FROM ESA/PLANCK AND THE DOE/NASA/ NSF INTERAGENCY TASK FORCE ON CMB RESEARCH)

This is why it’s of paramount importance to make sure you aren’t fooling yourself. You can fit galaxy rotation curves without dark matter, by modifying the laws of gravity. You can explain the orbits of a large suite of trans-Neptunian objects by invoking a large planet many times the Neptune-Sun distance. And you can explain the masses of protons and neutrons without requiring the Higgs boson.

If these are the outcomes you desire, though, you must make sure to protect yourself against discounting the evidence that contradicts these conclusions. You must look at the full suite of evidence, honestly and robustly, and make sure you meet all of the necessary criteria. When you do, dark matter succeeds in a myriad of regimes were modified gravity fails; Planet Nine’s existence faces serious doubts; and the Higgs exists as the explanation for the rest masses of fundamental particles.

A Higgs boson event as seen in the Compact Muon Solenoid detector at the Large Hadron Collider. This spectacular collision is 15 orders of magnitude below the Planck energy, but it’s the precision measurements of the detector that allow us to reconstruct what happened back at (and near) the collision point. Theoretically, the Higgs gives mass to the fundamental particles; experimentally, its existence has been verified and its properties measured. (CERN / CMS COLLABORATION)

It’s extraordinarily tempting to imagine that the next great advance in science is just a breath away, and even more tempting to believe that you, yourself, are going to be the one who discovers it. That’s why you should be extra skeptical of grandiose claims when they’re made by a person who’s spent a considerable fraction of their life promoting one particular alternative to the mainstream. Science requires independent checks and verification of experiments; science requires the gathering of more and better data to strengthen and further validate preliminary conclusions; science requires that theories have predictive power that can be robustly tested.

You must not look at only the data that supports your conclusion; you must examine and consider the full suite of data. We might sometimes fool ourselves, but sound, reproducible science will always carry the day. As Feynman said, “reality must take precedence over public relations, for Nature cannot be fooled.”

Ethan Siegel is the author of Beyond the Galaxy and Treknology. You can pre-order his third book, currently in development: the Encyclopaedia Cosmologica.


Up Next