The scientist’s 10 commandments

From the 10 biblical commandments to the first 10 amendments of the US constitution, which comprise the Bill of Rights, there are many lists of governing rules that apply to our individual lives as members of civilized society. Yet for scientists, there are many other “best practices” we need to constantly keep in mind as we investigate the natural phenomena around us, from the subatomic world to the cosmic realm and everything in between. Although we often talk about following the scientific method, we rarely lay out anything more than a rough procedure to follow when it comes to conducting science itself.

Sure, it’s important to have ideas, formulate hypotheses, and then devise methods to test those hypotheses, gather results, and draw conclusions that either support and validate or contradict and refute those hypotheses: the rough outline of how science is performed. But there’s so much more that goes into being a scientist that gets to the very core of what it means to investigate the origin, nature, and root cause of any phenomena that we dare to observe, design experiments around, and measure. Here, without further ado, are the 10 commandments that anyone who wishes to conduct good, quality science needs to follow.

The results of an experiment, showcased using laser light around a spherical object, with the actual optical data. Note the extraordinary validation of Fresnel’s theory’s prediction: that a bright, central spot would appear in the shadow cast by the sphere, verifying the absurd prediction of the wave theory of light. Logic, alone, would not have gotten us here.

1.) Thou shalt not assume thy preferred conclusion is correct. This is perhaps the most important starting point, and also the one that is most tempting to violate. Most of us don’t have ideas for how things work in an agnostic sense: where we wonder if something can be true and then, carefully and dispassionately, work toward robustly testing it to see if it pans out. Instead, most of us, whenever we have an idea, desperately want it to be a good idea. If we’ve seen an idea from another that strikes us as clever and/or compelling, it’s easy to be captured by it, and to want that idea to be true as well.

But this line of thinking is how you end up with a strongly held belief, not how you wind up determining whether something is true, and in line with the way reality works, or not. If you go into an endeavor thinking that you know what the end result is going to be — and you design your experiments and/or measurement apparatus with those expected results in mind — it’s all too easy to confirm your own preferred conclusions. This isn’t because the data will necessarily support it, but rather because you will look for errors and sources of uncertainty right up until you get an agreeable result, and then stop. If you don’t want to wind up with climate science denial, creationism, and seeing aliens in mundane phenomena, you won’t forget this first, all-important commandment.

Touted as UFO lights, this 2011 photograph over Liverpool does, in fact, document a series of lights in an oddly unnatural-appearing configuration, but this is by no means compelling evidence for aliens. In fact, it’s simply an image looking up at the lights of Radio City Tower, which does not constitute scientific evidence for UFOs.

2.) Thou shalt always consider the full suite of relevant data when drawing conclusions. When it comes to questions about how successful a theory, model, idea, or hypothesis is, it’s insufficient to look only at a subsample of data. The danger is that, by valuing only the data that supports-or-refutes one conclusion and discounting the data that refutes-or-supports other potential conclusions or explanations, we can wind up erroneously drawing incorrect inferences. The way to combat this is to fairly consider the full suite of data that’s relevant to a particular problem, and to draw the best conclusions one can with a view toward that entire set of data, combined.

The alternative is known as cherry-picking, and allows you to come up with whatever your preferred conclusion is by ignoring vast quantities of valuable data. If you look only at the portions of the data that support a certain conclusion or set of conclusions, you could easily disfavor the best scientific theories of our time and instead support alternatives that, on the whole, are non-viable. This is inevitably what happens when politics, religion, or ideology (including wishful thinking) infiltrates science and corrupts scientific reasoning. The only way out of such a trap is to conduct more science, better science, and scrupulous science, which is why it’s so important that the scientific enterprise be free and open to all.

This map shows the reconstructed mass from gravitational lensing of the Bullet Cluster: Galaxy Cluster 1E0657-558. The contours, overlaid atop the optical data (left) and the X-ray data (right), clearly show a separation of the normal matter from the effects of gravitation, making it impossibly hard for modified gravity models to mimic this without behaving identically to dark matter.

3.) Thou shalt remember the limits of thy theory’s range of validity, and only extend it cautiously. No matter how successful or powerful a scientific idea is, there is only ever a certain range over which its validity is established. No matter how sacrosanct your theories are — and no matter how towering the historical figures are that established them — we have to be incredibly cautious about extending those theories beyond their established range of validity. Einstein’s general relativity was demonstrated to hold on terrestrial and Solar System scales, for example, but extending it to the grander cosmic scales required an enormous marriage of theoretical and observational work before we were able to conclude that yes, the Universe is expanding and traces its early days back to a hot, dense, uniform state: the hot Big Bang.

Today, for example, we have quantum field theory governing all the particles and fields known to exist in this Universe, and the theory of cosmological inflation for setting up the initial conditions of the hot Big Bang: two well-defined and soundly established theories. Yet, when you put these two together, one of the consequences is that you arrive at a multiverse consisting of an enormous number of disconnected universes, all with their own hot Big Bangs. Although there are good reasons to believe that this multiverse is physically real, the application of these theories to this early epoch goes beyond their established range of validity, and thus any conclusions we draw must be given their proper caveats. When we venture into the unknown, we must always be appropriately cautious.

This figure shows the development and discovery of superconductors and their critical temperatures over time. The different colors represent different types of materials: BCS (dark green circle), Heavy-fermions-based (light green star), Cuprate (blue diamond), Buckminsterfullerene-based (purple inverted triangle), Carbon-allotrope (red triangle), and Iron-pnictogen-based (orange square). The novel states of matter achieved at high pressures have led to the current records, although recent dubious studies (many of which have since been retracted) have been omitted.

4.) Thou shalt make public thy data, methods, and results, for all to consider and scrutinize. One of the most frustrating experiences a scientist can have is to:

• read the news,
• see an outlandish claim being made on scientific grounds,
• and then go look for the supporting paper/data to view the source,
• only to find out that the paper/data is either not public or does not even exist.

One of the hallmarks of shady science is the existence of claims whose veracity cannot be checked and scrutinized, as independent confirmation and verification are always required for any claim that’s being made.

This is where “sham” research, like cold fusion (or low-energy nuclear reactions), is often conducted, and where would-be legitimate research can be polluted by fraud, including the faking of data. Only in the open light of day — where everything about the research is made public — can others hope to reproduce what one group, team, or researcher has claimed. Without open science, including open data, open methods, and open results, we are subject to being easily fooled by charlatans who would exploit us for their own personal gain. Science is one of the strongest guardrails a society has against that kind of exploitation, and whenever we weaken those guardrails, it increases the peril to not only science, but all of humanity.

In sexual reproduction, all organisms have two pairs of chromosomes, with each parent contributing 50% of their DNA (one set of each chromosome) to the child. Which 50% you get is a random process, allowing for enormous genetic variation from sibling to sibling, significantly different than either of the parents.

5.) Thou shalt remain tenaciously skeptical of any hypothesis that thou encounters. The first rule of science is that “nothing is sacred.” Not Einstein, not Newton, not Darwin; not anyone. Everything not only can be challenged, but must be challenged: wherever a new challenge can be devised. This is because even our greatest, most established theories not only have a limited range-of-validity that they demonstrably have been shown to apply to, but because science is the quest for an ever-better approximation of reality. The only way to show that the approximation is just that — an approximation — is to find at least one location, instance, or circumstance where it breaks down.

And this is why continued testing and skepticism of ideas is so important from a scientific standpoint. Einstein’s general relativity is perfect over the scales at which it’s been tested, but it cannot apply to quantum particles exhibiting quantum effects, as it’s purely a classical theory. Newton’s laws work great, except very close to large, compact masses and at speeds nearing the speed of light, which is how Einstein came to overthrow Newton. And Darwin’s theory of evolution by natural selection is wonderfully successful, but doesn’t shed light on the mechanism of inheritance (genetics and DNA) and doesn’t answer questions about the origin of life. No matter what hypothesis you encounter, the job of a scientist is to probe it and try to knock it down, as only the ones that are most robust and resilient to challenges will survive.

This association of nine gamma-ray bursts, all located around 9 billion light-years away but separated by more than 5 billion light-years from end-to-end, were seen over a period of 8 years: from 2004 through 2012. The “connected superstructure” shown here is only an artist’s representation; it is undetermined whether there is any physical relation between the various gamma-ray bursts and unknown whether any two of them are even part of the same cosmic structure. It is very likely that they are all mutually expanding away from one another: caught up in the expansion of the Universe.

6.) Thou shalt quantify, respect, and not minimize thine sources of error and thy potential biases. There have been a great many claims that are made in the popular media — that yes, even have their roots in claims made in the scientific literature — that completely ignore this commandment. These include:

• claims of large cosmic structures that violate the cosmological principle,
• claims of the existence of a large, massive, ninth major planet in the outer Solar System,
• and claims that wide binary star systems definitively either rule out Newtonian gravity or modified Newtonian dynamics,

to name just a few. But all of the studies that make such claims have something in common: they silently overlook and omit major sources of uncertainty, and in particular, sources of bias in the data sets selected.

It’s very easy to take a large data set, apply some corrections and a calibration mechanism to them, and draw whatever conclusion that data then supports. But what’s more difficult is to understand the difference between the data you’ve collected and what the full suite of relevant data — assuming you were capable of collecting and measuring it all — would indicate. The issue is that the data you have, and that you’re drawing from to infer your conclusions, is not necessarily representative of the true distribution of objects/data that underlie what you’re attempting to measure. Whether you’re missing faint stars and galaxies or overlooking objects that only appear when you aren’t taking measurements, these biases tremendously impact what types of conclusions can robustly be drawn. Biases may be present, but only by quantifying them, honestly, can we move forward in science.

If one wants to investigate the signals within the observable Universe for unambiguous evidence of super-horizon fluctuations, one needs to look at super-horizon scales at the TE cross-correlation spectrum of the CMB. With the final (2018) Planck data now in hand, the evidence is overwhelming in favor of their existence, validating an extraordinary prediction of inflation and flying in the face of a prediction that, without inflation, such fluctuations shouldn’t exist.

7.) Thou shalt not accept a new theory as representative of reality until it clears all three of the necessary, critical hurdles. If you pay any attention to science news, you know how popular stories that tout “alternatives to the mainstream” can become. But theories that are part of the mainstream have gotten there for a reason: the data from a wide variety of independent lines of evidence all support it, while alternative explanations all fail to account for the full suite of data. However, as noted earlier, any theory — even the best of theories — can only be treated as a provisional approximation to reality. At some point, we expect all theories to be superseded by others that approximate reality a little bit better.

To get there, there are “only” three big hurdles that need to be cleared.

• The new theory must successfully reproduce all of the successes of the old theory, with no exceptions.
• The new theory must explain something that the old theory could not; this is usually the impetus for considering a new idea.
• And finally, one must tease out new, untested predictions where the new theory and the old theory differ, and the critical data must then be taken: showing that it favors the new theory and refutes the old one.

Darwin’s evolution successfully did this against other rival theories of evolution; Mendel’s genetics then went a step further than Darwin by explaining the mechanism of evolution; Watson, Crick, and Franklin’s DNA went a step further than Mendel by explaining the underlying code behind genetics. No matter how far we’ve come, we must remember, there is always farther we can go.

Grains of the genetically engineered Golden Rice (right), compared to the Vitamin A-deficient white rice (left). Hundreds of thousands of children per year, worldwide, on primarily rice diets, go blind from lack of Vitamin A, with even greater numbers dying from it outright. Golden Rice could cure that problem. Scientists, but also many other professionals in public health, marketing, politics, and more are needed to transform this scientific solution into a practical solution, as misinformed advocates frequently and vociferously oppose this life-saving measure.

8.) Thou shalt obtain approval and consent from all relevant bodies before conducting research that may impact others. This is a big one that should apply, on ethical grounds, in more realms than we practically demand it. If you’re going to experiment on humans, you need to obtain IRB (internal review board) clearance to perform your work. Likewise, if you’re going to do something that has the potential to have a gross negative impact on the environment in which we all live, such as performing a massive geoengineering experiment, you should need to get consent and approval by whatever agencies are in charge of maintaining the planet’s environment.

Travel the Universe with astrophysicist Ethan Siegel. Subscribers will get the newsletter every Saturday. All aboard!

Science, at its core, is about improving human knowledge and equipping us with the tools to develop superior technology for the benefit of all humankind. What it isn’t about is obtaining results no matter the cost to others. Science is about investigating and learning about nature, and then taking what we learn and using it as a force for good in the world. We have ethical rules about governance and how to perform science, but scientists have historically been complicit in enormous atrocities against nature and against other humans. If we fail to heed this commandment, we not only run, but increase, the risk of our own self-destruction, and will have only unchecked hubris and unregulated ambition to fault.

Using publicly available data from NOAA, one can plot the global average land-and-ocean temperature anomaly, compared to the 20th century average, from the year 2000 to the present. The data is so strongly indicative of warming that a no-warming scenario can be ruled out with greater than 7-sigma confidence based on data from 2000-2021 (or further) alone.

9.) Thou shalt not exaggerate the significance of thine results in thine studies. One of the enormous difficulties science faces is that once something is widely disseminated as a “scientific fact,” it becomes a herculean task to correct the record. Yet many of these “scientific facts” that we were taught are based on:

• small studies,
• with small sample sizes,
• that often rely on highly biased samples (such as 18-21 year old college men),

that then draw sweeping conclusions about a very broad set of phenomena. This is not only scientifically without merit, but it poses a tremendous set of dangers to anyone impacted.

For example, have you heard about the conventional wisdom that “humans shouldn’t eat more than 30 grams of protein at a serving, as they can’t digest or utilize more than that?” It turns out that piece of information was based on one small study conducted many decades ago, and that was never reproduced. This was, in fact, a cornerstone of nutrition science until 2023, when a reptile researcher heard of it and thought it was absurd, and so put it to the test. (If a snake devours an alligator, is it limited to making use of just 30 grams of protein? Of course not!) What they found, in humans, was the same thing it found in reptiles: there is no effective limit, and the more you eat, the more you can use. Similarly, studies that are based on college-age men that are used to draw conclusions about middle-aged and elderly women are tremendously suspect; we must take care to recognize the limitations and the (oftentimes, very low) statistical significance of even major, impactful studies.

The results of the 1919 Eddington eclipse expedition showed, conclusively, that the general theory of relativity described the bending of starlight around massive objects, overthrowing the Newtonian picture. This was the first observational confirmation of Einstein’s theory of gravity.

10.) Thou shalt hold even the best of scientific theories, models, and frameworks as provisional only, and constantly seek to test, revise, and refine them. It’s easy, at any point in history, to assume that either we already know it all, or that once we get just a few relatively simple points-of-contention under control, we’ll then know it all. Of course, if the history of science has taught us anything it’s that nothing could be further from the truth. In science, we often talk about it as though it’s the search for truth, but in all honesty, it’s a search that will forever be limited and incomplete: we’ll only uncover the truth to the limits that our collected data, measurements, observations, and experiments are capable of revealing it.

This is why it’s always important to recognize that even our greatest scientific theories of all are only provisional. We need the freedom to speculate about the unknown, to challenge assumptions about what is presumed to be known, and to “play in the proverbial sandbox” when it comes to constructing new models or exploring ideas that have already been disfavored. It’s through this type of theoretical detective work, even when it’s just playful investigation, that the first hints of a deeper truth often appear. While science won’t ever tell us that “everything we think we know today is wrong,” it can teach us that there are better, more correct, and possibly more profound conceptions of reality than the ones that currently hold sway today. But unless we adhere to these ten commandments, we’ll either succumb to self-destruction or self-deception, and those are the greatest dangers to a society where free and fair scientific inquiry is a routine part of civilized life.