Goodbye, DAMA/LIBRA: World’s Most Controversial Dark Matter Experiment Fails Replication Test
We still don’t know what dark matter is, but at least we now know what it’s not.
When it comes to science, we often say that it only takes a single experiment to overturn a theory. But that’s predicated on a very big, often unspoken assumption: that the experiment is robust, and isn’t suffering from any major unseen errors. For decades, now, scientists have been searching valiantly for the elusive particle that might make up some or even all of the dark matter. While the astrophysical evidence supporting the existence of dark matter is overwhelming, every experiment designed and built to directly detect whatever particle might be responsible for dark matter has come up empty.
Every experiment, that is, except one: the DAMA/LIBRA experiment. While other experiments that are far more sensitive — including SuperCDMS, XENON, Edelweiss, LUX, and many others — have only detected negative results down to extreme precisions, DAMA/LIBRA has continuously observed a significant signal for about 20 years. At last, the critical test has been performed: a completely independent team, ANAIS, has carried out an identical experiment to DAMA/LIBRA, replicating the study and testing its validity. With three complete years of data collected, ANAIS has ruled out the DAMA/LIBRA results in a model-independent way to better than 99% confidence. The world’s most controversial dark matter experiment has been busted, and it’s an incredible success for the scientific method.
If you want to detect an elusive particle, you have a few different approaches you can take. You can:
- smash particles together, looking for missing energy-and-momentum that would result from producing an “invisible” particle,
- apply a detection technique designed to look for a specific class of particle — with a certain range of masses and interaction cross-sections — by building an enormous detector and accounting for all of your different backgrounds,
- or you can perform a more general detection technique, looking for an annual pattern in the detection rate as the Earth orbits the Sun, where (in theory) it should be colliding with dark matter particles at different rates throughout the year.
The first class of experiments are akin to what goes on at the Large Hadron Collider; unfortunately, the “missing energy” events we’ve seen are consistent with only the particles present in the Standard Model, like neutrinos. The second class of experiments are the large underground detectors that work very hard to measure nuclear recoils: events where particles, shielded by kilometers of Earth, interact with these large collections of mass. These direct detection efforts are generally sensitive only to particular energies and cross-sections, and are particularly model-dependent. (For example, they depend on what types of interaction with normal matter these hypothetical particles have, what their spins are, what their rest masses are, etc.)
But whereas the second class of experiment — like XENON, just to pick a data-rich example — approaches the problem by attempting to understand the root cause of every interaction that occurs in their detector, calculating their background exquisitely and identifying every possible source of contamination, there’s a third approach as well, taken by the DAMA/LIBRA experiment: look at your data for a change over time.
Why should this work?
The background events that occur are things that come from space and strike Earth, penetrating the crust, or otherwise come from things like neutrons (from radioactive decay), neutrinos (from cosmic rays and the Sun), muons (from cosmic rays), other radioactive products, and secondary particles that were produced by other, primary particles interacting and producing a shower of “daughter” particles.
But as Earth moves around the Sun and through the dark matter particles populating the entire galaxy, we should see what’s called an “annual modulation.”
During part of the year, the Earth should be moving with the Sun in its orbit around the galaxy, and so it’s passing more quickly through the dark matter particles that are present in the galaxy. Six months later, the Earth should be moving maximally against the Sun’s motion, decreasing the speed that the Earth moves through the galaxy’s dark matter. If even a fraction of the signal that’s produced in such a detector is due to dark matter, that portion of the signal will increase and decrease in a periodic fashion with a period of precisely one year.
Long before the idea of the DAMA/LIBRA experiment was put forth, a small team of theorists — Andrzej Drukier, Katherine Freese, and David Spergel — calculated exactly when and by what magnitude we’d expect this annual modulation to show up in a detector, and that’s what DAMA/LIBRA sought to detect. Instead of carefully calculating their background and teasing out different components until everything was accounted for, leaving only a potential signal behind, DAMA/LIBRA instead runs as a “model independent” detector, looking for this annual modulation alone.
What it began seeing, almost immediately, was a significant annual modulation in the event rate of the detector. Without an understanding of their noise background, however, much of the community has long been skeptical. Even as 20 years of data now yields an unambiguously significant result — in a world where a 5-sigma result is the gold standard, they’ve now crossed the 12-sigma threshold — the interpretation of the result has remained hotly disputed. The big question that we all want to know the answer to is why: why has this signal arisen at all?
- Is it because at least some of this signal in the detector is due to dark matter, where the quarter-ton of material used in the detector is occasionally getting hit by dark matter particles, and the dark matter interacts at different rates throughout the year?
- Or is it because the entirety of the signal in the detector is not due to dark matter in any way, and the annual modulation is either purely a function of noise, systematics, or the way the experiment itself is being performed?
For a long time, most scientists have suspected the latter, but they can’t prove it. Sitting on two decades of information, DAMA has yet to make their data, pipeline, and analysis public.
Last year, it was uncovered that one of the things the DAMA collaboration is doing is subtracting out their average noise value on a year-by-year basis from the data, and working with just the residuals: what’s left over when you subtract out the average.
That part, on its own, is okay if your noise is random.
But DAMA’s noise isn’t random, and neither is the noise of the two collaborations seeking to replicate the DAMA experiment: COSINE and ANAIS. Instead, the noise rises over time in the DAMA experiment, and the scientists basically press the “reset” button on the noise once a year, at the same time every year.
Combined with the subtraction of the average noise, this could lead to disaster: seeing a signal where there is none. If you have an increasing amount of noise but subtract the average noise from the whole thing, then the first part will show a below-average value, it will rise to the average value, and then it will rise to an above-average value. Then you hit the reset button for the next year, and it starts all over.
As the scientists who identified this problem last year showed, the entire 20-year span of data can be equally well-fit to a sawtooth wave, which is what this noise problem would yield, as it can be to a sine wave, which is what would lead to a conclusion of dark matter.
But the real test, even if we could independently analyze and verify that DAMA did everything in a correct and scrupulous way, would be to attempt to independently replicate DAMA’s results. That would involve performing the same type of experiment with the same materials, but with their own data acquisition and analysis pipeline. If they wanted to one-up DAMA, they would make their results publicly available.
Back in 2019, the COSINE collaboration published their first major set of results, finding no annual modulation but without a compelling enough data set to rule out what DAMA claims to have found.
But the new results from ANAIS — Annual modulation with NaI Scintillators (where NaI, sodium iodide, is the target material used in all three experiments) — are finally good enough to either confirm or refute the DAMA/LIBRA results. If they see an annual modulation, it supports the dark matter explanation; if they don’t, it support an “unknown noise artifact” explanation.
This graph, right here, is the most important one in the whole paper. It’s true that the error bars — a measure of the statistical uncertainty — are still quite large for the ANAIS collaboration, which is what you’d expect with just three years of data compared to 20 for DAMA. But these results are still good enough to see three major things from them.
- ANAIS shows that there is no evidence for annual modulation. As the authors put it, “We can conclude there are no statistically significant modulation in the frequency range searched in ANAIS-112 data.”
- The DAMA results — not just overall but over multiple energy ranges and with a variety of fitting procedures applied — are inconsistent at the ~99% level with the ANAIS results under every permutation.
- And that, by the autumn of 2022, they should have enough data to render the uncertainties so small that the ANAIS results will be able to rule out DAMA, all on its own, at a confidence of 3-sigma: consistent with the original goal of the experiment.
There were plenty of indirect reasons to not trust the DAMA results. They never made their data and methodology public, which meant that no one outside the collaboration ever got the opportunity to scrutinize what they did. At no point were they ever able to account for their background sufficiently. They did the very suspicious thing of resetting their noise floor every year and subtracting out an average noise on an annual basis, despite the fact that the noise floor was not constant over that year. And many, many other direct detection experiments that probed the same ranges (and much more sensitively) compared to DAMA never saw a hint of a signal there.
But this is the ultimate slam dunk. At last, we have two independent collaborations using the same tools as DAMA, except they’re performing science in an open and accessible way. Their data and methodologies are public, and they are not making the same questionable choices that the DAMA scientists have made and continue to make. In science, your results have to be replicable, and now we have robust, significant results from one of the collaborations that shows, in no uncertain terms, that no, the DAMA results were not independently replicable.
This is part of how science progresses. A novel theoretical idea can motivate an experimental search for a new signature. One of the groups that performs the search in a particular way might find a positive result, but that’s not the end of the science. We always have to look at the full suite of data, and ask ourselves the difficult questions that require us to look beyond any one experiment or result. What other tests were done, and what did those other tests show? What are the viable possibilities when everything is taken all together, both if the one disputed and/or outlier result is correct and also if it’s incorrect? Do we have reasons to suspect that anyone might be mistaken, and can we perform an independent test, either verifying or refuting the result, if so?
The 3-year ANAIS results are to be wholeheartedly celebrated, as they represent a giant step towards resolving what’s been a longstanding controversy: the dubious DAMA “detection” of dark matter. The most accurate and unambiguous independent check of a surprising annual modulation signal shows no annual modulation at all, consistent with the null hypothesis. Sometimes, finding nothing new at all is exactly what we need to help reveal the great truths of our Universe.
Starts With A Bang is written by Ethan Siegel, Ph.D., author of Beyond The Galaxy, and Treknology: The Science of Star Trek from Tricorders to Warp Drive.
