Uh-Oh. Is Our Faith in fMRI Brain Studies Misplaced?

A new study suggests that 70% of the time scientists see brain activity on an fMRI, they aren’t.

It seems like nearly every time there’s a new discovery made in neuroscience, it’s because researchers have been able to associate some process with a physical area of the brain for the first time. This is typically the result of observing activity in the area using fMRI, or “functional magnetic resonance imaging” technology. fMRI has been a critical tool for neuroscience, psychiatry, and psychology since it was introduced at a Society for Magnetic Resonance in Medicine meeting in 1991. However, a paper published last May in Proceedings of the National Academy of Sciences has raised serious questions about the way fMRI data is interpreted, and by extension, the validity of countless conclusions that have been drawn from these interpretations over the last 25 years.


At issue are the baseline assumptions upon which researchers depend to distinguish the brain’s background data — that is, “complex spatiotemporal noise” — from actual activity. The report says, “Despite the popularity of fMRI as a tool for studying brain function, the statistical methods used have rarely been validated using real data.” This is because not enough testing has been done to really know what kind of background data is produced when a brain is at rest. The brain of a patient in an awake MRI scanner is not really at rest, so it’s difficult to generate an image of what that would look like.

Not the most relaxing experience (LEON BROCARD)

As a result, what fMRI software shows as brain activity is data that stands out against a simulated model of what inactivity looks like, not the real thing.

The authors of the paper, researchers Anders Eklund, Thomas Nichols, and Hans Knutsson, used data from a resting-state control group of 499 actual patients to conduct 2,880,000 test analyses. They used common fMRI software packages (SPM, FSL, AFNI), expecting to find the programs’ simulated models would produce a false-positive error rate of no more than 5%. Instead, the rate was an astronomical 70%! If they’re right about this, this means that 70% of the time fMRI researchers think they’re seeing an area of the brain do something, or “light up,” it isn’t!

Wow. To say that the study’s conclusion is earth-shaking is no exaggeration, and we’ll just have to stay tuned for the fallout. But just how much of what we’ve been told about the brain will be ultimately just thrown out?

Skepticism about fMRI isn’t new. In an often-cited case from 2009 an fMRI appeared to show brain activity in a dead salmon.

(TIMOTHY KNEPP)

The American Journal of Psychiatry in particular is concerned about psychiatry relying too much on fMRI findings, issuing a report to that effect in January 2016. The report includes the point, "That conventional MRI does not allow us to make firm inferences about the primary biology of mental disorders and that we need to acknowledge this as a starting point in realizing the full value of MRI studies in psychiatry.” And Stanford’s Center for Reproducible Knowledge recently received a grant to develop better verification tools and methods for sharing data more effectively.

So what are we, as people who follow discoveries in science, and neuroscience specifically, supposed to do with this stunning revelation? Really, it’s just business in science as usual: It’s always a work in progress. New tools will always arise that cast doubts on data from previous devices. Do we ever really arrive at final knowledge? Of course not. The fun is in taking the ride.

​There are two kinds of failure – but only one is honorable

Malcolm Gladwell teaches "Get over yourself and get to work" for Big Think Edge.

Big Think Edge
  • Learn to recognize failure and know the big difference between panicking and choking.
  • At Big Think Edge, Malcolm Gladwell teaches how to check your inner critic and get clear on what failure is.
  • Subscribe to Big Think Edge before we launch on March 30 to get 20% off monthly and annual memberships.
Keep reading Show less

For a long time, the West shaped the world. That time is over.

The 21st century is experiencing an Asianization of politics, business, and culture.

Videos
  • Our theories about the world, even about history or the geopolitics of the present, tend to be shaped by Anglo perspectives of the Western industrial democracies, particularly those in the United States and the United Kingdom.
  • The West, however, is not united. Canada, for instance, acts in many ways that are not in line with American or British policies, particularly in regard to populism. Even if it were united, though, it would not represent most of the world's population.
  • European ideas, such as parliamentary democracy and civil service, spread across the world in the 19th century. In the 20th century, American values such as entrepreneurialism went global. In the 21st century, however, what we're seeing now is an Asianization — an Asian confidence that they can determine their own political systems, their own models, and adapt to their own circumstances.
Keep reading Show less

Vikings unwittingly made their swords stronger by trying to imbue them with spirits

They didn't know it, but the rituals of Iron Age Scandinavians turned their iron into steel.

Shutterstock
Culture & Religion
  • Iron Age Scandinavians only had access to poor quality iron, which put them at a tactical disadvantage against their neighbors.
  • To strengthen their swords, smiths used the bones of their dead ancestors and animals, hoping to transfer the spirit into their blades.
  • They couldn't have known that in so doing, they actually were forging a rudimentary form of steel.
Keep reading Show less

Why the ocean you know and love won’t exist in 50 years

Can sensitive coral reefs survive another human generation?

Videos
  • Coral reefs may not be able to survive another human decade because of the environmental stress we have placed on them, says author David Wallace-Wells. He posits that without meaningful changes to policies, the trend of them dying out, even in light of recent advances, will continue.
  • The World Wildlife Fund says that 60 percent of all vertebrate mammals have died since just 1970. On top of this, recent studies suggest that insect populations may have fallen by as much as 75 percent over the last few decades.
  • If it were not for our oceans, the planet would probably be already several degrees warmer than it is today due to the emissions we've expelled into the atmosphere.
Keep reading Show less