Secretive agency uses AI, human 'forecasters' to predict the future

A U.S. government intelligence agency develops cutting-edge tech to predict future events.

© Sergey Nivens - stock.adobe.com
  • The Intelligence Advanced Research Projects Activity (IARPA), a research arm of the U.S. government intelligence community, is focused on predicting the future.
  • The organization uses teams of human non-experts and AI machine learning to forecast future events.
  • IARPA also conducts advanced research in numerous other fields, funding rotating programs.

As far as secretive government projects go, the objectives of IARPA may be the riskiest and most far-reaching. With its mission to foster "high-risk, high-payoff" programs, this research arm of the U.S. intelligence community literally tries to predict the future. Staffed by spies and Ph.D.s, this organization aims to provide decision makers with real, accurate predictions of geopolitical events, using artificial intelligence and human "forecasters."

IARPA, which stands for Intelligence Advanced Research Projects Activity, was founded in 2006 as part of the Office of the Director of National Intelligence. Some of the projects that it has funded focused on advancements in quantum computing, cryogenic computing, face recognition, universal language translators, and other initiatives that would fit well in a Hollywood action movie plot. But perhaps its main goal is to produce "anticipatory intelligence." It's a spy agency, after all.

"Minority report" pre-cog

"Minority report" pre-cog

Dreamworks/20th Century Fox

In the interest of national security, IARPA wants to identify major world events before they happen, looking for terrorists, hackers or any perceived enemies of the United States. Wouldn't you rather stop a crime before it happens?

Of course, that's when we get into tricky political and sci-fi territory. Much of the research done by IARPA is actually out in the open, utilizing the public and experts in advancing technologies. It is available for "open solicitations," forecasting tournaments, and has prize challenges for the public. You can pretty much send your idea in right now. But what happens to the R&D once it leaves the lab is, of course, often for only the NSA and the CIA to know.

The National Security Agency expert James Bamford wrote that the agency is ultimately looking to create a system where huge amounts of data about people's lives would be mined in real-time, for the purpose of preventing actions detrimental to the nation. In his article for the Pittsburgh Post-Gazette, Bamford wrote that IARPA's goal is to create very powerful automated computer systems, managed through artificial intelligence, which would be "capable of cataloging the lives of everyone everywhere, 24/ 7." Such programs would be able to instantaneously access data streams belonging to citizens, whether from social media or anywhere else. As Bamford writes, being able to analyze "every Facebook post, tweet and YouTube video; every tollbooth tag number; every GPS download, web search and news feed; every street camera video; every restaurant reservation on Open Table — largely eliminates surprise from the intelligence equation."

Of course, one would suspect much of this is going on already. IARPA's Mercury program, for example, concentrates on data mining millions of private overseas communications that are gathered by the National Security Agency. While it can certainly be argued that such a program is a national security necessity, working to spot terrorists and elements that can lead to social unrest, the potential for misuse and infringement on privacy rights has alerted observers.


A fascinating recent project funded by IARPA is called SAGE, which stands for Synergistic Anticipation of Geopolitical Events. As you may expect from such a lofty title, the researchers involved in this endeavor are looking to predict the future. This project is aimed at utilizing non-experts – humans who would use AI machine learning to make qualified statements about what would happen.

Led by Aram Galstyan, director of the Artificial Intelligence Division at the USC Viterbi Information Sciences Institute (ISI), the project has been successful in making concrete predictions, like knowing when North Korea would launch its missile tests. SAGE works by utilizing large sets of human non-expert predictors, pooling their powers by working together, making them "more accurate and faster than a single human subject expert," as explains a USC press release. However, the information these humans or "forecasters" use to make predictions is gathered through a variety of machine learning technologies.

The topics looked at by the predictors include such questions as "Will any G7 nation engage in an acknowledged national military attack against Syria [by a given date]?" They may also want to figure out exactly how much oil Venezuela might produce in a specific month.

Leaders among the forecasters, or those who make the most accurate predictions, are ranked and highlighted with badges.

This AI-assisted crowd-sourced Nostradamus has worked out quite well, according to Fred Morstatter, a USC computer scientist. "We believe that's the case because the numbers we're seeing indicate we are outpacing a system that uses only humans," he remarked.

SAGE's hybrid model functions by offering humans information derived by the machines in charts that show trends, along with specific predictions by the AI. "SAGE works because humans have one side of the coin, and machines have the other side," said Morstatter. And on yet another side you would have the National Intelligence apparatus.

Do you have a good idea for future-oriented national security research? You can actually apply to be a IARPA program manager. Current managers, who rotate every 3 to 5 years, are working on a vast variety of fields, including forecasting, linguistics, underwater technology, aerospace propulsion, atomic physics, artificial intelligence, biometrics, neuroscience, and optics. Check out the list of existing programs.

Yug, age 7, and Alia, age 10, both entered Let Grow's "Independence Challenge" essay contest.

Photos: Courtesy of Let Grow
Sponsored by Charles Koch Foundation
  • The coronavirus pandemic may have a silver lining: It shows how insanely resourceful kids really are.
  • Let Grow, a non-profit promoting independence as a critical part of childhood, ran an "Independence Challenge" essay contest for kids. Here are a few of the amazing essays that came in.
  • Download Let Grow's free Independence Kit with ideas for kids.
Keep reading Show less

Divers discover world's largest underwater cave system filled with Mayan mysteries

Researchers in Mexico discover the longest underwater cave system in the world that's full of invaluable artifacts. 

Divers of the GAM project. Credit: Herbert Meyrl.
Technology & Innovation

Keep reading Show less

The surprise reason sleep-deprivation kills lies in the gut

New research establishes an unexpected connection.

Reactive oxygen species (ROS) accumulate in the gut of sleep-deprived fruit flies, one (left), seven (center) and ten (right) days without sleep.

Image source: Vaccaro et al, 2020/Harvard Medical School
Surprising Science
  • A study provides further confirmation that a prolonged lack of sleep can result in early mortality.
  • Surprisingly, the direct cause seems to be a buildup of Reactive Oxygen Species in the gut produced by sleeplessness.
  • When the buildup is neutralized, a normal lifespan is restored.

We don't have to tell you what it feels like when you don't get enough sleep. A night or two of that can be miserable; long-term sleeplessness is out-and-out debilitating. Though we know from personal experience that we need sleep — our cognitive, metabolic, cardiovascular, and immune functioning depend on it — a lack of it does more than just make you feel like you want to die. It can actually kill you, according to study of rats published in 1989. But why?

A new study answers that question, and in an unexpected way. It appears that the sleeplessness/death connection has nothing to do with the brain or nervous system as many have assumed — it happens in your gut. Equally amazing, the study's authors were able to reverse the ill effects with antioxidants.

The study, from researchers at Harvard Medical School (HMS), is published in the journal Cell.

An unexpected culprit

The new research examines the mechanisms at play in sleep-deprived fruit flies and in mice — long-term sleep-deprivation experiments with humans are considered ethically iffy.

What the scientists found is that death from sleep deprivation is always preceded by a buildup of Reactive Oxygen Species (ROS) in the gut. These are not, as their name implies, living organisms. ROS are reactive molecules that are part of the immune system's response to invading microbes, and recent research suggests they're paradoxically key players in normal cell signal transduction and cell cycling as well. However, having an excess of ROS leads to oxidative stress, which is linked to "macromolecular damage and is implicated in various disease states such as atherosclerosis, diabetes, cancer, neurodegeneration, and aging." To prevent this, cellular defenses typically maintain a balance between ROS production and removal.

"We took an unbiased approach and searched throughout the body for indicators of damage from sleep deprivation," says senior study author Dragana Rogulja, admitting, "We were surprised to find it was the gut that plays a key role in causing death." The accumulation occurred in both sleep-deprived fruit flies and mice.

"Even more surprising," Rogulja recalls, "we found that premature death could be prevented. Each morning, we would all gather around to look at the flies, with disbelief to be honest. What we saw is that every time we could neutralize ROS in the gut, we could rescue the flies." Fruit flies given any of 11 antioxidant compounds — including melatonin, lipoic acid and NAD — that neutralize ROS buildups remained active and lived a normal length of time in spite of sleep deprivation. (The researchers note that these antioxidants did not extend the lifespans of non-sleep deprived control subjects.)

fly with thought bubble that says "What? I'm awake!"

Image source: Tomasz Klejdysz/Shutterstock/Big Think

The experiments

The study's tests were managed by co-first authors Alexandra Vaccaro and Yosef Kaplan Dor, both research fellows at HMS.

You may wonder how you compel a fruit fly to sleep, or for that matter, how you keep one awake. The researchers ascertained that fruit flies doze off in response to being shaken, and thus were the control subjects induced to snooze in their individual, warmed tubes. Each subject occupied its own 29 °C (84F) tube.

For their sleepless cohort, fruit flies were genetically manipulated to express a heat-sensitive protein in specific neurons. These neurons are known to suppress sleep, and did so — the fruit flies' activity levels, or lack thereof, were tracked using infrared beams.

Starting at Day 10 of sleep deprivation, fruit flies began dying, with all of them dead by Day 20. Control flies lived up to 40 days.

The scientists sought out markers that would indicate cell damage in their sleepless subjects. They saw no difference in brain tissue and elsewhere between the well-rested and sleep-deprived fruit flies, with the exception of one fruit fly.

However, in the guts of sleep-deprived fruit flies was a massive accumulation of ROS, which peaked around Day 10. Says Vaccaro, "We found that sleep-deprived flies were dying at the same pace, every time, and when we looked at markers of cell damage and death, the one tissue that really stood out was the gut." She adds, "I remember when we did the first experiment, you could immediately tell under the microscope that there was a striking difference. That almost never happens in lab research."

The experiments were repeated with mice who were gently kept awake for five days. Again, ROS built up over time in their small and large intestines but nowhere else.

As noted above, the administering of antioxidants alleviated the effect of the ROS buildup. In addition, flies that were modified to overproduce gut antioxidant enzymes were found to be immune to the damaging effects of sleep deprivation.

The research leaves some important questions unanswered. Says Kaplan Dor, "We still don't know why sleep loss causes ROS accumulation in the gut, and why this is lethal." He hypothesizes, "Sleep deprivation could directly affect the gut, but the trigger may also originate in the brain. Similarly, death could be due to damage in the gut or because high levels of ROS have systemic effects, or some combination of these."

The HMS researchers are now investigating the chemical pathways by which sleep-deprivation triggers the ROS buildup, and the means by which the ROS wreak cell havoc.

"We need to understand the biology of how sleep deprivation damages the body so that we can find ways to prevent this harm," says Rogulja.

Referring to the value of this study to humans, she notes,"So many of us are chronically sleep deprived. Even if we know staying up late every night is bad, we still do it. We believe we've identified a central issue that, when eliminated, allows for survival without sleep, at least in fruit flies."

Withdrawal symptoms from antidepressants can last over a year, new study finds

We must rethink the "chemical imbalance" theory of mental health.

Bottles of antidepressant pills named (L-R) Wellbutrin, Paxil, Fluoxetine and Lexapro are shown March 23, 2004 photographed in Miami, Florida.

Photo Illustration by Joe Raedle/Getty Images
Surprising Science
  • A new review found that withdrawal symptoms from antidepressants and antipsychotics can last for over a year.
  • Side effects from SSRIs, SNRIs, and antipsychotics last longer than benzodiazepines like Valium or Prozac.
  • The global antidepressant market is expected to reach $28.6 billion this year.
Keep reading Show less
Scroll down to load more…