New AI improves itself through Darwinian-style evolution
AutoML-Zero is a proof-of-concept project that suggests the future of machine learning may be machine-created algorithms.
- Automatic machine learning is a fast-developing branch of deep learning.
- It seeks to vastly reduce the amount of human input and energy needed to apply machine learning to real-world problems.
- AutoML-Zero, developed by scientists at Google, serves as a simple proof-of-concept that shows how this kind of technology might someday be scaled up and applied to more complex problems.
Machine learning has fundamentally changed how we engage with technology. Today, it's able to curate social media feeds, recognize complex images, drive cars down the interstate, and even diagnose medical conditions, to name a few tasks.
But while machine learning technology can do some things automatically, it still requires a lot of input from human engineers to set it up, and point it in the right direction. Inevitably, that means human biases and limitations are baked into the technology.
So, what if scientists could minimize their influence on the process by creating a system that generates its own machine-learning algorithms? Could it discover new solutions that humans never considered?
To answer these questions, a team of computer scientists at Google developed a project called AutoML-Zero, which is described in a preprint paper published on arXiv.
"Human-designed components bias the search results in favor of human-designed algorithms, possibly reducing the innovation potential of AutoML," the paper states. "Innovation is also limited by having fewer options: you cannot discover what you cannot search for."
Automatic machine learning (AutoML) is a fast-growing area of deep learning. In simple terms, AutoML seeks to automate the end-to-end process of applying machine learning to real-world problems. Unlike other machine-learning techniques, AutoML requires relatively little human effort, which means companies might soon be able to utilize it without having to hire a team of data scientists.
AutoML-Zero is unique because it uses simple mathematical concepts to generate algorithms "from scratch," as the paper states. Then, it selects the best ones, and mutates them through a process that's similar to Darwinian evolution.
AutoML-Zero first randomly generates 100 candidate algorithms, each of which then performs a task, like recognizing an image. The performance of these algorithms is compared to hand-designed algorithms. AutoML-Zero then selects the top-performing algorithm to be the "parent."
"This parent is then copied and mutated to produce a child algorithm that is added to the population, while the oldest algorithm in the population is removed," the paper states.
The system can create thousands of populations at once, which are mutated through random procedures. Over enough cycles, these self-generated algorithms get better at performing tasks.
"The nice thing about this kind of AI is that it can be left to its own devices without any pre-defined parameters, and is able to plug away 24/7 working on developing new algorithms," Ray Walsh, a computer expert and digital researcher at ProPrivacy, told Newsweek.
If computer scientists can scale up this kind of automated machine-learning to complete more complex tasks, it could usher in a new era of machine learning where systems are designed by machines instead of humans. This would likely make it much cheaper to reap the benefits of deep learning, while also leading to novel solutions to real-world problems.
Still, the recent paper was a small-scale proof of concept, and the researchers note that much more research is needed.
"Starting from empty component functions and using only basic mathematical operations, we evolved linear regressors, neural networks, gradient descent... multiplicative interactions. These results are promising, but there is still much work to be done," the scientists' preprint paper noted.
Young people could even end up less anxiety-ridden, thanks to newfound confidence
- The coronavirus pandemic may have a silver lining: It shows how insanely resourceful kids really are.
- Let Grow, a non-profit promoting independence as a critical part of childhood, ran an "Independence Challenge" essay contest for kids. Here are a few of the amazing essays that came in.
- Download Let Grow's free Independence Kit with ideas for kids.
Researchers in Mexico discover the longest underwater cave system in the world that's full of invaluable artifacts.
New research establishes an unexpected connection.
- A study provides further confirmation that a prolonged lack of sleep can result in early mortality.
- Surprisingly, the direct cause seems to be a buildup of Reactive Oxygen Species in the gut produced by sleeplessness.
- When the buildup is neutralized, a normal lifespan is restored.
We don't have to tell you what it feels like when you don't get enough sleep. A night or two of that can be miserable; long-term sleeplessness is out-and-out debilitating. Though we know from personal experience that we need sleep — our cognitive, metabolic, cardiovascular, and immune functioning depend on it — a lack of it does more than just make you feel like you want to die. It can actually kill you, according to study of rats published in 1989. But why?
A new study answers that question, and in an unexpected way. It appears that the sleeplessness/death connection has nothing to do with the brain or nervous system as many have assumed — it happens in your gut. Equally amazing, the study's authors were able to reverse the ill effects with antioxidants.
The study, from researchers at Harvard Medical School (HMS), is published in the journal Cell.
An unexpected culprit
The new research examines the mechanisms at play in sleep-deprived fruit flies and in mice — long-term sleep-deprivation experiments with humans are considered ethically iffy.
What the scientists found is that death from sleep deprivation is always preceded by a buildup of Reactive Oxygen Species (ROS) in the gut. These are not, as their name implies, living organisms. ROS are reactive molecules that are part of the immune system's response to invading microbes, and recent research suggests they're paradoxically key players in normal cell signal transduction and cell cycling as well. However, having an excess of ROS leads to oxidative stress, which is linked to "macromolecular damage and is implicated in various disease states such as atherosclerosis, diabetes, cancer, neurodegeneration, and aging." To prevent this, cellular defenses typically maintain a balance between ROS production and removal.
"We took an unbiased approach and searched throughout the body for indicators of damage from sleep deprivation," says senior study author Dragana Rogulja, admitting, "We were surprised to find it was the gut that plays a key role in causing death." The accumulation occurred in both sleep-deprived fruit flies and mice.
"Even more surprising," Rogulja recalls, "we found that premature death could be prevented. Each morning, we would all gather around to look at the flies, with disbelief to be honest. What we saw is that every time we could neutralize ROS in the gut, we could rescue the flies." Fruit flies given any of 11 antioxidant compounds — including melatonin, lipoic acid and NAD — that neutralize ROS buildups remained active and lived a normal length of time in spite of sleep deprivation. (The researchers note that these antioxidants did not extend the lifespans of non-sleep deprived control subjects.)
Image source: Tomasz Klejdysz/Shutterstock/Big Think
The study's tests were managed by co-first authors Alexandra Vaccaro and Yosef Kaplan Dor, both research fellows at HMS.
You may wonder how you compel a fruit fly to sleep, or for that matter, how you keep one awake. The researchers ascertained that fruit flies doze off in response to being shaken, and thus were the control subjects induced to snooze in their individual, warmed tubes. Each subject occupied its own 29 °C (84F) tube.
For their sleepless cohort, fruit flies were genetically manipulated to express a heat-sensitive protein in specific neurons. These neurons are known to suppress sleep, and did so — the fruit flies' activity levels, or lack thereof, were tracked using infrared beams.
Starting at Day 10 of sleep deprivation, fruit flies began dying, with all of them dead by Day 20. Control flies lived up to 40 days.
The scientists sought out markers that would indicate cell damage in their sleepless subjects. They saw no difference in brain tissue and elsewhere between the well-rested and sleep-deprived fruit flies, with the exception of one fruit fly.
However, in the guts of sleep-deprived fruit flies was a massive accumulation of ROS, which peaked around Day 10. Says Vaccaro, "We found that sleep-deprived flies were dying at the same pace, every time, and when we looked at markers of cell damage and death, the one tissue that really stood out was the gut." She adds, "I remember when we did the first experiment, you could immediately tell under the microscope that there was a striking difference. That almost never happens in lab research."
The experiments were repeated with mice who were gently kept awake for five days. Again, ROS built up over time in their small and large intestines but nowhere else.
As noted above, the administering of antioxidants alleviated the effect of the ROS buildup. In addition, flies that were modified to overproduce gut antioxidant enzymes were found to be immune to the damaging effects of sleep deprivation.
The research leaves some important questions unanswered. Says Kaplan Dor, "We still don't know why sleep loss causes ROS accumulation in the gut, and why this is lethal." He hypothesizes, "Sleep deprivation could directly affect the gut, but the trigger may also originate in the brain. Similarly, death could be due to damage in the gut or because high levels of ROS have systemic effects, or some combination of these."
The HMS researchers are now investigating the chemical pathways by which sleep-deprivation triggers the ROS buildup, and the means by which the ROS wreak cell havoc.
"We need to understand the biology of how sleep deprivation damages the body so that we can find ways to prevent this harm," says Rogulja.
Referring to the value of this study to humans, she notes,"So many of us are chronically sleep deprived. Even if we know staying up late every night is bad, we still do it. We believe we've identified a central issue that, when eliminated, allows for survival without sleep, at least in fruit flies."
We must rethink the "chemical imbalance" theory of mental health.
- A new review found that withdrawal symptoms from antidepressants and antipsychotics can last for over a year.
- Side effects from SSRIs, SNRIs, and antipsychotics last longer than benzodiazepines like Valium or Prozac.
- The global antidepressant market is expected to reach $28.6 billion this year.