Get smarter, faster. Subscribe to our daily newsletter.
Signs of Covid-19 may be hidden in speech signals
Studying voice recordings of infected but asymptomatic people reveals potential indicators of Covid-19.
Maybe their voices are lower or have a nasally tone. Infections change the quality of our voices in various ways. But MIT Lincoln Laboratory researchers are detecting these changes in Covid-19 patients even when these changes are too subtle for people to hear or even notice in themselves.
By processing speech recordings of people infected with Covid-19 but not yet showing symptoms, these researchers found evidence of vocal biomarkers, or measurable indicators, of the disease. These biomarkers stem from disruptions the infection causes in the movement of muscles across the respiratory, laryngeal, and articulatory systems. A technology letter describing this research was recently published in IEEE Open Journal of Engineering in Medicine and Biology.
While this research is still in its early stages, the initial findings lay a framework for studying these vocal changes in greater detail. This work may also hold promise for using mobile apps to screen people for the disease, particularly those who are asymptomatic.
"I had this 'aha' moment while I was watching the news," says Thomas Quatieri, a senior staff member in the laboratory's Human Health and Performance Systems Group. Quatieri has been leading the group's research in vocal biomarkers for the past decade; their focus has been on discovering vocal biomarkers of neurological disorders such as amyotrophic lateral sclerosis (ALS) and Parkinson's disease. These diseases, and many others, change the brain's ability to turn thoughts into words, and those changes can be detected by processing speech signals.
He and his team wondered whether vocal biomarkers might also exist for Covid-19. The symptoms led them to think so. When symptoms manifest, a person typically has difficulty breathing. Inflammation in the respiratory system affects the intensity with which air is exhaled when a person talks. This air interacts with hundreds of other potentially inflamed muscles on its journey to speech production. These interactions impact the loudness, pitch, steadiness, and resonance of the voice — measurable qualities that form the basis of their biomarkers.
While watching the news, Quatieri realized there were speech samples in front of him of people who had tested positive for Covid-19. He and his colleagues combed YouTube for clips of celebrities or TV hosts who had given interviews while they were Covid-19 positive but asymptomatic. They identified five subjects. Then, they downloaded interviews of those people from before they had Covid-19, matching audio conditions as best they could.
They then used algorithms to extract features from the vocal signals in each audio sample. "These vocal features serve as proxies for the underlying movements of the speech production systems," says Tanya Talkar, a PhD candidate in the Speech and Hearing Bioscience and Technology program at Harvard University.
The signal's amplitude, or loudness, was extracted as a proxy for movement in the respiratory system. For studying movements in the larynx, they measured pitch and the steadiness of pitch, two indicators of how stable the vocal cords are. As a proxy for articulator movements — like those of the tongue, lips, jaw, and more — they extracted speech formants. Speech formants are frequency measurements that correspond to how the mouth shapes sound waves to create a sequence of phonemes (vowels and consonants) and to contribute to a certain vocal quality (nasally versus warm, for example).
They hypothesized that Covid-19 inflammation causes muscles across these systems to become overly coupled, resulting in a less complex movement. "Picture these speech subsystems as if they are the wrist and fingers of a skilled pianist; normally, the movements are independent and highly complex," Quatieri says. Now, picture if the wrist and finger movements were to become stuck together, moving as one. This coupling would force the pianist to play a much simpler tune.
The researchers looked for evidence of coupling in their features, measuring how each feature changed in relation to another in 10 millisecond increments as the subject spoke. These values were then plotted on an eigenspectrum; the shape of this eigenspectrum plot indicates the complexity of the signals. "If the eigenspace of the values forms a sphere, the signals are complex. If there is less complexity, it might look more like a flat oval," Talkar says.
In the end, they found a decreased complexity of movement in the Covid-19 interviews as compared to the pre-Covid-19 interviews. "The coupling was less prominent between larynx and articulator motion, but we're seeing a reduction in complexity between respiratory and larynx motion," Talkar says.
These preliminary results hint that biomarkers derived from vocal system coordination can indicate the presence of Covid-19. However, the researchers note that it's still early to draw conclusions, and more data are needed to validate their findings. They're working now with a publicly released dataset from Carnegie Mellon University that contains audio samples from individuals who have tested positive for Covid-19.
Beyond collecting more data to fuel this research, the team is looking at using mobile apps to implement it. A partnership is underway with Satra Ghosh at the MIT McGovern Institute for Brain Research to integrate vocal screening for Covid-19 into its VoiceUp app, which was initially developed to study the link between voice and depression. A follow-on effort could add this vocal screening into the How We Feel app. This app asks users questions about their daily health status and demographics, with the aim to use these data to pinpoint hotspots and predict the percentage of people who have the disease in different regions of the country. Asking users to also submit a daily voice memo to screen for biomarkers of Covid-19 could potentially help scientists catch on to an outbreak.
"A sensing system integrated into a mobile app could pick up on infections early, before people feel sick or, especially, for these subsets of people who don't ever feel sick or show symptoms," says Jeffrey Palmer, who leads the research group. "This is also something the U.S. Army is interested in as part of a holistic Covid-19 monitoring system." Even after a diagnosis, this sensing ability could help doctors remotely monitor their patients' progress or monitor the effects of a vaccine or drug treatment.
As the team continues their research, they plan to do more to address potential confounders that could cause inaccuracies in their results, such as different recording environments, the emotional status of the subjects, or other illnesses causing vocal changes. They're also supporting similar research. The Mass General Brigham Center for COVID Innovation has connected them to international scientists who are following the team's framework to analyze coughs.
"There are a lot of other interesting areas to look at. Here, we looked at the physiological impacts on the vocal tract. We're also looking to expand our biomarkers to consider neurophysiological impacts linked to Covid-19, like the loss of taste and smell," Quatieri says. "Those symptoms can affect speaking, too."
- Coronavirus – this is how much you touch your face | World ... ›
- Social distancing: simple math explains how well it works - Big Think ›
- Tales from the frontline: COVID-19 survivors share their stories - Big Think ›
- COVID-19 symptoms appear in this specific order - Big Think ›
- AI can detect Covid-19 infections in coughs over the phone - Big Think ›
- A speech coach's tips for communicating while wearing a mask - Big Think ›
- Coronavirus study: could Twitter have predicted outbreaks? - Big Think ›
- Covid-19 personality types: what society learns from COVID - Big Think ›
Scientists are using bioelectronic medicine to treat inflammatory diseases, an approach that capitalizes on the ancient "hardwiring" of the nervous system.
- Bioelectronic medicine is an emerging field that focuses on manipulating the nervous system to treat diseases.
- Clinical studies show that using electronic devices to stimulate the vagus nerve is effective at treating inflammatory diseases like rheumatoid arthritis.
- Although it's not yet approved by the US Food and Drug Administration, vagus nerve stimulation may also prove effective at treating other diseases like cancer, diabetes and depression.
The nervous system’s ancient reflexes<p>You accidentally place your hand on a hot stove. Almost instantaneously, your hand withdraws.</p><p>What triggered your hand to move? The answer is <em>not</em> that you consciously decided the stove was hot and you should move your hand. Rather, it was a reflex: Skin receptors on your hand sent nerve impulses to the spinal cord, which ultimately sent back motor neurons that caused your hand to move away. This all occurred before your "conscious brain" realized what happened.</p><p>Similarly, the nervous system has reflexes that protect individual cells in the body.</p><p>"The nervous system evolved because we need to respond to stimuli in the environment," said Dr. Tracey. "Neural signals don't come from the brain down first. Instead, when something happens in the environment, our peripheral nervous system senses it and sends a signal to the central nervous system, which comprises the brain and spinal cord. And then the nervous system responds to correct the problem."</p><p>So, what if scientists could "hack" into the nervous system, manipulating the electrical activity in the nervous system to control molecular processes and produce desirable outcomes? That's the chief goal of bioelectronic medicine.</p><p>"There are billions of neurons in the body that interact with almost every cell in the body, and at each of those nerve endings, molecular signals control molecular mechanisms that can be defined and mapped, and potentially put under control," Dr. Tracey said in a <a href="https://www.youtube.com/watch?v=AJH9KsMKi5M" target="_blank">TED Talk</a>.</p><p>"Many of these mechanisms are also involved in important diseases, like cancer, Alzheimer's, diabetes, hypertension and shock. It's very plausible that finding neural signals to control those mechanisms will hold promises for devices replacing some of today's medication for those diseases."</p><p>How can scientists hack the nervous system? For years, researchers in the field of bioelectronic medicine have zeroed in on the longest cranial nerve in the body: the vagus nerve.</p>
The vagus nerve<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNTYyOTM5OC9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY0NTIwNzk0NX0.UCy-3UNpomb3DQZMhyOw_SQG4ThwACXW_rMnc9mLAe8/img.jpg?width=1245&coordinates=0%2C0%2C0%2C0&height=700" id="09add" class="rm-shortcode" data-rm-shortcode-id="f38dbfbbfe470ad85a3b023dd5083557" data-rm-shortcode-name="rebelmouse-image" data-width="1245" data-height="700" />
Electrical signals, seen here in a synapse, travel along the vagus nerve to trigger an inflammatory response.
Credit: Adobe Stock via solvod<p>The vagus nerve ("vagus" meaning "wandering" in Latin) comprises two nerve branches that stretch from the brainstem down to the chest and abdomen, where nerve fibers connect to organs. Electrical signals constantly travel up and down the vagus nerve, facilitating communication between the brain and other parts of the body.</p><p>One aspect of this back-and-forth communication is inflammation. When the immune system detects injury or attack, it automatically triggers an inflammatory response, which helps heal injuries and fend off invaders. But when not deployed properly, inflammation can become excessive, exacerbating the original problem and potentially contributing to diseases.</p><p>In 2002, Dr. Tracey and his colleagues discovered that the nervous system plays a key role in monitoring and modifying inflammation. This occurs through a process called the <a href="https://www.nature.com/articles/nature01321" target="_blank" rel="noopener noreferrer">inflammatory reflex</a>. In simple terms, it works like this: When the nervous system detects inflammatory stimuli, it reflexively (and subconsciously) deploys electrical signals through the vagus nerve that trigger anti-inflammatory molecular processes.</p><p>In rodent experiments, Dr. Tracey and his colleagues observed that electrical signals traveling through the vagus nerve control TNF, a protein that, in excess, causes inflammation. These electrical signals travel through the vagus nerve to the spleen. There, electrical signals are converted to chemical signals, triggering a molecular process that ultimately makes TNF, which exacerbates conditions like rheumatoid arthritis.</p><p>The incredible chain reaction of the inflammatory reflex was observed by Dr. Tracey and his colleagues in greater detail through rodent experiments. When inflammatory stimuli are detected, the nervous system sends electrical signals that travel through the vagus nerve to the spleen. There, the electrical signals are converted to chemical signals, which trigger the spleen to create a white blood cell called a T cell, which then creates a neurotransmitter called acetylcholine. The acetylcholine interacts with macrophages, which are a specific type of white blood cell that creates TNF, a protein that, in excess, causes inflammation. At that point, the acetylcholine triggers the macrophages to stop overproducing TNF – or inflammation.</p><p>Experiments showed that when a specific part of the body is inflamed, specific fibers within the vagus nerve start firing. Dr. Tracey and his colleagues were able to map these relationships. More importantly, they were able to stimulate specific parts of the vagus nerve to "shut off" inflammation.</p><p>What's more, clinical trials show that vagus nerve stimulation not only "shuts off" inflammation, but also triggers the production of cells that promote healing.</p><p>"In animal experiments, we understand how this works," Dr. Tracey said. "And now we have clinical trials showing that the human response is what's predicted by the lab experiments. Many scientific thresholds have been crossed in the clinic and the lab. We're literally at the point of regulatory steps and stages, and then marketing and distribution before this idea takes off."<br></p>
The future of bioelectronic medicine<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNTYxMDYxMy9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYzNjQwOTExNH0.uBY1TnEs_kv9Dal7zmA_i9L7T0wnIuf9gGtdRXcNNxo/img.jpg?width=980" id="8b5b2" class="rm-shortcode" data-rm-shortcode-id="c005e615e5f23c2817483862354d2cc4" data-rm-shortcode-name="rebelmouse-image" data-width="2000" data-height="1125" />
Vagus nerve stimulation can already treat Crohn's disease and other inflammatory diseases. In the future, it may also be used to treat cancer, diabetes, and depression.
Credit: Adobe Stock via Maridav<p>Vagus nerve stimulation is currently awaiting approval by the US Food and Drug Administration, but so far, it's proven safe and effective in clinical trials on humans. Dr. Tracey said vagus nerve stimulation could become a common treatment for a wide range of diseases, including cancer, Alzheimer's, diabetes, hypertension, shock, depression and diabetes.</p><p>"To the extent that inflammation is the problem in the disease, then stopping inflammation or suppressing the inflammation with vagus nerve stimulation or bioelectronic approaches will be beneficial and therapeutic," he said.</p><p>Receiving vagus nerve stimulation would require having an electronic device, about the size of lima bean, surgically implanted in your neck during a 30-minute procedure. A couple of weeks later, you'd visit, say, your rheumatologist, who would activate the device and determine the right dosage. The stimulation would take a few minutes each day, and it'd likely be unnoticeable.</p><p>But the most revolutionary aspect of bioelectronic medicine, according to Dr. Tracey, is that approaches like vagus nerve stimulation wouldn't come with harmful and potentially deadly side effects, as many pharmaceutical drugs currently do.</p><p>"A device on a nerve is not going to have systemic side effects on the body like taking a steroid does," Dr. Tracey said. "It's a powerful concept that, frankly, scientists are quite accepting of—it's actually quite amazing. But the idea of adopting this into practice is going to take another 10 or 20 years, because it's hard for physicians, who've spent their lives writing prescriptions for pills or injections, that a computer chip can replace the drug."</p><p>But patients could also play a role in advancing bioelectronic medicine.</p><p>"There's a huge demand in this patient cohort for something better than they're taking now," Dr. Tracey said. "Patients don't want to take a drug with a black-box warning, costs $100,000 a year and works half the time."</p><p>Michael Dowling, president and CEO of Northwell Health, elaborated:</p><p>"Why would patients pursue a drug regimen when they could opt for a few electronic pulses? Is it possible that treatments like this, pulses through electronic devices, could replace some drugs in the coming years as preferred treatments? Tracey believes it is, and that is perhaps why the pharmaceutical industry closely follows his work."</p><p>Over the long term, bioelectronic approaches are unlikely to completely replace pharmaceutical drugs, but they could replace many, or at least be used as supplemental treatments.</p><p>Dr. Tracey is optimistic about the future of the field.</p><p>"It's going to spawn a huge new industry that will rival the pharmaceutical industry in the next 50 years," he said. "This is no longer just a startup industry. [...] It's going to be very interesting to see the explosive growth that's going to occur."</p>
Japan looks to replace China as the primary source of critical metals
- Enough rare earth minerals have been found off Japan to last centuries
- Rare earths are important materials for green technology, as well as medicine and manufacturing
- Where would we be without all of our rare-earth magnets?
What are the rare earth elements?<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8xOTA2MTM0Ni9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYzODExMjMyMn0.owchAgxSBwji5IofgwKtueKSbHNyjPfT7hTJrHpTi98/img.jpg?width=980" id="fd315" class="rm-shortcode" data-rm-shortcode-id="d8ed70e3d0b67b9cbe78414ffd02c43e" data-rm-shortcode-name="rebelmouse-image" />
(julie deshaies/Shutterstock)<p>The rare earth metals can be mostly found in the second row from the bottom in the Table of Elements. According to the <a href="http://www.rareearthtechalliance.com/What-are-Rare-Earths" target="_blank"><u>Rare Earth Technology Alliance</u></a>, due to the "unique magnetic, luminescent, and electrochemical properties, these elements help make many technologies perform with reduced weight, reduced emissions, and energy consumption; or give them greater efficiency, performance, miniaturization, speed, durability, and thermal stability."</p><p>In order of atomic number, the rare earths are:</p> <ul> <li>Scandium or Sc (21) — This is used in TVs and energy-saving lamps.</li> <li>Yttrium or Y (39) — Yttrium is important in the medical world, used in cancer drugs, rheumatoid arthritis medications, and surgical supplies. It's also used in superconductors and lasers.</li> <li>Lanthanum or La (57) — Lanthanum finds use in camera/telescope lenses, special optical glasses, and infrared absorbing glass.</li> <li>Cerium or Ce (58) — Cerium is found in catalytic converters, and is used for precision glass-polishing. It's also found in alloys, magnets, electrodes, and carbon-arc lighting. </li> <li>Praseodymium or Pr (59) — This is used in magnets and high-strength metals.</li> <li>Neodymium or Nd (60) — Many of the magnets around you have neodymium in them: speakers and headphones, microphones, computer storage, and magnets in your car. It's also found in high-powered industrial and military lasers. The mineral is especially important for green tech. Each <a href="https://www.reuters.com/article/us-mining-toyota/as-hybrid-cars-gobble-rare-metals-shortage-looms-idUSTRE57U02B20090831" target="_blank"><u>Prius</u></a> motor, for example, requires 2.2 lbs of neodymium, and its battery another 22-33 lbs. <a href="https://pubs.usgs.gov/sir/2011/5036/sir2011-5036.pdf" target="_blank"><u>Wind turbine batteries</u></a> require 450 lbs of neodymium per watt. </li> <li>Promethium or Pm (61) — This is used in pacemakers, watches, and research.</li> <li>Samarium or Sm (62) — This mineral is used in magnets in addition to intravenous cancer radiation treatments and nuclear reactor control rods.</li> <li>Europium or Eu (63) — Europium is used in color displays and compact fluorescent light bulbs.</li> <li>Gadolinium or Gd (64) — It's important for nuclear reactor shielding, cancer radiation treatments, as well as x-ray and bone-density diagnostic equipment.</li> <li>Terbium or Tb (65) — Terbium has similar uses to Europium, though it's also soft and thus possesses unique shaping capabilities .</li> <li>Dysprosium or Dy (66) — This is added to other rare-earth magnets to help them work at high temperatures. It's used for computer storage, in nuclear reactors, and in energy-efficient vehicles.</li> <li>Holmium or Ho (67) — Holmium is used in nuclear control rods, microwaves, and magnetic flux concentrators.</li> <li>Erbium or Er (68) — This is used in fiber-optic communication networks and lasers.</li> <li>Thulium or Tm (69) — Thulium is another laser rare earth.</li> <li>Ytterbium or Yb (70) — This mineral is used in cancer treatments, in stainless steel, and in seismic detection devices.</li> <li>Lutetium or Lu (71) — Lutetium can target certain cancers, and is used in petroleum refining and positron emission tomography.</li></ul>
Where Japan found is rare earths<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8xOTA2MTM0OC9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY1MTA0NzUxNn0.N3t_iKf6lnnoJ6yVUtl8-wNZICEG2ZxyPzm9ZdE99ks/img.jpg?width=980" id="021b7" class="rm-shortcode" data-rm-shortcode-id="d9dd843fde547a0b69f8798aca18a706" data-rm-shortcode-name="rebelmouse-image" />
Minimatori Torishima Island
(Chief Master Sergeant Don Sutherland, U.S. Air Force)<p>Japan located the rare earths about 1,850 kilometers off the shore of <a href="https://en.wikipedia.org/wiki/Minami-Tori-shima" target="_blank"><u>Minamitori Island</u></a>. Engineers located the minerals in 10-meter-deep cores taken from sea floor sediment. Mapping the cores revealed and area of approximately 2,500 square kilometers containing rare earths.</p><p>Japan's engineers estimate there's 16 million tons of rare earths down there. That's <a href="https://minerals.usgs.gov/minerals/pubs/historical-statistics/ds140-raree.xlsx" target="_blank"><u>five times</u></a> the amount of the rare earth elements ever mined since 1900. According to <a href="https://www.businessinsider.com.au/rare-earth-minerals-found-in-japan-2018-4?r=US&IR=T" target="_blank"><u>Business Insider</u></a>, there's "enough yttrium to meet the global demand for 780 years, dysprosium for 730 years, europium for 620 years, and terbium for 420 years."</p><p>The bad news, of course, is that Japan has to figure out how to extract the minerals from 6-12 feet under the seabed four miles beneath the ocean surface — that's the <a href="https://www.nature.com/articles/s41598-018-23948-5" target="_blank"><u>next step</u></a> for the country's engineers. The good news is that the location sits squarely within Japan's Exclusive Economic Zone, so their rights to the lucrative discovery will be undisputed.</p>
A physicist creates an AI algorithm that predicts natural events and may prove the simulation hypothesis.
- Princeton physicist Hong Qin creates an AI algorithm that can predict planetary orbits.
- The scientist partially based his work on the hypothesis which believes reality is a simulation.
- The algorithm is being adapted to predict behavior of plasma and can be used on other natural phenomena.
Physicist Hong Qin with images of planetary orbits and computer code.
Credit: Elle Starkman
Are we living in a simulation? | Bill Nye, Joscha Bach, Donald Hoffman | Big Think<span style="display:block;position:relative;padding-top:56.25%;" class="rm-shortcode" data-rm-shortcode-id="4dbe18924f2f42eef5669e67f405b52e"><iframe type="lazy-iframe" data-runner-src="https://www.youtube.com/embed/KDcNVZjaNSU?rel=0" width="100%" height="auto" frameborder="0" scrolling="no" style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe></span>
How different people react to threats of violence.
Beyond making up 70% of the world's health workers, women researchers have been at the cutting edge of coronavirus research.