Signs of Covid-19 may be hidden in speech signals
Studying voice recordings of infected but asymptomatic people reveals potential indicators of Covid-19.
Maybe their voices are lower or have a nasally tone. Infections change the quality of our voices in various ways. But MIT Lincoln Laboratory researchers are detecting these changes in Covid-19 patients even when these changes are too subtle for people to hear or even notice in themselves.
By processing speech recordings of people infected with Covid-19 but not yet showing symptoms, these researchers found evidence of vocal biomarkers, or measurable indicators, of the disease. These biomarkers stem from disruptions the infection causes in the movement of muscles across the respiratory, laryngeal, and articulatory systems. A technology letter describing this research was recently published in IEEE Open Journal of Engineering in Medicine and Biology.
While this research is still in its early stages, the initial findings lay a framework for studying these vocal changes in greater detail. This work may also hold promise for using mobile apps to screen people for the disease, particularly those who are asymptomatic.
"I had this 'aha' moment while I was watching the news," says Thomas Quatieri, a senior staff member in the laboratory's Human Health and Performance Systems Group. Quatieri has been leading the group's research in vocal biomarkers for the past decade; their focus has been on discovering vocal biomarkers of neurological disorders such as amyotrophic lateral sclerosis (ALS) and Parkinson's disease. These diseases, and many others, change the brain's ability to turn thoughts into words, and those changes can be detected by processing speech signals.
He and his team wondered whether vocal biomarkers might also exist for Covid-19. The symptoms led them to think so. When symptoms manifest, a person typically has difficulty breathing. Inflammation in the respiratory system affects the intensity with which air is exhaled when a person talks. This air interacts with hundreds of other potentially inflamed muscles on its journey to speech production. These interactions impact the loudness, pitch, steadiness, and resonance of the voice — measurable qualities that form the basis of their biomarkers.
While watching the news, Quatieri realized there were speech samples in front of him of people who had tested positive for Covid-19. He and his colleagues combed YouTube for clips of celebrities or TV hosts who had given interviews while they were Covid-19 positive but asymptomatic. They identified five subjects. Then, they downloaded interviews of those people from before they had Covid-19, matching audio conditions as best they could.
They then used algorithms to extract features from the vocal signals in each audio sample. "These vocal features serve as proxies for the underlying movements of the speech production systems," says Tanya Talkar, a PhD candidate in the Speech and Hearing Bioscience and Technology program at Harvard University.
The signal's amplitude, or loudness, was extracted as a proxy for movement in the respiratory system. For studying movements in the larynx, they measured pitch and the steadiness of pitch, two indicators of how stable the vocal cords are. As a proxy for articulator movements — like those of the tongue, lips, jaw, and more — they extracted speech formants. Speech formants are frequency measurements that correspond to how the mouth shapes sound waves to create a sequence of phonemes (vowels and consonants) and to contribute to a certain vocal quality (nasally versus warm, for example).
They hypothesized that Covid-19 inflammation causes muscles across these systems to become overly coupled, resulting in a less complex movement. "Picture these speech subsystems as if they are the wrist and fingers of a skilled pianist; normally, the movements are independent and highly complex," Quatieri says. Now, picture if the wrist and finger movements were to become stuck together, moving as one. This coupling would force the pianist to play a much simpler tune.
The researchers looked for evidence of coupling in their features, measuring how each feature changed in relation to another in 10 millisecond increments as the subject spoke. These values were then plotted on an eigenspectrum; the shape of this eigenspectrum plot indicates the complexity of the signals. "If the eigenspace of the values forms a sphere, the signals are complex. If there is less complexity, it might look more like a flat oval," Talkar says.
In the end, they found a decreased complexity of movement in the Covid-19 interviews as compared to the pre-Covid-19 interviews. "The coupling was less prominent between larynx and articulator motion, but we're seeing a reduction in complexity between respiratory and larynx motion," Talkar says.
These preliminary results hint that biomarkers derived from vocal system coordination can indicate the presence of Covid-19. However, the researchers note that it's still early to draw conclusions, and more data are needed to validate their findings. They're working now with a publicly released dataset from Carnegie Mellon University that contains audio samples from individuals who have tested positive for Covid-19.
Beyond collecting more data to fuel this research, the team is looking at using mobile apps to implement it. A partnership is underway with Satra Ghosh at the MIT McGovern Institute for Brain Research to integrate vocal screening for Covid-19 into its VoiceUp app, which was initially developed to study the link between voice and depression. A follow-on effort could add this vocal screening into the How We Feel app. This app asks users questions about their daily health status and demographics, with the aim to use these data to pinpoint hotspots and predict the percentage of people who have the disease in different regions of the country. Asking users to also submit a daily voice memo to screen for biomarkers of Covid-19 could potentially help scientists catch on to an outbreak.
"A sensing system integrated into a mobile app could pick up on infections early, before people feel sick or, especially, for these subsets of people who don't ever feel sick or show symptoms," says Jeffrey Palmer, who leads the research group. "This is also something the U.S. Army is interested in as part of a holistic Covid-19 monitoring system." Even after a diagnosis, this sensing ability could help doctors remotely monitor their patients' progress or monitor the effects of a vaccine or drug treatment.
As the team continues their research, they plan to do more to address potential confounders that could cause inaccuracies in their results, such as different recording environments, the emotional status of the subjects, or other illnesses causing vocal changes. They're also supporting similar research. The Mass General Brigham Center for COVID Innovation has connected them to international scientists who are following the team's framework to analyze coughs.
"There are a lot of other interesting areas to look at. Here, we looked at the physiological impacts on the vocal tract. We're also looking to expand our biomarkers to consider neurophysiological impacts linked to Covid-19, like the loss of taste and smell," Quatieri says. "Those symptoms can affect speaking, too."
- Coronavirus – this is how much you touch your face | World ... ›
- Social distancing: simple math explains how well it works - Big Think ›
- Tales from the frontline: COVID-19 survivors share their stories - Big Think ›
- COVID-19 symptoms appear in this specific order - Big Think ›
Join Radiolab's Latif Nasser at 1pm ET on Monday as he chats with Malcolm Gladwell live on Big Think.
University of Utah research finds that men are especially well suited for fisticuffs.
- With males having more upper-body mass than women, a study looks to find the reason.
- The study is based on the assumption that men have been fighters for so long that evolution has selected those best-equipped for the task.
- If men fought other men, winners would have survived and reproduced, losers not so much.
Built for mayhem<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMjY2NDIyMy9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYxMzk4NTQ2OX0.my6nML12F3fEQu3H4G0BScdqgaMZkRQHxgyj-Cmjmzk/img.jpg?width=980" id="906fc" class="rm-shortcode" data-rm-shortcode-id="dd77af7a881631355ed8972437846394" data-rm-shortcode-name="rebelmouse-image" />
Image source: Ollyy/Shutterstock<p>The researchers are, of course, talking averages here, not stating a rule: There are plenty of accomplished female pugilists, as well as lots of males who have no idea how to throw a punch.</p><p>Even so, says co-author <a href="https://www.wofford.edu/academics/majors-and-programs/biology/faculty-and-staff" target="_blank">Jeremy Morris</a> says, "The general approach to understanding why sexual dimorphism evolves is to measure the actual differences in the muscles or the skeletons of males and females of a given species, and then look at the behaviors that might be driving those differences."</p><p>Carrier has been interested in the idea that millennia of male fighting has shaped certain structures in male bodies. Previous research has reinforced his hunch:</p> <ul> <li><a href="https://jeb.biologists.org/content/216/2/236" target="_blank">When a hand is formed into a fist, its structure is self-protective</a>.</li> <li><a href="https://unews.utah.edu/flat-footed-fighters/" target="_blank">Heels planted firmly on the ground augment upper-body power</a>.</li> <li><a href="https://www.ncbi.nlm.nih.gov/pubmed/24909544" target="_blank">A study examined facial bone structure as being especially well-suited for taking a punch</a>.</li> </ul> <p>(That last one is our favorite. Do you know the German word "<a href="https://www.urbandictionary.com/define.php?term=Backpfeifengesicht" target="_blank">backpfeifengesicht</a>?" It's an adjective describing "a face that badly needs a punching.")</p><p>"One of the predictions that comes out of those," asserts Carrier, "is if we are specialized for punching, you might expect males to be particularly strong in the muscles that are associated with throwing a punch."</p>
Testing the theory<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMjY2NDIzMy9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYwNzMxMTE2MH0.UXJICMy57UPYUWskhK98alctOrPidJL9yxMkz3HDQrM/img.jpg?width=980" id="98718" class="rm-shortcode" data-rm-shortcode-id="b12287684ac3e740b70392e6433a6b8f" data-rm-shortcode-name="rebelmouse-image" />
Image source: Ollyy/Shutterstock<p>The researchers measured the punching — and spear-throwing — force of 20 men and 19 women. The assumption was that early humans were punchers <em>and</em> spear-throwers.</p><p>Prior to testing, each participant had filled out an activity questionnaire so that "we weren't getting couch potatoes, we were getting people that were very fit and active," says Morris.</p><p>For punching, participants operated a hand crank that required movement similar to throwing a haymaker. The purpose of the hand crank was to spare participants any damage that might be inflicted on their fists by throwing actual punches. Subjects were also measured pulling a line forward over their heads to assess their strength at throwing a spear.</p><p>Even though all of the participants, male and female, were routinely fit, the average power of males was assessed as being 162% greater than females. There were no gender differences in throwing strength recorded. Other untested, though presumably likely, hand-to-hand combat activities come to mind including tackling, clubbing, running, kicking, scratching, and biting.</p><p>Carrier's takeaway: "This is a dramatic example of sexual dimorphism that's consistent with males becoming more specialized for fighting, and males fighting in a particular way, which is throwing punches."</p>
Boys will be boys<p>It, er, strikes us as odd that, even in science fiction — hi-tech weaponry notwithstanding — the hero <em>is</em> going to wind up duking it out with some bad guy, or alien, in the climactic battle. What is it about men punching, anyway? Are they more sexually attractive? The study suggests so:</p><p style="margin-left: 20px;"><em>The results of this study add to a set of recently identified characters indicating that sexual selection on male aggressive performance has played a role in the evolution of the human musculoskeletal system and the evolution of sexual dimorphism in hominins.</em></p><p>It's tough to contribute to the gene pool after being killed in battle.</p><p>Also, while the authors aren't <em>quite</em> saying that males' historical fighting role is mandated by biology and not by social expectations, neither are they quite <em>not</em> saying it.</p><p>As Carrier explain to <a href="https://attheu.utah.edu/facultystaff/carrier-punch/" target="_blank">theU</a>: "Human nature is also characterized by avoiding violence and finding ways to be cooperative and work together, to have empathy, to care for each other, right? There are two sides to who we are as a species. If our goal is to minimize all forms of violence in the future, then understanding our tendencies and what our nature really is, is going to help."</p>
Innovators don't ignore risk; they are just better able to analyze it in uncertain situations.
The Labour Economics study suggests two potential reasons for the increase: corruption and increased capacity.
Cool hand rebuke<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNDQyMTIyNy9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY0NjY1NTYyOH0.0MCPKN3If94mYCNf3mMNrnTvJXjXN_bKLhgk9203EXk/img.jpg?width=917&coordinates=0%2C0%2C0%2C0&height=453" id="1627b" class="rm-shortcode" data-rm-shortcode-id="6d76421ba1ea0de4b09956b97e80c384" data-rm-shortcode-name="rebelmouse-image" />
A chart showing prison population rates (per 100,000 people) in 2018. The United States has the highest rate of incarceration in the world.