Once a week.
Subscribe to our weekly newsletter.
Signs of Covid-19 may be hidden in speech signals
Studying voice recordings of infected but asymptomatic people reveals potential indicators of Covid-19.
Maybe their voices are lower or have a nasally tone. Infections change the quality of our voices in various ways. But MIT Lincoln Laboratory researchers are detecting these changes in Covid-19 patients even when these changes are too subtle for people to hear or even notice in themselves.
By processing speech recordings of people infected with Covid-19 but not yet showing symptoms, these researchers found evidence of vocal biomarkers, or measurable indicators, of the disease. These biomarkers stem from disruptions the infection causes in the movement of muscles across the respiratory, laryngeal, and articulatory systems. A technology letter describing this research was recently published in IEEE Open Journal of Engineering in Medicine and Biology.
While this research is still in its early stages, the initial findings lay a framework for studying these vocal changes in greater detail. This work may also hold promise for using mobile apps to screen people for the disease, particularly those who are asymptomatic.
"I had this 'aha' moment while I was watching the news," says Thomas Quatieri, a senior staff member in the laboratory's Human Health and Performance Systems Group. Quatieri has been leading the group's research in vocal biomarkers for the past decade; their focus has been on discovering vocal biomarkers of neurological disorders such as amyotrophic lateral sclerosis (ALS) and Parkinson's disease. These diseases, and many others, change the brain's ability to turn thoughts into words, and those changes can be detected by processing speech signals.
He and his team wondered whether vocal biomarkers might also exist for Covid-19. The symptoms led them to think so. When symptoms manifest, a person typically has difficulty breathing. Inflammation in the respiratory system affects the intensity with which air is exhaled when a person talks. This air interacts with hundreds of other potentially inflamed muscles on its journey to speech production. These interactions impact the loudness, pitch, steadiness, and resonance of the voice — measurable qualities that form the basis of their biomarkers.
While watching the news, Quatieri realized there were speech samples in front of him of people who had tested positive for Covid-19. He and his colleagues combed YouTube for clips of celebrities or TV hosts who had given interviews while they were Covid-19 positive but asymptomatic. They identified five subjects. Then, they downloaded interviews of those people from before they had Covid-19, matching audio conditions as best they could.
They then used algorithms to extract features from the vocal signals in each audio sample. "These vocal features serve as proxies for the underlying movements of the speech production systems," says Tanya Talkar, a PhD candidate in the Speech and Hearing Bioscience and Technology program at Harvard University.
The signal's amplitude, or loudness, was extracted as a proxy for movement in the respiratory system. For studying movements in the larynx, they measured pitch and the steadiness of pitch, two indicators of how stable the vocal cords are. As a proxy for articulator movements — like those of the tongue, lips, jaw, and more — they extracted speech formants. Speech formants are frequency measurements that correspond to how the mouth shapes sound waves to create a sequence of phonemes (vowels and consonants) and to contribute to a certain vocal quality (nasally versus warm, for example).
They hypothesized that Covid-19 inflammation causes muscles across these systems to become overly coupled, resulting in a less complex movement. "Picture these speech subsystems as if they are the wrist and fingers of a skilled pianist; normally, the movements are independent and highly complex," Quatieri says. Now, picture if the wrist and finger movements were to become stuck together, moving as one. This coupling would force the pianist to play a much simpler tune.
The researchers looked for evidence of coupling in their features, measuring how each feature changed in relation to another in 10 millisecond increments as the subject spoke. These values were then plotted on an eigenspectrum; the shape of this eigenspectrum plot indicates the complexity of the signals. "If the eigenspace of the values forms a sphere, the signals are complex. If there is less complexity, it might look more like a flat oval," Talkar says.
In the end, they found a decreased complexity of movement in the Covid-19 interviews as compared to the pre-Covid-19 interviews. "The coupling was less prominent between larynx and articulator motion, but we're seeing a reduction in complexity between respiratory and larynx motion," Talkar says.
These preliminary results hint that biomarkers derived from vocal system coordination can indicate the presence of Covid-19. However, the researchers note that it's still early to draw conclusions, and more data are needed to validate their findings. They're working now with a publicly released dataset from Carnegie Mellon University that contains audio samples from individuals who have tested positive for Covid-19.
Beyond collecting more data to fuel this research, the team is looking at using mobile apps to implement it. A partnership is underway with Satra Ghosh at the MIT McGovern Institute for Brain Research to integrate vocal screening for Covid-19 into its VoiceUp app, which was initially developed to study the link between voice and depression. A follow-on effort could add this vocal screening into the How We Feel app. This app asks users questions about their daily health status and demographics, with the aim to use these data to pinpoint hotspots and predict the percentage of people who have the disease in different regions of the country. Asking users to also submit a daily voice memo to screen for biomarkers of Covid-19 could potentially help scientists catch on to an outbreak.
"A sensing system integrated into a mobile app could pick up on infections early, before people feel sick or, especially, for these subsets of people who don't ever feel sick or show symptoms," says Jeffrey Palmer, who leads the research group. "This is also something the U.S. Army is interested in as part of a holistic Covid-19 monitoring system." Even after a diagnosis, this sensing ability could help doctors remotely monitor their patients' progress or monitor the effects of a vaccine or drug treatment.
As the team continues their research, they plan to do more to address potential confounders that could cause inaccuracies in their results, such as different recording environments, the emotional status of the subjects, or other illnesses causing vocal changes. They're also supporting similar research. The Mass General Brigham Center for COVID Innovation has connected them to international scientists who are following the team's framework to analyze coughs.
"There are a lot of other interesting areas to look at. Here, we looked at the physiological impacts on the vocal tract. We're also looking to expand our biomarkers to consider neurophysiological impacts linked to Covid-19, like the loss of taste and smell," Quatieri says. "Those symptoms can affect speaking, too."
- Coronavirus – this is how much you touch your face | World ... ›
- Social distancing: simple math explains how well it works - Big Think ›
- Tales from the frontline: COVID-19 survivors share their stories - Big Think ›
- COVID-19 symptoms appear in this specific order - Big Think ›
- AI can detect Covid-19 infections in coughs over the phone - Big Think ›
- A speech coach's tips for communicating while wearing a mask - Big Think ›
- Coronavirus study: could Twitter have predicted outbreaks? - Big Think ›
- Covid-19 personality types: what society learns from COVID - Big Think ›
"You dream about these kinds of moments when you're a kid," said lead paleontologist David Schmidt.
- The triceratops skull was first discovered in 2019, but was excavated over the summer of 2020.
- It was discovered in the South Dakota Badlands, an area where the Triceratops roamed some 66 million years ago.
- Studying dinosaurs helps scientists better understand the evolution of all life on Earth.
David Schmidt, a geology professor at Westminster College, had just arrived in the South Dakota Badlands in summer 2019 with a group of students for a fossil dig when he received a call from the National Forest Service. A nearby rancher had discovered a strange object poking out of the ground. They wanted Schmidt to take a look.
"One of the very first bones that we saw in the rock was this long cylindrical bone," Schmidt told St. Louis Public Radio. "The first thing that came out of our mouths was, 'That kind of looks like the horn of a triceratops.'"
After authorities gave the go-ahead, Schmidt and a small group of students returned this summer and spent nearly every day of June and July excavating the skull.
Credit: David Schmidt / Westminster College
"We had to be really careful," Schmidt told St. Louis Public Radio. "We couldn't disturb anything at all, because at that point, it was under law enforcement investigation. They were telling us, 'Don't even make footprints,' and I was thinking, 'How are we supposed to do that?'"
Another difficulty was the mammoth size of the skull: about 7 feet long and more than 3,000 pounds. (For context, the largest triceratops skull ever unearthed was about 8.2 feet long.) The skull of Schmidt's dinosaur was likely a Triceratops prorsus, one of two species of triceratops that roamed what's now North America about 66 million years ago.
Credit: David Schmidt / Westminster College
The triceratops was an herbivore, but it was also a favorite meal of the Tyrannosaurus rex. That probably explains why the Dakotas contain many scattered triceratops bone fragments, and, less commonly, complete bones and skulls. In summer 2019, for example, a separate team on a dig in North Dakota made headlines after unearthing a complete triceratops skull that measured five feet in length.
Michael Kjelland, a biology professor who participated in that excavation, said digging up the dinosaur was like completing a "multi-piece, 3-D jigsaw puzzle" that required "engineering that rivaled SpaceX," he jokingly told the New York Times.
Morrison Formation in Colorado
James St. John via Flickr
The Badlands aren't the only spot in North America where paleontologists have found dinosaurs. In the 1870s, Colorado and Wyoming became the first sites of dinosaur discoveries in the U.S., ushering in an era of public fascination with the prehistoric creatures — and a competitive rush to unearth them.
Since, dinosaur bones have been found in 35 states. One of the most fruitful locations for paleontologists has been the Morrison formation, a sequence of Upper Jurassic sedimentary rock that stretches under the Western part of the country. Discovered here were species like Camarasaurus, Diplodocus, Apatosaurus, Stegosaurus, and Allosaurus, to name a few.
|Credit: Nobu Tamura/Wikimedia Commons|
As for "Shady" (the nickname of the South Dakota triceratops), Schmidt and his team have safely transported it to the Westminster campus. They hope to raise funds for restoration, and to return to South Dakota in search of more bones that once belonged to the triceratops.
Studying dinosaurs helps scientists gain a more complete understanding of our evolution, illuminating a through-line that extends from "deep time" to present day. For scientists like Schmidt, there's also the simple joy of coming to face-to-face with a lost world.
"You dream about these kinds of moments when you're a kid," Schmidt told St. Louis Public Radio. "You don't ever think that these things will ever happen."
We spend much of our early years learning arithmetic and algebra. What's the use?
- For the average person, math seems to play little to no role in their day-to-day life.
- But, the fanciest gadgets and technologies are all heavily reliant on mathematics.
- Without advanced (and often obscure) mathematics, modern society would not be possible.
The following is an adapted excerpt from the book What's the Use? It is reprinted with permission of the author and Hachette Book Group.
What is mathematics for?
What is it doing for us, in our daily lives?
Not so long ago, there were easy answers to these questions. The typical citizen used basic arithmetic all the time, if only to check the bill when shopping. Carpenters needed to know elementary geometry. Surveyors and navigators needed trigonometry as well. Engineering required expertise in calculus.
Today, things are different. The supermarket checkout totals the bill, sorts out the special meal deal, adds the sales tax. We listen to the beeps as the laser scans the barcodes, and as long as the beeps match the goods, we assume the electronic gizmos know what they are doing. Many professions still rely on extensive mathematical knowledge, but even there, we have outsourced most of the mathematics to electronic devices with built-in algorithms.
My subject is conspicuous by its absence. The elephant isn't even in the room.
It would be easy to conclude that mathematics has become outdated and obsolete, but that view is mistaken. Without mathematics, today's world would fall apart. As evidence, I am going to show you applications to politics, the law, kidney transplants, supermarket delivery schedules, Internet security, movie special effects, and making springs. We will see how mathematics plays an essential role in medical scanners, digital photography, ﬁber broadband, and satellite navigation. How it helps us predict the effects of climate change; how it can protect us against terrorists and Internet hackers.
Remarkably, many of these applications rely on mathematics that originated for totally different reasons, often just the sheer fascination of following your nose. While researching this book, I was repeatedly surprised when I came across uses of my subject that I had never dreamed existed. Often, they exploited topics that I would not have expected to have practical applications, like space-ﬁlling curves, quaternions, and topology.
Mathematics is a boundless, hugely creative system of ideas and methods. It lies just beneath the surface of the transformative technologies that are making the twenty-ﬁrst century totally different from any previous era — video games, international air travel, satellite communications, computers, the Internet, mobile phones. Scratch an iPhone, and you will see the bright glint of mathematics.
Please don't take that literally.
There is a tendency to assume that computers, with their almost miraculous abilities, are making mathematicians, indeed mathematics itself, obsolete. But computers no more displace mathematicians than the microscope displaced biologists. Computers change the way we go about doing mathematics, but mostly they relieve us of the tedious bits. They give us time to think, they help us search for patterns, and they add a powerful new weapon to help advance the subject more rapidly and more effectively.
In fact, a major reason why mathematics is becoming ever more essential is the ubiquity of cheap, powerful computers. Their rise has opened up new opportunities to apply mathematics to real-world issues. Methods that were hitherto impractical, because they needed too many calculations, have now become routine. The greatest mathematicians of the pencil-and-paper era would have ﬂung up their hands in despair at any method requiring a billion calculations. Today, we routinely use such methods, because we have technology that can do the sums in a split second. Mathematicians have long been at the forefront of the computer revolution — along with countless other professions, I hasten to add. Think of George Boole, who pioneered the symbolic logic that forms the basis of current computer architecture. Think of Alan Turing, and his universal Turing machine, a mathematical system that can compute anything that is computable. Think of Muhammad al-Khwarizmi, whose algebra text of 820 AD emphasized the role of systematic computational procedures, now named after him: algorithms.
Most of the algorithms that give computers their impressive abilities are ﬁrmly based on mathematics. Many of the techniques concerned have been taken "off the shelf" from the existing store of mathematical ideas, such as Google's PageRank algorithm, which quantiﬁes how important a website is and founded a multi-billion-dollar industry. Even the snazziest deep learning algorithm in artiﬁcial intelligence uses tried and tested mathematical concepts such as matrices and weighted graphs. A task as prosaic as searching a document for a particular string of letters involves, in one common method at least, a mathematical gadget called a ﬁnite-state automaton.
The involvement of mathematics in these exciting developments tends to get lost. So next time the media propel some miraculous new ability of computers to center stage, bear in mind that hiding in the wings there will be a lot of mathematics, and a lot of engineering, physics, chemistry, and psychology as well, and that without the support of this hidden cast of helpers, the digital superstar would be unable to strut its stuff in the spotlight.
The importance of mathematics in today's world is easily underestimated because nearly all of it goes on behind the scenes. Walk along a city street, and you are overwhelmed by signs proclaiming the daily importance of banks, greengrocers, supermarkets, fashion outlets, car repairs, lawyers, fast food, antiques, charities, and a thousand other activities and professions. You do not ﬁnd a brass plaque announcing the presence of a consulting mathematician. Supermarkets do not sell you mathematics in a can.
Dig a little deeper, however, and the importance of mathematics quickly becomes apparent. The mathematical equations of aerodynamics are vital to aircraft design. Navigation depends on trigonometry. The way we use it today is different from how Christopher Columbus used it, because we embody the mathematics in electronic devices instead of pen, ink, and navigation tables, but the underlying principles are much the same. The development of new medicines relies on statistics to make sure the drugs are safe and effective. Satellite communications depend on a deep understanding of orbital dynamics. Weather forecasting requires the solution of equations for how the atmosphere moves, how much moisture it contains, how warm or cold it is, and how all of those features interact. There are thousands of other examples. We do not notice they involve mathematics, because we do not need to know that to beneﬁt from the results.
A socially minded franchise model makes money while improving society.
- A social enterprise in California makes their franchises affordable with low interest loans and guaranteed salaries.
- The loans are backed by charitable foundations.
- If scaled up, the model could support tens of thousands of entrepreneurs who are currently financially incapable of entering franchise agreements.
The underdog challenging McDonald’s & Wall Street | Hard Reset by Freethink www.youtube.com
Social responsibility is becoming a major focus of many businesses. While turning a profit is always the ultimate goal — nobody can eat good intentions, after all — having a positive impact on society is becoming an equally important goal.
A restaurant chain in California, already focused on providing healthy food at a competitive cost, is testing a new way to create more entrepreneurs. Specifically, it is working with charitable foundations to provide business opportunities to those who normally would not have access.
When a company wants to expand without paying all of the upfront costs itself or taking on the entire risk of operating in a new market, it can enter into a franchise agreement with an entrepreneur. In exchange for a share of the profits (as well as some fees and adherence to certain quality standards), the entrepreneur — now a franchisee — can open their own branch of a larger brand. The entrepreneur enjoys the benefits of owning a business, while the brand owner can cash in on intellectual property.
This model is wildly successful. There is a reason you can find fast food joints like McDonald's everywhere from Times Square to Prague (next to the Museum of Communism, no less). According to the International Franchise Association, there were more than 733,000 franchised business establishments in the United States in 2018, accounting for nearly 3 percent of GDP.
The franchise model — in which a local agent keeps some earnings while handing over a portion to a central authority — isn't new. Indeed, variations have been around since the Middle Ages, though it only took off after WWII. Franchising is now a recognized system in many countries and is used in all manner of industries, including restaurants, pet supply stores, automotive repair shops, hotels, and even senior care.
The Catch-22: you have to spend money to make money
The biggest problem with franchising is the high cost of becoming a franchisee.
While the costs vary, opening a restaurant as a franchisee can easily cost $500,000. A franchise car repair shop can require $250,000, and opening a hotel under a franchise's banner can set a person back millions. In some cases, the franchiser also will set a minimum net worth requirement or insist that the money that pays their fees not be borrowed. Even if a person can find a way around that, most new businesses do not turn a profit for quite some time after opening. These limitations essentially rule out all but the wealthy from becoming a franchisee.
As a result, there are some social enterprises that are looking to make franchising more accessible to the less affluent.
As a business that hopes to rapidly expand, they looked to franchising. However, the idea of seeking out a bunch of rich people to support a business like theirs struck CEO Sam Polk as out of step with its vision. So, the company came up with a better idea.
Their Social Equity Franchise Program helps tenured Everytable employees open their own franchise locations through free training and assistance in securing low interest loans to finance the store. To help the entrepreneurs survive the difficult early years, participants in the program are assured an income of $40,000 in their first three years of operations. Repayments on the loans do not begin until after the business is turning a profit.
The capital for all these low interest loans comes from a number of foundations such as the California Wellness Foundation (Cal Wellness). Foundations like these are required to give away a small portion of their endowments every year on causes aligned with their missions. However, most of the rest of it is simply invested in the stock market to assure the endowment continues to exist.
People like Cal Wellness CEO Judy Belk have begun to invest that money elsewhere, like in loans to provide the money needed to open an Everytable franchise. As she explained to FreeThink:
"Cal Wellness and many other foundations are saying, 'I think we can do a little better with that [money]. Why not use that capital to invest in the communities that we're supposed to serve?'"
In the end, Everytable gets a new restaurant that expands the brand, foundations get returns on their investment, and the franchisee gets an opportunity that they likely never would have had without the program.
Expanding the Everytable model
If even a small share of the $2 trillion foundations in the U.S. have are invested into this sort of social cause, tens of thousands of loans could be given to those less affluent people who are looking to start a business. While this model likely would lower returns to institutional investors like charities, they could enjoy more tangible results in the communities they exist to serve. According to a report published by the Federal Reserve Bank of Atlanta, local entrepreneurship increases income and employment and decreases poverty.
At the individual level, this would help a lot of people who otherwise never would be able to seriously consider going into business for themselves. By a number of measures, business owners make more than wage workers and can also claim ownership of the assets that comprise the business. Beyond that, many small business owners enjoy the non-financial benefits of their position as well, including the independence and autonomy that often come with business ownership.
When working optimally, good business is good for society.
Fintech companies are using elements of video games to make personal finance more fun. But does it work, and what are the risks?