This Twitter algorithm predicts mental illness better than trained professionals

supervised learning algorithm can predict clinical depression much earlier and more accurately than trained health professionals.


One of the more surprising, and upsetting, uses of social media has been suicides performed on Facebook Live. Though reasons for suicide are complex, the mere threat is often a cry for help, acceptance, or recognition. During the two years I worked as a patient monitor in an emergency room, I discovered most people that attempt to take their own lives desire a pair of ears to listen to their problems more than anything else.

It's hard to gauge a person's reality based on social media habits, however. Those who spout vitriolic rhetoric are often quite approachable and reserved in person. We can't read inflections and temperament from words on a screen, or take into consideration that the person might just be having a bad day.

That said, social media can be a powerful indicator of those at risk for suffering from mental health disorders, a new study published in Scientific Reports suggests. A team led by Andrew Reece, in the Department of Psychology at Harvard, collected Twitter data from 204 individuals. Of those, 105 suffered from depression, with a control of 99 healthy subjects. The team then used a supervised learning algorithm to see if changes in language predicted clinical depression.

The answer is yes. Depressed patients used more words like death, no, and never, while posting fewer positive words—like happy, beach, and photo—in the lead-up to their diagnosis.

Figure 4. Depression word-shift graph revealing contributions to difference in Twitter happiness observed between depressed (5.98) and healthy (6.11) participants. In column 3, (−) indicates a relatively negative word, and (+) indicates a relatively positive word, both with respect to the average happiness of all healthy tweets. An up (down) arrow indicates that word was used more (less) by the depressed class. Words on the left (right) contribute to a decrease (increase) in happiness in the depressed class. [Credit: Andrew G. Reece et. al.]

A second group pf 174 Twitter users was also studied. Of these, 63 suffered from PTSD. Again, changes in language revealed that they were likely to be diagnosed.

These results are not perfect. In both situations there was a preselected pool of Twitter users with a close ratio of healthy to unhealthy, which does not reflect society as a whole. Add to this fact that many depressed people or those suffering from PTSD do not use social media. It would be hard to acquire firm numbers based on these shortcomings.

That said, Reece and his team are borrowing this predictive model from similar early warning systems in place for hard-to-detect cancers, disease outbreaks, and regional dietary health issues. Diseases like addiction and suicidal ideation have already been studied through social media. While this trend of using public facing data to detect potential cognitive disorders is new, cries for help might be detected, and treated, much sooner.

Reece and team believe they have found if not a silver bullet for predicting depression and PTSD, at least a shinier one than has so far been developed:

Our findings strongly support the claim that computational methods can effectively screen Twitter data for indicators of depression and PTSD. Our method identified these mental health conditions earlier and more accurately than the performance of trained health professionals, and was more precise than previous computational approaches.

With the current rise in depression and anxiety, especially among teens, a particularly vulnerable group that has now fully grown up on social media, such predictive tools could prove to be a valuable source of therapy and recovery moving forward.

"We hope that our research will eventually help improve mental health care, for example in preventive screening," Stanford researcher Katharina Lix told Digital Trends. “We could imagine clinicians using this technology as a supporting tool during a patient's initial assessment, provided that the patient has agreed to have their social media data used in this way. However, before we get to that point, the technology needs to be validated using a larger sample of people that's representative of the general population. We want to emphasize that any real-world application of this technology must carefully take into account ethical and privacy concerns."

--

Derek is the author of Whole Motion: Training Your Brain and Body For Optimal Health. Based in Los Angeles, he is working on a new book about spiritual consumerism. Stay in touch on Facebook and Twitter.

America’s education system is centuries old. Can we build something better?

The Lumina Foundation lays out steps for increasing access to quality post-secondary education credentials.

Sponsored by Lumina Foundation
  • America's post-high school education landscape was not created with the modern student in mind. Today, clear and flexible pathways are necessary to help individuals access education that can help them lead a better life.
  • Elizabeth Garlow explains the Lumina Foundation's strategy to create a post-secondary education system that works for all students. This includes credential recognition, affordability, a more competency-based system, and quality assurance.
  • Systemic historic factors have contributed to inequality in the education system. Lumina aims to close those gaps in educational attainment.
  • In 2019, Lumina Foundation and Big Think teamed up to create the Lumina Prize, a search to find the most innovative and scalable ideas in post-secondary education. You can see the winners of the Lumina Prize here – congratulations to PeerForward and Greater Commons!

First solar roadway in France turned out to be a 'total disaster'

French newspapers report that the trial hasn't lived up to expectations.

Image source: Charly Triballeau / AFP / Getty Images
Technology & Innovation
  • The French government initially invested in a rural solar roadway in 2016.
  • French newspapers report that the trial hasn't lived up to expectations.
  • Solar panel "paved" roadways are proving to be inefficient and too expensive.
Keep reading Show less

What was it like to live in a Japanese concentration camp?

During World War II, the U.S. incarcerated over 100,000 Japanese Americans in concentration camps throughout the West.

Universal History Archive/Universal Images Group via Getty Images
Politics & Current Affairs
  • Now that the issue of concentration camps in the U.S. has once again reared its head, it can be beneficial to recall the last time such camps were employed in the U.S.
  • After Pearl Harbor, the U.S. incarcerated over 100,000 Japanese Americans in camps, ostensibly for national security purposes.
  • In truth, the incarceration was primarily motivated by racism. What was life like in the U.S.'s concentration camps?

On February 19, 1942, President Roosevelt issued Executive Order 9066, which authorized and directed military commanders "to prescribe military areas … from which any or all persons may be excluded, and with respect to which, the right of any person to enter, remain in, or leave shall be subject to whatever restrictions the Secretary of War or the appropriate Military Commander may impose in his discretion." Under the authority of this executive order, roughly 112,000 men, women, and children of Japanese descent — nearly two-thirds of which were American citizens — were detained in concentration camps.

How did the camps get their start?

With the benefit of a nearly 80-year perspective, it's clear that the internment of Japanese Americans was racially motivated. In response to Japan's growing military power in the buildup to World War II, President Roosevelt commissioned two reports to determine whether it would be necessary to intern Japanese Americans should conflict break out between Japan and the U.S. Neither's conclusions supported the plan, with one even going so far as to "certify a remarkable, even extraordinary degree of loyalty among this generally suspect ethnic group." But of course, the Pearl Harbor attacks proved to be far more persuasive than these reports.

Pearl Harbor turned simmering resentment against the Japanese to a full boil, putting pressure on the Roosevelt administration to intern Japanese Americans. Lieutenant General John DeWitt, who would become the administrator of the internment program, testified to Congress

"I don't want any of them here. They are a dangerous element. There is no way to determine their loyalty... It makes no difference whether he is an American citizen, he is still a Japanese. American citizenship does not necessarily determine loyalty... But we must worry about the Japanese all the time until he is wiped off the map."

DeWitt's position was backed up by a number of pre-existing anti-immigrant groups based out of the West Coast, such as the Joint Immigration Committee and the Native Sons and Daughters of the Golden West. For many, the war simply served as an excuse to get rid of Japanese Americans. In an interview with the Saturday Evening Post, Austin Anson, the managing secretary of the Salinas Vegetable Grower-Shipper Administration, said:

"We're charged with wanting to get rid of the Japs for selfish reasons. We do. It's a question of whether the White man lives on the Pacific Coast or the brown men. ... If all the Japs were removed tomorrow, we'd never miss them in two weeks because the White farmers can take over and produce everything the Jap grows. And we do not want them back when the war ends, either."

Ironically for Anson, the mass deportation of Japanese Americans under Executive Order 9066 meant there was a significant shortage of agricultural labor. Many Caucasians left to fight the war, so the U.S. signed an agreement with Mexico to permit the immigration of several million Mexicans agricultural workers under the so-called bracero program.

Life in the camps

Japanese American concentration camp

Hulton Archive/Getty Images

Circa 1943: Aerial view of a Japanese American relocation center in Amache, Colorado, during World War II. Each family was provided with a space 20 by 25 ft. The barracks were set in blocks and each block was provided with a community bath house and mess hall.

For the most part, Japanese Americans remained stoic in the face of their incarceration. The phrase shikata ga nai was frequently invoked — the phrase roughly translates to "it cannot be helped," which, for many, represents the perceived attitude of the Japanese people to withstand suffering that's out of their control.

Initially, most Japanese Americans were sent to temporary assembly centers, typically located at fairgrounds or racetracks. These were hastily constructed barracks, where prisoners were often packed into tight quarters and made to use toilets that were little more than pits in the ground. From here, they were relocated to more permanent camps — replete with barbed wire and armed guards — in remote, isolated places across the seven states of California, Arizona, Colorado, Wyoming, Idaho, Utah, and Arkansas.

Many of these camps, also known as War Relocation Centers, were little better than the temporary assembly centers. One report described the buildings as "tar paper-covered barracks of simple frame construction without plumbing or cooking facilities of any kind." Again, overcrowding was common.

As a result, disease became a major concern, including dysentery, malaria, and tuberculosis. This was problematic due to the chronic shortage of medical professionals and supplies, an issue that was not helped by the War Relocation Authority's decision to cap Japanese American medical professional's pay at $20 a month (about $315 in 2019 dollars), while Caucasian workers had no such restriction. As a comparison, Caucasian nurses earned $150 ($2,361) a month in one camp.

The U.S. government also administered loyalty questionnaires to incarcerated Japanese Americans with the ultimate goal of seeing whether they could be used as soldiers and to segregate "loyal" citizens from "disloyal" ones. The questionnaires often asked whether they would be willing to join the military and if they would completely renounce their loyalty to Japan. Due to fears of being drafted, general confusion, and justified anger at the U.S. government, thousands of Japanese Americans "failed" the loyalty questionnaire and were sent to the concentration camp at Tule Lake. When Roosevelt later signed a bill that would permit Japanese Americans to renounce their citizenship, 98 percent of the 5,589 who did were located at Tule Lake. Some apologists cite this an example of genuine disloyalty towards the U.S., but this argument clearly ignores the gross violation of Japanese Americans' rights. Later, it became clear that many of these renunciations had been made under duress, and nearly all of those who had renounced their citizenship sought to gain it back.

Since many children lived in the camps, they came equipped with schools. Of course, these schools weren't ideal — student-teacher ratios reached as high as 48:1, and supplies were limited. The irony of learning about American history and ideals was not lost on the students, one of whom wrote in an essay --

"They, the first generation [of Japanese immigrants], without the least knowledge of the English language nor the new surroundings, came to this land with the American pioneering spirit of resettling. ...Though undergoing many hardships, they did reach their goal only to be resettled by the order of evacuation under the emergency for our protection and public security."

Potentially the best part of life in the camps — and the best way for determined prisoners to demonstrate their fundamental American-ness — was playing baseball. One camp even featured nearly 100 baseball teams. Former prisoner Herb Kurima recalled the importance of baseball in their lives in an interview with Christian Science Monitor. "I wanted our fathers, who worked so hard, to have a chance to see a ball game," he said. "Over half the camp used to come out to watch. It was the only enjoyment in the camps."

The aftermath

When the camps finally closed in 1945, the lives of the incarcerated Japanese Americans had been totally upended. Some were repatriated to Japan, while others settled in whichever part of the country they had been arbitrarily placed in. Those who wished to return to the West Coast were given $25 and a train ticket, but few had anything to return to. Many had sold their property to predatory buyers prior to being incarcerated, while theft had wiped out whatever else they had left behind. Many, many years later, the 1988 Civil Liberties Act mandated that each surviving victim be paid $20,000, though that seems like a small fine to pay for irrevocably changing the courses of more than 100,000 lives.