Get smarter, faster. Subscribe to our daily newsletter.
Six ways machine learning threatens social justice
Machine learning is a powerful and imperfect tool that should not go unmonitored.
- When you harness the power and potential of machine learning, there are also some drastic downsides that you've got to manage.
- Deploying machine learning, you face the risk that it be discriminatory, biased, inequitable, exploitative, or opaque.
- In this article, I cover six ways that machine learning threatens social justice and reach an incisive conclusion: The remedy is to take on machine learning standardization as a form of social activism.
When you harness the power and potential of machine learning, there are also some drastic downsides that you've got to manage. Deploying machine learning, you face the risk that it be discriminatory, biased, inequitable, exploitative, or opaque. In this article, I cover six ways that machine learning threatens social justice and reach an incisive conclusion: The remedy is to take on machine learning standardization as a form of social activism.
When you use machine learning, you aren't just optimizing models and streamlining business. You're governing. In essence, the models embody policies that control access to opportunities and resources for many people. They drive consequential decisions as to whom to investigate, incarcerate, set up on a date, or medicate – or to whom to grant a loan, insurance coverage, housing, or a job.
For the same reason that machine learning is valuable—that it drives operational decisions more effectively—it also wields power in the impact it has on millions of individuals' lives. Threats to social justice arise when that impact is detrimental, when models systematically limit the opportunities of underprivileged or protected groups.
Here are six ways machine learning threatens social justice
Credit: metamorworks via Shutterstock
1) Blatantly discriminatory models are predictive models that base decisions partly or entirely on a protected class. Protected classes include race, religion, national origin, gender, gender identity, sexual orientation, pregnancy, and disability status. By taking one of these characteristics as an input, the model's outputs – and the decisions driven by the model – are based at least in part on membership in a protected class. Although models rarely do so directly, there is precedent and support for doing so.
This would mean that a model could explicitly hinder, for example, black defendants for being black. So, imagine sitting across from a person being evaluated for a job, a loan, or even parole. When they ask you how the decision process works, you inform them, "For one thing, our algorithm penalized your score by seven points because you're black." This may sound shocking and sensationalistic, but I'm only literally describing what the model would do, mechanically, if race were permitted as a model input.
2) Machine bias. Even when protected classes are not provided as a direct model input, we find, in some cases, that model predictions are still inequitable. This is because other variables end up serving as proxies to protected classes. This is a bit complicated, since it turns out that models that are fair in one sense are unfair in another.
For example, some crime risk models succeed in flagging both black and white defendants with equal precision – each flag tells the same probabilistic story, regardless of race – and yet the models falsely flag black defendants more often than white ones. A crime-risk model called COMPAS, which is sold to law enforcement across the US, falsely flags white defendants at a rate of 23.5%, and Black defendants at 44.9%. In other words, black defendants who don't deserve it are erroneously flagged almost twice as much as white defendants who don't deserve it.
3) Inferring sensitive attributes—predicting pregnancy and beyond. Machine learning predicts sensitive information about individuals, such as sexual orientation, whether they're pregnant, whether they'll quit their job, and whether they're going to die. Researchers have shown that it is possible to predict race based on Facebook likes. These predictive models deliver dynamite.
In a particularly extraordinary case, officials in China use facial recognition to identify and track the Uighurs, a minority ethnic group systematically oppressed by the government. This is the first known case of a government using machine learning to profile by ethnicity. One Chinese start-up valued at more than $1 billion said its software could recognize "sensitive groups of people." It's website said, "If originally one Uighur lives in a neighborhood, and within 20 days six Uighurs appear, it immediately sends alarms" to law enforcement.
4) A lack of transparency. A computer can keep you in jail, or deny you a job, a loan, insurance coverage, or housing – and yet you cannot face your accuser. The predictive models generated by machine learning to drive these weighty decisions are generally kept locked up as a secret, unavailable for audit, inspection, or interrogation. Such models, inaccessible to the public, perpetrate a lack of due process and a lack of accountability.
Two ethical standards oppose this shrouding of electronically-assisted decisions: 1) model transparency, the standard that predictive models be accessible, inspectable, and understandable. And 2) the right to explanation, the standard that consequential decisions that are driven or informed by a predictive model are always held up to that standard of transparency. Meeting those standards would mean, for example, that a defendant be told which factors contributed to their crime risk score -- which aspects of their background, circumstances, or past behavior caused the defendant to be penalized. This would provide the defendant the opportunity to respond accordingly, establishing context, explanations, or perspective on these factors.
5) Predatory micro-targeting. Powerlessness begets powerlessness – and that cycle can magnify for consumers when machine learning increases the efficiency of activities designed to maximize profit for companies. Improving the micro-targeting of marketing and the predictive pricing of insurance and credit can magnify the cycle of poverty. For example, highly-targeted ads are more adept than ever at exploiting vulnerable consumers and separating them from their money.
And insurance pricing can lead to the same result. With insurance, the name of the game is to charge more for those at higher risk. Left unchecked, this process can quickly slip into predatory pricing. For example, a churn model may find that elderly policyholders don't tend to shop around and defect to better offers, so there's less of an incentive to keep their policy premiums in check. And pricing premiums based on other life factors also contributes to a cycle of poverty. For example, individuals with poor credit ratings are charged more for car insurance. In fact, a low credit score can increase your premium more than an at-fault car accident.
6) The coded gaze. If a group of people is underrepresented in the data from which the machine learns, the resulting model won't work as well for members of that group. This results in exclusionary experiences and discriminatory practices. This phenomenon can occur for both facial image processing and speech recognition.
Recourse: Establish machine learning standards as a form of social activism
To address these problems, take on machine learning standardization as a form of social activism. We must establish standards that go beyond nice-sounding yet vague platitudes such as "be fair", "avoid bias", and "ensure accountability". Without being precisely defined, these catch phrases are subjective and do little to guide concrete action. Unfortunately, such broad language is fairly common among the principles released by many companies. In so doing, companies protect their public image more than they protect the public.
People involved in initiatives to deploy machine learning have a powerful, influential voice. These relatively small numbers of people mold and set the trajectory for systems that automatically dictate the rights and resources that great numbers of consumers and citizens gain access to.
Famed machine learning leader and educator Andrew Ng drove it home: "AI is a superpower that enables a small team to affect a huge number of people's lives... Make sure the work you do leaves society better off."
And Allan Sammy, Director, Data Science and Audit Analytics at Canada Post, clarified the level of responsibility: "A decision made by an organization's analytic model is a decision made by that entity's senior management team."
Implementing ethical data science is as important as ensuring a self-driving car knows when to put on the breaks.
Establishing well-formed ethical standards for machine learning will be an intensive, ongoing process. For more, watch this short video, in which I provide some specifics meant to kick-start the process.
Eric Siegel, Ph.D., is a leading consultant and former Columbia University professor who makes machine learning understandable and captivating. He is the founder of the long-running Predictive Analytics World and the Deep Learning World conference series and the instructor of the end-to-end, business-oriented Coursera specialization Machine learning for Everyone. Stay in touch with Eric on Twitter @predictanalytic.
- Prejudice AI? Machine Learning Can Pick up Society's Biases - Big ... ›
- Your safety depends on machine learning - Big Think ›
- Dawn of a new era: AI, machine learning, and robotics - Big Think ›
The COVID-19 pandemic is making health disparities in the United States crystal clear. It is a clarion call for health care systems to double their efforts in vulnerable communities.
- The COVID-19 pandemic has exacerbated America's health disparities, widening the divide between the haves and have nots.
- Studies show disparities in wealth, race, and online access have disproportionately harmed underserved U.S. communities during the pandemic.
- To begin curing this social aliment, health systems like Northwell Health are establishing relationships of trust in these communities so that the post-COVID world looks different than the pre-COVID one.
COVID-19 deepens U.S. health disparities<p>Communities on the pernicious side of America's health disparities have their unique histories, environments, and social structures. They are spread across the United States, but they all have one thing in common.</p><p>"There is one common divide in American communities, and that is poverty," said <a href="https://www.northwell.edu/about/leadership/debbie-salas-lopez" target="_blank">Debbie Salas-Lopez, MD, MPH</a>, senior vice president of community and population health at Northwell Health. "That is the undercurrent that manifests poor health, poor health outcomes, or poor health prognoses for future wellbeing."</p><p>Social determinants have far-reaching effects on health, and poor communities have unfavorable social determinants. To pick one of many examples, <a href="https://www.npr.org/2020/09/27/913612554/a-crisis-within-a-crisis-food-insecurity-and-covid-19" target="_blank" rel="noopener noreferrer">food insecurity</a> reduces access to quality food, leading to poor health and communal endemics of chronic medical conditions. The U.S. Centers for Disease Control and Prevention has identified some of these conditions, such as obesity and Type 2 diabetes, as increasing the risk of developing a severe case of coronavirus.</p><p>The pandemic didn't create poverty or food insecurity, but it exacerbated both, and the results have been catastrophic. A study published this summer in the <em><a href="https://link.springer.com/article/10.1007/s11606-020-05971-3" target="_blank">Journal of General Internal Medicine</a></em> suggested that "social factors such as income inequality may explain why some parts of the USA are hit harder by the COVID-19 pandemic than others."</p><p>That's not to say better-off families in the U.S. weren't harmed. A <a href="https://voxeu.org/article/poverty-inequality-and-covid-19-us" target="_blank" rel="noopener noreferrer">paper from the Centre for Economic Policy Research</a> noted that families in counties with a higher median income experienced adjustment costs associated with the pandemic—for example, lowering income-earning interactions to align with social distancing policies. However, the paper found that the costs of social distancing were much greater for poorer families, who cannot easily alter their living circumstances, which often include more individuals living in one home and a reliance on mass transit to reach work and grocery stores. They are also disproportionately represented in essential jobs, such as retail, transportation, and health care, where maintaining physical distance can be all but impossible.</p><p>The paper also cited a positive correlation between higher income inequality and higher rates of coronavirus infection. "Our interpretation is that poorer people are less able to protect themselves, which leads them to different choices—they face a steeper trade-off between their health and their economic welfare in the context of the threats posed by COVID-19," the authors wrote.</p><p>"There are so many pandemics that this pandemic has exacerbated," Dr. Salas-Lopez noted.</p><p>One example is the health-wealth gap. The mental stressors of maintaining a low socioeconomic status, especially in the face of extreme affluence, can have a physically degrading impact on health. <a href="https://www.scientificamerican.com/index.cfm/_api/render/file/?method=inline&fileID=123ECD96-EF81-46F6-983D2AE9A45FA354" target="_blank" rel="noopener noreferrer">Writing on this gap</a>, Robert Sapolsky, professor of biology and neurology at Stanford University, notes that socioeconomic stressors can increase blood pressure, reduce insulin response, increase chronic inflammation, and impair the prefrontal cortex and other brain functions through anxiety, depression, and cognitive load. </p><p>"Thus, from the macro level of entire body systems to the micro level of individual chromosomes, poverty finds a way to produce wear and tear," Sapolsky writes. "It is outrageous that if children are born into the wrong family, they will be predisposed toward poor health by the time they start to learn the alphabet."</p>Research on the economic and mental health fallout of COVID-19 is showing two things: That unemployment is hitting <a href="https://www.pewsocialtrends.org/2020/09/24/economic-fallout-from-covid-19-continues-to-hit-lower-income-americans-the-hardest/" target="_blank" rel="noopener noreferrer">low-income and young Americans</a> most during the pandemic, potentially widening the health-wealth gap further; and that the pandemic not only exacerbates mental health stressors, but is doing so at clinically relevant levels. As <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7413844/" target="_blank" rel="noopener noreferrer">the authors of one review</a> wrote, the pandemic's effects on mental health is itself an international public health priority.
Working to close the health gap<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNDc5MDk1MS9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYxNTYyMzQzMn0.KSFpXH7yHYrfVPtfgcxZqAHHYzCnC2bFxwSrJqBbH4I/img.jpg?width=980" id="b40e2" class="rm-shortcode" data-rm-shortcode-id="1b9035370ab7b02a0dc00758e494412b" data-rm-shortcode-name="rebelmouse-image" />
Northwell Health coronavirus testing center at Greater Springfield Community Church.
Credit: Northwell Health<p>Novel coronavirus may spread and infect indiscriminately, but pre-existing conditions, environmental stressors, and a lack of access to care and resources increase the risk of infection. These social determinants make the pandemic more dangerous, and erode communities' and families' abilities to heal from health crises that pre-date the pandemic.</p><p>How do we eliminate these divides? Dr. Salas-Lopez says the first step is recognition. "We have to open our eyes to see the suffering around us," she said. "Northwell has not shied away from that."</p><p>"We are steadfast in improving health outcomes for our vulnerable and underrepresented communities that have suffered because of the prevalence of chronic disease, a problem that led to the disproportionately higher death rate among African-Americans and Latinos during the COVID-19 pandemic," said Michael Dowling, Northwell's president and CEO. "We are committed to using every tool at our disposal—as a provider of health care, employer, purchaser and investor—to combat disparities and ensure the <a href="https://www.northwell.edu/education-and-resources/community-engagement/center-for-equity-of-care" target="_blank" rel="noopener noreferrer">equity of care</a> that everyone deserves." </p><p>With the need recognized, Dr. Salas-Lopez calls for health care systems to travel upstream and be proactive in those hard-hit communities. This requires health care systems to play a strong role, but not a unilateral one. They must build <a href="https://www.northwell.edu/news/insights/faith-based-leaders-are-the-key-to-improving-community-health" target="_blank" rel="noopener noreferrer">partnerships with leaders in those communities</a> and utilize those to ensure relationships last beyond the current crisis. </p><p>"We must meet with community leaders and talk to them to get their perspective on what they believe the community needs are and should be for the future. Together, we can co-create a plan to measurably improve [community] health and also to be ready for whatever comes next," she said.</p><p>Northwell has built relationships with local faith-based and community organizations in underserved communities of color. Those partnerships enabled Northwell to test more than 65,000 people across the metro New York region. The health system also offered education on coronavirus and precautions to curb its spread.</p><p>These initiatives began the process of building trust—trust that Northwell has counted on to return to these communities to administer flu vaccines to prepare for what experts fear may be a difficult flu season.</p><p>While Northwell has begun building bridges across the divides of the New York area, much will still need to be done to cure U.S. health care overall. There is hope that the COVID pandemic will awaken us to the deep disparities in the US.</p><p>"COVID has changed our world. We have to seize this opportunity, this pandemic, this crisis to do better," Dr. Salas-Lopez said. "Provide better care. Provide better health. Be better partners. Be better community citizens. And treat each other with respect and dignity.</p><p>"We need to find ways to unify this country because we're all human beings. We're all created equal, and we believe that health is one of those important rights."</p>
A study by UK archaeologists finds that longbows caused horrific injuries similar to modern gunshot wounds.
- UK archaeologists discover medieval longbows caused injuries similar to modern gunshot wounds.
- The damage was caused by the arrows spinning clockwise.
- No longbows from medieval times survived until our times.
Battle of Agincourt.
The angle of entry into a cranium found during the excavation at a medieval Dominican friary in Exeter, England.
Credit: Oliver Creighton/University of Exeter
Can passenger airships make a triumphantly 'green' comeback?
Large airships were too sensitive to wind gusts and too sluggish to win against aeroplanes. But today, they have a chance to make a spectacular return.
Vegans and vegetarians often have nutrient deficiencies and lower BMI, which can increase the risk of fractures.
- The study found that vegans were 43% more likely to suffer fractures than meat eaters.
- Similar results were observed for vegetarians and fish eaters, though to a lesser extent.
- It's possible to be healthy on a vegan diet, though it takes some strategic planning to compensate for the nutrients that a plant-based diet can't easily provide.
Comparison of fracture cases by diet group
Credit: Tong et al.<p>The results showed that vegans were especially vulnerable to hip fractures, suffering 2.3 times more cases than meat-eaters. Vegetarians and pescatarians were also more likely to suffer hip fractures, though to a lesser extent.</p><p>One explanation may be that non-meat eaters consume less calcium and protein. Calcium helps the body build strong bones, particularly before age 30, after which the body begins to lose bone mineral density (though consuming enough calcium through diet or supplement can <a href="https://ods.od.nih.gov/factsheets/Calcium-Consumer/" target="_blank">help offset losses</a>). Lower bone mineral density means higher risk of fracture.</p><p>Protein seems to help the body absorb calcium, <a href="https://www.bonejoint.net/blog/did-you-know-that-certain-foods-block-calcium-absorption/#:~:text=Historically%2C%20nutritionists%20have%20warned%20that,may%20increase%20intestinal%20calcium%20absorption." target="_blank" rel="noopener noreferrer">when consumed in normal levels</a>. The recent study, along with past research, shows that people who don't eat meat tend to have lower levels of both protein and calcium. When the researchers accounted for non-meat eaters who supplemented their diets with calcium and protein, fracture risk decreased, but still remained significant.</p>
Credit: Pixabay<p>Another explanation is body mass index (BMI). Non-meat eaters tend to have a lower BMI, which is associated with higher fracture risk, particularly hip fractures. In the new study, vegans with a low BMI were especially likely to suffer hip fractures. That might be because having more body mass provides a cushioning effect when people fall.</p><p>Still, the study has some limitations. For one, White European women were overrepresented in the sample. The researchers also didn't collect precise data on the type of calcium or protein supplementation, diet quality or causes of fractures.</p><p>Another complicating factor: Producers of vegan products, such as plant-based milk, are increasingly fortifying foods with nutrients like calcium and protein, so modern vegans are potentially at lower risk of deficiency.</p><p>The researchers wrote that their findings "suggest that bone health in vegans requires further research."</p>