What responsibility should government authorities and Big Tech take in policing the spread of sedition-oriented content?
- It can be hard to believe that comical images online are enough to rile people up enough that they'll actually attack.
- Originating in the darker corners of the internet, Bugaloo is now prominent on mainstream online platforms like Facebook and Instagram.
- The Network Contagion Research Institute's recent series of Contagion and Ideology Reports uses machine learning to examine how memes spread.
What’s in a meme?<p>The Network Contagion Research Institute's recent series of <a href="https://ncri.io/reports/" target="_blank" rel="noopener noreferrer">Contagion and Ideology Reports </a>leverages machine learning to examine how memes spread. The idea is to unearth a better understanding of the role memes play in encouraging real-world violence. </p><p>So what exactly is a meme, and how did it become a tool for weaponizing the web? Originally coined by Oxford biologist Richard Dawkins in his 1976 book "<a href="https://www.theguardian.com/science/2016/may/29/selfish-gene-40-years-richard-dawkins-do-ideas-stand-up-adam-rutherford" target="_blank" rel="noopener noreferrer">The Selfish Gene</a><em>," </em>the term is defined there as "a unit of cultural transmission" that spreads like a virus from host to host and conveys an idea which changes the host's worldview. </p><p>Dawkins echoes "Naked Lunch" author William Burroughs's concept of <a href="https://www.twisttheknife.com/william-s-burroughs-playback-from-eden-to-watergate/" target="_blank" rel="noopener noreferrer">written language as a virus</a> that infects the reader and consequently builds realities that are brought to fruition through the act of speech. In this sense, a meme is an idea that spreads by iterative, collaborative imitation from person to person within a culture and carries symbolic meaning.</p><p>The militant alt-right's use of internet memes <a href="https://journals.openedition.org/angles/369" target="_blank" rel="noopener noreferrer">follows this pattern</a>. They are carefully designed on underground social media using codes, before attaining approval by like-minded users that disseminate these messages on mainstream platforms like Twitter or Facebook.</p><p>Sometimes these messages are lifted from innocuous sources and altered to convey hate. Take for example Pepe the Frog. Initially created by cartoonist Matt Furie as a mascot for slackers, Pepe was appropriated and altered by racists and homophobes until the Anti-Defamation League branded the frog as a <a href="https://www.adl.org/education/references/hate-symbols/pepe-the-frog" target="_blank">hate symbol</a> in 2016. This, of course, hasn't stopped public figures from sharing variations of Pepe's likeness in subversive social posts.</p>
Slenderman and the Boogaloo Bois<p>Indeed, these types of memes have a tricky way of moving from the shadows of the internet to the mainstream, picking up supporters along the way – including those who are willing to commit horrific crimes.</p><p>This phenomenon extends well beyond politics. Take, for example, the Slenderman. Created as a shadowy figure for online horror stories, Slenderman achieved mainstream popularity and a cult-like following. When two girls tried to <a href="https://www.washingtonpost.com/news/the-intersect/wp/2014/06/03/the-complete-terrifying-history-of-slender-man-the-internet-meme-that-compelled-two-12-year-olds-to-stab-their-friend/" target="_blank">stab their friend to death</a> to prove that Slenderman was real in 2014, the power of the meme to prompt violence became a national conversation. </p><p>Slenderman creator Victor Surge told Know Your Meme that <a href="https://knowyourmeme.com/memes/slender-man" target="_blank" rel="noopener noreferrer">he never thought</a> the character would spread beyond the fringe Something Awful forums. "An urban legend requires an audience ignorant of the origin of the legend. It needs unverifiable third and forth [sic] hand (or more) accounts to perpetuate the myth," he explained. "On the Internet, anyone is privy to its origins as evidenced by the very public Somethingawful thread. But what is funny is that despite this, it still spread. Internet memes are finicky things and by making something at the right place and time it can swell into an 'Internet Urban Legend.'"</p><p>While Slenderman is obviously a fiction, political memes walk many of the same fine lines – and have galvanized similarly obsessed followers to commit attacks in the real world.</p>
Understanding digital indoctrination<p><a href="http://www.ncri.io/" target="_blank" rel="noopener noreferrer">The Network Contagion Research Institute</a> (NCRI) uses advanced data analysis to expose hate on social media. The institute's scientists work with experts at the ADL's <a href="https://www.adl.org/who-we-are/our-organization/advocacy-centers/center-on-extremism" target="_blank">Center on Extremism </a>(COE) to track hateful propaganda online and offer strategies to combat this phenomenon. By better understanding how memes spread hateful propaganda, the NCRI is helping law enforcement, social media companies and citizens get better at preventing online chatter about violence from becoming real action.</p><p>Founded by Princeton's Dr. Joel Finkelstein, the NCRI is finding that the spread of hate online can be examined using epidemiological models of how viruses spread, only applied to language and ideas, much as Burroughs imagined decades ago. </p><p>Now that the internet has obviated the need for people to meet in person or communicate directly, recruiting and deploying members of violent groups is easier than ever before.</p><p>Before social media, as <a href="https://njjewishnews.timesofisrael.com/princeton-scientist-connects-web-hate-and-the-acts-it-spawns/" target="_blank">Finkelstein reminds us</a>, organizing underground hate groups was harder. "You would have to have an interpersonal organization to train and cultivate those people," he said. "With the advance of the internet, those people can find each other; what it lacks in depth it makes up in reach. The entire phenomenon can happen by itself without a trace of anyone being groomed."</p>
Can anything be done?<p>Memes generate hysteria in part by being outrageous enough for people to share – even the mainstream media. This is a concrete strategy called <a href="https://www.technologyreview.com/2019/10/24/132228/political-war-memes-disinformation/" target="_blank">amplification</a>. The extreme alt-right manages to reach mainstream audiences with racialist and supremacist memes, even as these messages are being denounced.</p><p>Content spreads virally whether those spreading it support the embedded ideology or not, making it difficult to intervene and prevent meme-incited violence.</p><p>Some people may unwittingly amplify messaging designed to spread violence, mistaking memes like ACAB (all cops are bastards) for harmless if somewhat dark humor.</p>
NCRI<p>So what can be done about the meme wars, and what onus is on law enforcement and Big Tech?</p><p>Finkelstein recommends a three-pronged approach to combating the violent outcome of subversive memes.</p><ol><li>Push for more stringent boundaries defining civility online using technology.</li><li>Educate courts, lawmakers and civil institutions about how extreme online communities operate and create better industry standards to regulate these groups.</li><li>Bring the federal government, including the FBI and Homeland Security, up to speed on the new reality so they can intervene as necessary.</li></ol><p>But not everyone thinks Big Tech or the law have much ground to stand on in combating viral hate. Tech companies haven't gone the mile in truly enforcing stricter standards for online content, but some companies are closing down accounts associated with <a href="https://www.voanews.com/silicon-valley-technology/can-shutting-down-online-hate-sites-curb-violence" target="_blank">extremist leaders' websites</a> and their movements. </p><p>But this is largely tilting at windmills. The proliferation of memes online spreading ideology virally might be too big to combat without dramatically limiting freedom of speech online. Facebook and YouTube have <a href="https://bigthink.com/politics-current-affairs/social-media-2020-us-election?rebelltitem=4#rebelltitem4" target="_self">banned thousands of profiles</a>, but there's no way of knowing how many remain – or are being added every day. This leaves the danger of meme radicalization and weaponization lurking beneath the surface even as we head into elections. </p>
What we stand to lose<p>Boogaloo and other seditious groups have been empowered by recent crises, including the pandemic and the anti-racist protests sweeping the country. Greater numbers of these community members are showing up at anti-quarantine protests and are disrupting other gatherings, and the press they're getting – this article included – is only bringing more attention to their violent messages.</p><p>Extremist memes continue to circulate, as America heads to the polls amidst a global pandemic and widespread civil unrest. It stands to reason that more blood will be spilled. Getting a handle on Twitter handles that spread hate and shutting down groups whose sole purpose is sowing the seeds of brutality is vital – the only question that remains is how. </p>
This medieval-themed meme highlights a shady yet all too common rhetorical move people make in arguments.
- The "Motte and Bailey Doctrine" was developed by philosopher Nicholas Shackel.
- It describes a rhetorical move in which an arguer advances an indefensible opinion, but when challenged falls back upon a similar yet easier-to-defend opinion.
- Motte-and-baileys have become a weapon of choice in political and culture-war arguments.
Credit: MotteAndBaileyMemes<p>The format is sort of confusing at first glance. But once understood, it offers a solid way to visualize bad arguments because it highlights a shady rhetorical move common in political discourse. </p><p>Here's an example in a hypothetical argument about homeopathic medicine:</p><p><strong>A: Homeopathic medicine can cure cancer.<br>B: There's no evidence showing homeopathy is effective.<br>A: Actually there are many ways for people to be healthy besides taking doctor-prescribed drugs.</strong></p><p><em></em>Spot it? Person A started with a bold and controversial opinion that's hard to defend (homeopathic medicine cures cancer). But when challenged, they retreated to an uncontroversial argument that's much easier to defend (prescription drugs aren't the only route to good health).</p><p>Person B would probably agree: Sure, there are many ways to be healthy besides drugs. But then, having deflected the first attack, Person A could go right back to arguing for homeopathic medicine as a cancer treatment.</p>
Credit: MotteAndBaileyMemes<p>In 2014, the psychiatrist Scott Alexander <a href="https://en.wikipedia.org/wiki/Slate_Star_Codex" target="_blank" rel="noopener noreferrer">(not his real name)</a> helped popularize the motte-and-bailey doctrine after writing about it on his blog <a href="https://slatestarcodex.com/2014/11/03/all-in-all-another-brick-in-the-motte/" target="_blank" rel="noopener noreferrer">Slate Star Codex</a>, a popular rationalist hub. Alexander wrote:</p><p style="margin-left: 20px;">"[The doctrine] draws its strength from people's usual failure to debate specific propositions rather than vague clouds of ideas. If I'm debating "does quackery cure cancer?", it might be easy to view that as a general case of the problem of "is quackery okay?" or "should quackery be illegal?", and from there it's easy to bring up the motte objection."</p><p>Overlapping with the Slate Star Codex community is a subreddit named after the doctrine called <a href="https://www.reddit.com/r/TheMotte/" target="_blank">r/TheMotte</a>, which describes itself as a place for people to "test their ideas in a court of people who don't all share the same biases." The subreddit calls on users to "always attempt to remain inside your defensible territory, even if you are not being pressed."<br></p><p>And then there are the memes. It's unclear who created the first one, or when, but since at least 2018 people have been posting motte-and-bailey memes to critique the often-shoddy ways in which people argue about issues ranging from immigration, to the problems of capitalism, to ideas about truth.</p>
Credit: MotteAndBaileyMemes<p>Motte-and-baileys aren't a new phenomenon. But it does seem like they're becoming a rhetorical weapon of choice in political and culture-war arguments.</p><p style="margin-left: 20px;">"I think [the motte-and-bailey doctrine] is a very useful concept to have in my arsenal of concepts to analyze what's going on," Kenny Easwaran, philosophy professor at Texas A&M University and co-editor of the Journal of Philosophical Logic, told <a href="https://www.realclearinvestigations.com/articles/2020/06/19/the_motte__bailey_political_joustings_deceptive_new_weapon_from_the_middle_ages_124084.html" target="_blank">Real Clear Investigations</a><em>. </em>"It's behavior we've seen, but we see so much more of it now."</p><p>It's hard to say why. You could blame the <a href="https://www.latimes.com/opinion/story/2020-08-13/nuance-memes-protests-ideas" target="_blank">fall of nuance</a>, <a href="https://news.gallup.com/opinion/polling-matters/268982/impact-increased-political-polarization.aspx" target="_blank">increasing political polarization</a> and the <a href="https://www.valleycenter.com/articles/the-year-of-motte-bailey-arguments/" target="_blank">absence of a middle ground</a>, and the <a href="https://www.openpolitics.com/tag/social-media-incentives/" target="_blank">tendency of social media to incentivize tribalism</a>, to name a few. </p><p>It's also worth considering how motte-and-baileys change when they include moral claims. For example, it's one thing to pull a motte-and-bailey to advance an argument about, say, 18th-century economic theory. But hot-button issues change the game. Take debates <a href="https://www.nytimes.com/2018/04/30/sports/track-gender-rules.html" target="_blank">about transgender and intersex athletes</a> as an example. </p><p>An argument might unfold like:</p><p><strong>A: Every transgender athlete should be able to compete in whichever gender category they identify with.<br>B: Wouldn't that give some athletes an unfair or even dangerous physical advantage?<br>A: Transgender people have been discriminated against for too long, it has to stop.</strong></p><p>Everyone agrees with the motte: transgender discrimination should stop. But notice how it becomes much easier to advance the bailey when the motte is a sensitive moral claim that's (rightfully) taboo to disagree with? </p><p>You might have good arguments against the bailey. But if it's tied to a sensitive motte, you might decide it's not even worth challenging. After all, it can be costly to your reputation to even look like you're challenging a sensitive motte, even if you're actually questioning the bailey in good faith.</p>
Credit: MotteAndBaileyMemes<p><br>You can see this play out in political arguments. For example, a Trump supporter might argue for unprecedentedly harsh immigration policies at the U.S./Mexico border. (That's the bailey). If someone challenges that position, the Trump supporter could shame them for being unpatriotic, considering immigration reform is part of the Make America Great Again platform, and who doesn't want to make America great (motte)?</p><p>Similarly, someone might question Black Lives Matter's goal of disrupting "the Western-prescribed nuclear family structure requirement" (bailey). They might get a reply like: "What, are you trying to argue that Black lives don't matter (motte)?"</p>
Credit: MotteAndBaileyMemes<p>It might sound like motte-and-baileys are always easy to spot. But as Alexander wrote on <a href="https://slatestarcodex.com/2014/11/03/all-in-all-another-brick-in-the-motte/" target="_blank">Slate Star Codex</a>, "all fallacies sound that way when you're thinking about them."<br></p>
We are what we are because of genes; we are who we are because of memes. Philosopher Daniel Dennett muses on an idea put forward by Richard Dawkins in 1976.
Ever wondered where the word ‘meme’ comes from? Philosopher and cognitive scientist Daniel Dennett explains the term, coined by Richard Dawkins in his 1976 book The Selfish Gene, and its effects on our lives and history. How did we, as a species, become what we are – or more relevantly who we are? Natural selection and genetic evolution have made our physical bodies, but we are so much more than a collection of cells. We are also a conscious community, with language, music, cooking, art, poetry, dance, rituals, and humor. Dennett explains how these behaviors are the product of our cultural evolution. Memes are cultural replicators that spread like viruses, and only the most advantageous – or "the fittest" – of them survive. Daniel Dennett's most recent book is From Bacteria to Bach and Back: The Evolution of Minds.