What responsibility should government authorities and Big Tech take in policing the spread of sedition-oriented content?
- It can be hard to believe that comical images online are enough to rile people up enough that they'll actually attack.
- Originating in the darker corners of the internet, Bugaloo is now prominent on mainstream online platforms like Facebook and Instagram.
- The Network Contagion Research Institute's recent series of Contagion and Ideology Reports uses machine learning to examine how memes spread.
There has been talk lately about fringe hate groups using memes to incite real world violence, but many prefer to casually dismiss this narrative as hysteria. It can be hard to believe that comical images online are enough to rile people up enough that they'll actually attack, and for most of us, this is thankfully true.
If you spend your time scrolling through pumpkin spice recipes and cat videos, you're probably not the target of a systematic hate campaign. But that doesn't mean these campaigns don't exist. Not every Facebook user is susceptible to radicalization, but memetic warfare is increasingly being recognized as an important element of counter-governmental sedition and racist propaganda.
Studies have long shown that exposure to hate makes it easier for people to hate – and to act on their hate. If there is an organized effort by hate groups to manipulate people with memes, then these visuals can actually be influencing people to perpetrate violent attacks.
What kind of memes were used on the Facebook Event listing that the Kenosha Guard used to mobilize Kyle Rittenhouse?
Many netizens appreciate the dangers and want social media platforms to do much more to curb the spread of hate across their networks. From Snapchat to Gab, TikTok to WhatsApp, memes are fueling hatred and inciting violence, and more Americans are asking what responsibility these companies should take in policing the spread of harmful content.
What’s in a meme?
The Network Contagion Research Institute's recent series of Contagion and Ideology Reports leverages machine learning to examine how memes spread. The idea is to unearth a better understanding of the role memes play in encouraging real-world violence.
So what exactly is a meme, and how did it become a tool for weaponizing the web? Originally coined by Oxford biologist Richard Dawkins in his 1976 book "The Selfish Gene," the term is defined there as "a unit of cultural transmission" that spreads like a virus from host to host and conveys an idea which changes the host's worldview.
Dawkins echoes "Naked Lunch" author William Burroughs's concept of written language as a virus that infects the reader and consequently builds realities that are brought to fruition through the act of speech. In this sense, a meme is an idea that spreads by iterative, collaborative imitation from person to person within a culture and carries symbolic meaning.
The militant alt-right's use of internet memes follows this pattern. They are carefully designed on underground social media using codes, before attaining approval by like-minded users that disseminate these messages on mainstream platforms like Twitter or Facebook.
Sometimes these messages are lifted from innocuous sources and altered to convey hate. Take for example Pepe the Frog. Initially created by cartoonist Matt Furie as a mascot for slackers, Pepe was appropriated and altered by racists and homophobes until the Anti-Defamation League branded the frog as a hate symbol in 2016. This, of course, hasn't stopped public figures from sharing variations of Pepe's likeness in subversive social posts.
The dissemination of hateful memes is often coordinated by an underground movement, so even its uptake by mainstream internet users and media outlets is claimed as a victory by a subculture that prides itself on its trolling.
Slenderman and the Boogaloo Bois
Indeed, these types of memes have a tricky way of moving from the shadows of the internet to the mainstream, picking up supporters along the way – including those who are willing to commit horrific crimes.
This phenomenon extends well beyond politics. Take, for example, the Slenderman. Created as a shadowy figure for online horror stories, Slenderman achieved mainstream popularity and a cult-like following. When two girls tried to stab their friend to death to prove that Slenderman was real in 2014, the power of the meme to prompt violence became a national conversation.
Slenderman creator Victor Surge told Know Your Meme that he never thought the character would spread beyond the fringe Something Awful forums. "An urban legend requires an audience ignorant of the origin of the legend. It needs unverifiable third and forth [sic] hand (or more) accounts to perpetuate the myth," he explained. "On the Internet, anyone is privy to its origins as evidenced by the very public Somethingawful thread. But what is funny is that despite this, it still spread. Internet memes are finicky things and by making something at the right place and time it can swell into an 'Internet Urban Legend.'"
While Slenderman is obviously a fiction, political memes walk many of the same fine lines – and have galvanized similarly obsessed followers to commit attacks in the real world.
Take, for example, the Boogaloo movement. This extremist militia has strong online roots and uses memes to incite violent insurrection and terror against the government and law enforcement. Originating in the darker corners of the internet, Bugaloo is now prominent on mainstream online platforms like Facebook and Instagram. Boogaloo followers share how to build explosives and 3D-printed firearms, spread encrypted messages and distribute violent propaganda.
Boogaloo's intentions seem to be twofold: recruit new followers and communicate tactical information to incumbent members. While finding more people to adopt their ideology is one thing, there is a strong possibility that Boogaloo memes and messaging can be used to trigger sleeper cells to violent action.
Boogaloo isn't alone. The use of memes by groups like the Proud Boys, which operate on an ideology consisting of both symbolic and physical violence, is spreading across America – especially since the start of the pandemic.
Understanding digital indoctrination
The Network Contagion Research Institute (NCRI) uses advanced data analysis to expose hate on social media. The institute's scientists work with experts at the ADL's Center on Extremism (COE) to track hateful propaganda online and offer strategies to combat this phenomenon. By better understanding how memes spread hateful propaganda, the NCRI is helping law enforcement, social media companies and citizens get better at preventing online chatter about violence from becoming real action.
Founded by Princeton's Dr. Joel Finkelstein, the NCRI is finding that the spread of hate online can be examined using epidemiological models of how viruses spread, only applied to language and ideas, much as Burroughs imagined decades ago.
Now that the internet has obviated the need for people to meet in person or communicate directly, recruiting and deploying members of violent groups is easier than ever before.
Before social media, as Finkelstein reminds us, organizing underground hate groups was harder. "You would have to have an interpersonal organization to train and cultivate those people," he said. "With the advance of the internet, those people can find each other; what it lacks in depth it makes up in reach. The entire phenomenon can happen by itself without a trace of anyone being groomed."
Can anything be done?
Memes generate hysteria in part by being outrageous enough for people to share – even the mainstream media. This is a concrete strategy called amplification. The extreme alt-right manages to reach mainstream audiences with racialist and supremacist memes, even as these messages are being denounced.
Content spreads virally whether those spreading it support the embedded ideology or not, making it difficult to intervene and prevent meme-incited violence.
Some people may unwittingly amplify messaging designed to spread violence, mistaking memes like ACAB (all cops are bastards) for harmless if somewhat dark humor.
So what can be done about the meme wars, and what onus is on law enforcement and Big Tech?
Finkelstein recommends a three-pronged approach to combating the violent outcome of subversive memes.
- Push for more stringent boundaries defining civility online using technology.
- Educate courts, lawmakers and civil institutions about how extreme online communities operate and create better industry standards to regulate these groups.
- Bring the federal government, including the FBI and Homeland Security, up to speed on the new reality so they can intervene as necessary.
But not everyone thinks Big Tech or the law have much ground to stand on in combating viral hate. Tech companies haven't gone the mile in truly enforcing stricter standards for online content, but some companies are closing down accounts associated with extremist leaders' websites and their movements.
But this is largely tilting at windmills. The proliferation of memes online spreading ideology virally might be too big to combat without dramatically limiting freedom of speech online. Facebook and YouTube have banned thousands of profiles, but there's no way of knowing how many remain – or are being added every day. This leaves the danger of meme radicalization and weaponization lurking beneath the surface even as we head into elections.
What we stand to lose
Boogaloo and other seditious groups have been empowered by recent crises, including the pandemic and the anti-racist protests sweeping the country. Greater numbers of these community members are showing up at anti-quarantine protests and are disrupting other gatherings, and the press they're getting – this article included – is only bringing more attention to their violent messages.
Extremist memes continue to circulate, as America heads to the polls amidst a global pandemic and widespread civil unrest. It stands to reason that more blood will be spilled. Getting a handle on Twitter handles that spread hate and shutting down groups whose sole purpose is sowing the seeds of brutality is vital – the only question that remains is how.
While it can often be tempting to unfollow friends who have differing political views than you, one philosopher tells us why we should embrace, rather than shun, such challenges to our worldviews.
We've all done it, unfollowed that conspiracy spouting friend we have on Facebook rather than endure one more post about how the Earth is flat and Obama was born on Neptune. Sometimes we go a step further, removing those with opposing political views from our friend lists. After all, social media is for fun! Why should I have to see my nutty uncle's viewpoints when I am looking for pictures of cute cats?
This problem has been discussed here at Big Think before and Eli Pariser gave an excellent Ted Talk on the subject. While the issue is often viewed as one of immaturity or tribalism, it has another dimension. Rarely do we think of it as the problem of our moral development it is.
It is a major part of that development, at least according to philosopher John Stuart Mill.
In On Liberty, his defense of a liberal democratic society, Mill argues brilliantly for the necessity of the freedom of speech. While he references our dependence on it for the success of a free society, he also points out it has another, vital, function; it allows us to make steady progress towards the truth that is otherwise impossible.
Mill suggests that any viewpoint can be one of three things; true, false, or a little of both. He further reminds us that humans are never sure of anything and that we are often wrong when it comes to knowing if our ideas are the whole truth or not. He explains that:
"First, if any opinion is compelled to silence, that opinion may, for aught we can certainly know, be true. To deny this is to assume our own infallibility. Secondly, though the silenced opinion be an error, it may, and very commonly does, contain a portion of truth; and since the general or prevailing opinion on any subject is rarely or never the whole truth, it is only by the collision of adverse opinions that the remainder of the truth has any chance of being supplied."
When you are living in your bubble falsehoods don't get checked. New truths are never considered if you unfollow those who offer them. Comforting lies can get passed from person to person without anybody noticing that they cannot be true. Progress towards better ideas can be stalled as there is no active questioning of the ideas you have.
To be fair, we have all felt this woman's rage at a post before. Maybe some people should be unfollowed?
Of course, sometimes we know which side is correct with as much certainty as is possible. If you have a Facebook friend who is a flat-earther, it is clear that they are very wrong when they try to explain that the Earth is flat. Censoring them from your feed can't be too terrible, right? Mill disagrees, he argues that testing our true beliefs with rational debate is essential.
"Thirdly, even if the received opinion be not only true, but the whole truth; unless it is suffered to be, and actually is, vigorously and earnestly contested, it will, by most of those who receive it, be held in the manner of a prejudice, with little comprehension or feeling of its rational grounds. And not only this, but, fourthly, the meaning of the doctrine itself will be in danger of being lost, or enfeebled, and deprived of its vital effect on the character and conduct: the dogma becoming a mere formal profession, inefficacious for good, but cumbering the ground, and preventing the growth of any real and heartfelt conviction, from reason or personal experience."
If we don't occasionally interact with objections to our ideas, even if our thoughts are already correct, they will lose a bit of their punch and decay into mere dogma. Debate keeps our ideas fresh; it reminds us of why we support one side over the other and stimulates our intellect. For Mill, who saw personal development as a stop on the road to a just and happy society, this is all but a moral imperative.
So, how can we get back to challenging our ideas? How can we pop the bubble?
Luckily, there are lots of ways to do it. While not all of us can bear to see our family’s political posts every time we log in, we can still expose ourselves to challenging viewpoints that keep us from becoming too trapped in our echo boxes.
The KIND foundation has the Pop Your Bubble campaign which adds a simple extension to your browser that determines what kind of person you are, demographically speaking. It then suggests pages for you to follow on Facebook that differ from what pages you are likely to see already. While the effectiveness of it seems variable, it is a simple way to add new viewpoints to your feed.
There is also the option of patience. Waiting for your friends with differing viewpoints to post, viewing those posts, and then getting on with your life is a rather simple, yet effective, method. Some people are going to disagree with you, but this is no reason to shut them out of your circle. For each political post you manage to dodge by unfollowing someone, an apolitical post about their newborn could be missed as well.
While it can be tempting to banish opposing viewpoints from your screen, it isn't good for us. Preventing yourself from seeing different opinions, even more than the internet does already without you knowing, can reduce your worldview to mere dogma and even your ability to form a reasoned opinion. While enduring the posts of family members who tend to make Thanksgiving political isn't always enjoyable, it does pay off in the long run.