Once a week.
Subscribe to our weekly newsletter.
Was It Right to Fool Millions of People into Thinking Chocolate Helps You Lose Weight?
A journalist tricked news outlets into reporting a bogus study to demonstrate the sorry state of science journalism, but was the sting operation ethical?
If you opened a newspaper at the end of March, it is possible that you may have noticed an article telling you that chocolate makes you lose weight. The more skeptical amongst you probably already know that these kinds of nutritional claims are very often not worth the paper they are written on; nutrition is by its very nature an inexact science requiring very large, very well-controlled studies that sadly are the exception rather than the rule. As John Ioannidis once wrote in the British Medical Journal:
"Almost every single nutrient imaginable has peer reviewed publications associating it with almost any outcome"
The paper cited in the coverage in May however was different from the countless similar studies we see reported in newspapers on a regular basis; this time the misinformation was intentional. Journalist Johannes Bohannon (real name John) conducted the study to demonstrate how statistically significant — but yet completely meaningless — findings can be extracted from a poorly conducted study. The experiment wasn't really about chocolate at all; it was a test to see whether professional journalists would buy an obviously flawed paper.
Unsurprisingly perhaps, they bought it in spades. The paper made the cover story of Bild - Europe's most read newspaper, and also featured in "the Daily Star, the Irish Examiner, Cosmopolitan’s German website, the Times of India, both the German and Indian site of The Huffington Post, and even television news in Texas and an Australian morning talk show". While the study was submitted to a newswire distributed to 5,000 journalists, not one journalist sought to expose the obviously ridiculous paper or the headlines created by it.
Though not at all dissimilar to a great deal of poorly conducted nutrition research that makes big claims based on small sample and effect sizes; this particular paper was actually published in a bogus "pay to play" journal that conducted no peer review whatsoever. This is far and away my biggest criticism of the sting, which, had it also been submitted to bona fide nutrition journals, could have truly shaken the field, rather than merely rattled journalists. I made a very similar criticism of the same journalist's "sting operation" against open access journals in 2013, in which a bogus paper was submitted to hundreds of open-access journals but no traditional paywalled journals.
The response online has ranged from outrage to glee. Like many, my initial reaction was one of mixed feelings. On the one hand, the study knowingly put misinformation into the public domain on a grand scale, misinformation that will likely impact the lives of countless people who never had the opportunity to consent and who may well never know that they were part of an enormous social experiment. For all we know, the readers of the news reports may continue eating chocolate bars with reckless abandon until they die of a heart attack, obese and alone. Full disclaimer: I am being facetious.
On the other hand, can we really honestly say that this one news story could have any discernible negative impact amongst the continual Niagara Falls of junk science stories just like it? Anyone who might have believed this story enough to change their diet because of it, frankly, has bigger problems. Problems that this sting may help tackle. One British newspaper, the Daily Mail, known for both its cavalier attitude to misreporting science and for running the most widely read news website on the planet, has suggested over the years that so many hundreds of different things both cause and prevent cancer that an entire website is dedicated to tracking its progress. Chocolate by the way, is already in there nestled in between chillis, Chinese medicine, chips, Chlamydia, chlorine, cholesterol, and circumcision:
If the Bohannon paper wasn't just like hundreds of other papers that get published every month and the stories covering it weren't just like the stories we see in the papers day in, day out, then the paper might be cause for greater concern. Unlike the sea of poorly conducted science we see in the papers every day, the fallout from Bohannon's paper at least has the potential to shock people into thinking differently and have a real impact.
On balance, it seems reasonable to assume that the overall impact of Bohannon's paper will be resoundingly positive. Scientists who read the story may decide to design experiments that are well-controlled enough and with big enough samples to provide valid results that don't risk becoming an embarrassment. Journalists may recognise that their industry has a problem and learn to treat scientific findings with an appropriate level of skepticism; pigs might fly, but that's a topic for another blog post (see you next week).
Most importantly of all, within only 48 hours of publication, Bohannon's explanation for his study had already been read hundreds of thousands of times. We can only hope that the people reading his article will take home lessons that are far more valuable and meaningful than any two-bit breaking news story ever could be: That just because a study is published in a scientific journal doesn't mean its results are valid. Just because a study's finding is statistically significant, doesn't mean it didn't occur by chance. That a single study on its own should never cause us to drastically change our behaviour, especially if its sample is small and the effect size is weak.
Even if the only thing people take away from this affair is that they cannot trust newspapers to communicate science to them, then this will be a very good thing. For so many people it seems that the only time they are genuinely skeptical of what they read in the papers is on the first of April. For this reason I would argue that claims that the sting is "shameful" for "undermining (the public's) trust in both journalism and science" couldn't be wider off the mark. The last thing we want is a public that blindly trusts journalists to report science accurately and uncritically. A culture of informed skepticism where people are inclined to actually check what they read in the science pages is far more healthy.
None of this changes the fact that the study crossed a big red line in research ethics, a point that, like so many things in life, was well-made by Ed Yong on Twitter:
It is entirely unethical for a journalist to deliberately introduce false info into the media to prove a point, IDGAF what that point is
— Ed Yong (@edyong209) May 28, 2015
"Deepfakes" and "cheap fakes" are becoming strikingly convincing — even ones generated on freely available apps.
- A writer named Magdalene Visaggio recently used FaceApp and Airbrush to generate convincing portraits of early U.S. presidents.
- "Deepfake" technology has improved drastically in recent years, and some countries are already experiencing how it can weaponized for political purposes.
- It's currently unknown whether it'll be possible to develop technology that can quickly and accurately determine whether a given video is real or fake.
The future of deepfakes<p>In 2018, Gabon's president Ali Bongo had been out of the country for months receiving medical treatment. After Bongo hadn't been seen in public for months, rumors began swirling about his condition. Some suggested Bongo might even be dead. In response, Bongo's administration released a video that seemed to show the president addressing the nation.</p><p>But the <a href="https://www.facebook.com/watch/?v=324528215059254" target="_blank">video</a> is strange, appearing choppy and blurry in parts. After political opponents declared the video to be a deepfake, Gabon's military attempted an unsuccessful coup. What's striking about the story is that, to this day, experts in the field of deepfakes can't conclusively verify whether the video was real. </p><p>The uncertainty and confusion generated by deepfakes poses a "global problem," according to a <a href="https://www.brookings.edu/research/is-seeing-still-believing-the-deepfake-challenge-to-truth-in-politics/#cancel" target="_blank">2020 report from The Brookings Institution</a>. In 2018, the U.S. Department of Defense released some of the first tools able to successfully detect deepfake videos. The problem, however, is that deepfake technology keeps improving, meaning forensic approaches may forever be one step behind the most sophisticated forms of deepfakes. </p><p>As the 2020 report noted, even if the private sector or governments create technology to identify deepfakes, they will:</p><p style="margin-left: 20px;">"...operate more slowly than the generation of these fakes, allowing false representations to dominate the media landscape for days or even weeks. "A lie can go halfway around the world before the truth can get its shoes on," warns David Doermann, the director of the Artificial Intelligence Institute at the University of Buffalo. And if defensive methods yield results short of certainty, as many will, technology companies will be hesitant to label the likely misrepresentations as fakes."</p>
Context is everything.
The COVID-19 pandemic has introduced a number of new behaviours into daily routines, like physical distancing, mask-wearing and hand sanitizing. Meanwhile, many old behaviours such as attending events, eating out and seeing friends have been put on hold.
A new study looks at how images of coffee's origins affect the perception of its premiumness and quality.
- Images can affect how people perceive the quality of a product.
- In a new study, researchers show using virtual reality that images of farms positively influence the subjects' experience of coffee.
- The results provide insights on the psychology and power of marketing.