Of Political Orgasms and the Scientific Method

This week’s theme is epistemological unease in the sciences: Complaints in a number of disciplines that studies didn’t really find the effects they’re reporting. One reason for these worries is that many studies nowadays are never repeated. So today I’m going to consciously and rationally resist the innate bias of the human mind for novelty and against systematic repetition. Let us instead revisit the subject of politically motivated orgasm.

Yup, it’s back to porn. But first some adults-only epistemology. It is a problem that experiments aren’t often repeated, and that journals aren’t prone to publish “mere replication” papers. Replication is supposed to be science’s basic reality check: if my lab results arose from a real phenomenon, then you, by doing what I did, should get the same result. If no one ever repeats my work, this check doesn’t happen. The paper stands on the basis of one success. And if its claim is sexy, we media types add publicity’s thrills to the authority of scientific publication. We write articles declaring that “x study found y,” and never take a second look.

That matters because scientists are social creatures like the rest of us, and are swayed by the “popular press.” In 1991, for instance David P. Phillips and his colleagues found that The New York Times had a big impact on how often scientists cited other scientists’ work: In a decade’s worth of studies published in The New England Journal of Medicine, those the Times wrote about received 72.8 percent more citations than others that the newspaper ignored. (When the Times was published but not circulating because of three-month strike, the citation effect disappeared. That’s evidence that Phillips et al. found a media effect, not just a reflection of the importance of some studies over others.)

So maybe replication deserves more respect than it gets in both the professional and general-interest press. That’s why I want to return to a paper I discussed a few weeks back. (This is where the porn comes in.)

In early October I wrote about Patrick and Charlotte Markey’s assertion that Google searches for smut go up after American elections—among men who voted for the winners. (My BT blog colleague Marina Adshade also picked up on it.) The Markeys believe this reflects a testosterone boost in men whose side has won a contest. Thinking that a testostesterone increase would be reflected in Internet searches for porn, they reasoned that when Republicans triumph, smut-googling should rise in Red states (defined as those which voted Republican by a more than 5 percent margin in the last five Presidential elections); when Democrats win, Blue states (same definition, but for the other party preference) would show the increase. And the states that aren’t so predictably Blue or Red, the “swing states,” wouldn’t show the effect.

Their study compared each state’s volume of porn searches after voting to its average for the year, to discern if, post-election, its citizens had been searching more or less than usual. On that basis, their analysis found the effect they predicted: Red state jumps in 2004, after Republicans won the Presidential election, and the same effect in Blue states after the Democratic triumphs of 2006 and 2008.

If they’re right, of course, then the November 2010 American elections should show the same pattern—after the Democratic “shellacking,” porn searches should have jumped in “Red” states. So yesterday I emailed Patrick Markey to ask if that prediction held up.

It did. Last November 8, the Markeys gathered data on Google Analytics on searches for common porn words in the days before and after the November 2 election. They haven’t sent their analysis out to journals yet (and who knows if it will be published, as it is “mere” replication). But they did write it up. As they predicted, the 2010 Republican triumph was followed by a pattern that is the mirror image of what happened after Democrats swept through in 2006: Porn searches were higher in Red states, lower in Blue.

Now, this post would be narratively more exciting if the effect hadn’t held up; it would have surprise and suspense (can they save their hypothesis?). But the scientific method is supposed to help us counteract our story-telling biases. Finding the same effect again is important; it adds to the probability that the effect isn’t an accident.

Of course that doesn’t mean the Markeys’ theory—that an evolutionary explanation of testosterone swings in birds will also apply to humans—is right. Something else could be causing the result they are getting, or their method itself could be flawed. (To address the former someone would have to test some alternative hypotheses for the data and show why theirs is best; to address the latter, a completely different method should get compatible results.) Still, it’s significant that their published study was replicated. Over time, I hope to revisit other ideas and results I’ve described, to see how they’re faring.

Phillips, D., Kanter, E., Bednarczyk, B., & Tastad, P. (1991). Importance of the Lay Press in the Transmission of Medical Knowledge to the Scientific Community New England Journal of Medicine, 325 (16), 1180-1183 DOI: 10.1056/NEJM199110173251620

Markey, P., & Markey, C. (2010). Changes in pornography-seeking behaviors following political elections: an examination of the challenge hypothesis Evolution and Human Behavior, 31 (6), 442-446 DOI: 10.1016/j.evolhumbehav.2010.06.004