Would Online Open Review Lead to an Academic Popularity Contest?

Double blind peer-review in science and other fields has been the norm for decades. Now some scholars, as featured at the NY Times this week, are arguing that peer-review needs to move online, eliciting unblinded judgments from a greater number and diversity of experts. But would removing the anonymity of peer-review and widening the scope of participation play into the hands of several common human biases?


Following the New York Times story, I discussed the advantages of open-review and open-access publishing, noting though some of the challenges in moving academic fields and universities in this direction.  In reaction, my colleague at American University Wendy Melillo Farrill offered several important insights, referencing in part the arguments made in Nicholas Carr's The Shallows: What the Internet is Doing to Our Brains. I asked Wendy to share her insights with readers in a guest post.--Matthew Nisbet

I would like to add one more piece to Matt's suggestion about peer review in the digital age. I just finished reading Nicholas Carr's The Shallows: What the Internet is Doing to Our Brains.

At the end of the book, Carr refers to a study by James Evans, a sociologist at the University of Chicago. Evans analyzed 34 million scholarly articles published in academic journals from 1945 through 2005. He looked at the citations in the articles to determine if there were changes in the patterns of citation and research since academic journals shifted from being published in print to being published online.

The assumption was that that making journals available on the Web would "significantly broaden the scope of scholarly research, leading to a much more diverse set of citations," as Carr put it in his book. But that's not what happened in the study Evans did.

Since journals moved online, scholars cited fewer journals than they had previously. Scholars also cited more recent articles with increasing frequency. The broadening of information led to a "narrowing of science and scholarship," Evans found. When Evans discussed his findings in a 2008 article in Science, he explained his counter-intuitive results by suggesting that search engines, which emphasize popular search findings, quickly establish and reinforce a consensus about what information is important and what isn't. The ease of clicking on hyperlinks leads online researchers to "bypass many of the marginally related articles that print researchers" would routinely skim as they browsed through journal articles and books in a library.

One question that should be addressed in this discussion about the movement described in The Times article should be: Would establishing an online, more open peer review system mimic some of the patterns Evans found? Would the most popular academic submissions get the most votes from the reviewers simply because they were popular? Does all of this actually lead to more narrow scholarship or to better scholarship?

--Wendy Melillo Farrill

--

 

China’s artificial sun reaches fusion temperature: 100 million degrees

In a breakthrough for nuclear fusion research, scientists at China's Experimental Advanced Superconducting Tokamak (EAST) reactor have produced temperatures necessary for nuclear fusion on Earth.

Credit: EAST Team
Surprising Science
  • The EAST reactor was able to heat hydrogen to temperatures exceeding 100 million degrees Celsius.
  • Nuclear fusion could someday provide the planet with a virtually limitless supply of clean energy.
  • Still, scientists have many other obstacles to pass before fusion technology becomes a viable energy source.
Keep reading Show less

A dark matter hurricane is crashing into Earth

Giving our solar system a "slap in the face."

Surprising Science
  • A stream of galactic debris is hurtling at us, pulling dark matter along with it
  • It's traveling so quickly it's been described as a hurricane of dark matter
  • Scientists are excited to set their particle detectors at the onslffaught
Keep reading Show less

Here's how diverse the 116th Congress is set to become

The 116th Congress is set to break records in term of diversity among its lawmakers, though those changes are coming almost entirely from Democrats.

(Photo: MANDEL NGAN/AFP/Getty Images)
Politics & Current Affairs
  • Women and nonwhite candidates made record gains in the 2018 midterms.
  • In total, almost half of the newly elected Congressional representatives are not white men.
  • Those changes come almost entirely from Democrats; Republican members-elect are all white men except for one woman.
Keep reading Show less