Would Online Open Review Lead to an Academic Popularity Contest?

Double blind peer-review in science and other fields has been the norm for decades. Now some scholars, as featured at the NY Times this week, are arguing that peer-review needs to move online, eliciting unblinded judgments from a greater number and diversity of experts. But would removing the anonymity of peer-review and widening the scope of participation play into the hands of several common human biases?


Following the New York Times story, I discussed the advantages of open-review and open-access publishing, noting though some of the challenges in moving academic fields and universities in this direction.  In reaction, my colleague at American University Wendy Melillo Farrill offered several important insights, referencing in part the arguments made in Nicholas Carr's The Shallows: What the Internet is Doing to Our Brains. I asked Wendy to share her insights with readers in a guest post.--Matthew Nisbet

I would like to add one more piece to Matt's suggestion about peer review in the digital age. I just finished reading Nicholas Carr's The Shallows: What the Internet is Doing to Our Brains.

At the end of the book, Carr refers to a study by James Evans, a sociologist at the University of Chicago. Evans analyzed 34 million scholarly articles published in academic journals from 1945 through 2005. He looked at the citations in the articles to determine if there were changes in the patterns of citation and research since academic journals shifted from being published in print to being published online.

The assumption was that that making journals available on the Web would "significantly broaden the scope of scholarly research, leading to a much more diverse set of citations," as Carr put it in his book. But that's not what happened in the study Evans did.

Since journals moved online, scholars cited fewer journals than they had previously. Scholars also cited more recent articles with increasing frequency. The broadening of information led to a "narrowing of science and scholarship," Evans found. When Evans discussed his findings in a 2008 article in Science, he explained his counter-intuitive results by suggesting that search engines, which emphasize popular search findings, quickly establish and reinforce a consensus about what information is important and what isn't. The ease of clicking on hyperlinks leads online researchers to "bypass many of the marginally related articles that print researchers" would routinely skim as they browsed through journal articles and books in a library.

One question that should be addressed in this discussion about the movement described in The Times article should be: Would establishing an online, more open peer review system mimic some of the patterns Evans found? Would the most popular academic submissions get the most votes from the reviewers simply because they were popular? Does all of this actually lead to more narrow scholarship or to better scholarship?

--Wendy Melillo Farrill

--

 

LinkedIn meets Tinder in this mindful networking app

Swipe right to make the connections that could change your career.

Getty Images
Sponsored
Swipe right. Match. Meet over coffee or set up a call.

No, we aren't talking about Tinder. Introducing Shapr, a free app that helps people with synergistic professional goals and skill sets easily meet and collaborate.

Keep reading Show less

4 reasons Martin Luther King, Jr. fought for universal basic income

In his final years, Martin Luther King, Jr. become increasingly focused on the problem of poverty in America.

(Photo by J. Wilds/Keystone/Getty Images)
Politics & Current Affairs
  • Despite being widely known for his leadership role in the American civil rights movement, Martin Luther King, Jr. also played a central role in organizing the Poor People's Campaign of 1968.
  • The campaign was one of the first to demand a guaranteed income for all poor families in America.
  • Today, the idea of a universal basic income is increasingly popular, and King's arguments in support of the policy still make a good case some 50 years later.
Keep reading Show less

Dead – yes, dead – tardigrade found beneath Antarctica

A completely unexpected discovery beneath the ice.

(Goldstein Lab/Wkikpedia/Tigerspaws/Big Think)
Surprising Science
  • Scientists find remains of a tardigrade and crustaceans in a deep, frozen Antarctic lake.
  • The creatures' origin is unknown, and further study is ongoing.
  • Biology speaks up about Antarctica's history.
Keep reading Show less

Why I wear my life on my skin

For Damien Echols, tattoos are part of his existential armor.

Videos
  • In prison Damien Echols was known by his number SK931, not his name, and had his hair sheared off. Stripped of his identity, the only thing he had left was his skin.
  • This is why he began tattooing things that are meaningful to him — to carry a "suit of armor" made up the images of the people and objects that have significance to him, from his friends to talismans.
  • Echols believes that all places are imbued with divinity: "If you interact with New York City as if there's an intelligence behind... then it will behave towards you the same way."
Keep reading Show less