To answer this question, a six-member research team from the Universities of Innsbruck (Jürgen Huber, Rudolf Kerschbamer and Christian König-Kersting) and Graz (Stefan Palan) and Chapman University (Sabiou Inoua and Vernon Smith) conducted a simple experiment:
Vernon Smith, winner of the 2002 Nobel Prize in Economics and professor at Chapman University, wrote a scientific article in the field of finance together with Sabiou Inoua, a junior researcher at Chapman University. The two authors submitted the article to the Journal of Behavioral and Experimental Finance for publication. Normally, such an article would be sent out to two or three "peers" (other scientists working on similar research topics) for review. But Stefan Palan, Associate Professor at the University of Graz, editor of the journal and member of the research team, sent the article to a total of more than 3300 peer reviewers.
While all reviewers received exactly the same article for review, they received different information about the authors of the article. Some of the reviewers received no information about the authors, others were told that one of the authors was the young scientist (almost certainly unknown to them), still others were told that one of the authors was the Nobel Prize winner.
The results of the study show a clear pattern: of the peer reviewers who had no information about the authors, about 25% recommended a minor revision or immediate acceptance of the article for publication; among those peer reviewers who were informed that one of the authors was the junior scientist, this proportion fell to below 10%; among those reviewers who were told that one of the authors was the Nobel laureate, it rose to almost 60%. At the negative end of the recommendation spectrum, a similar picture emerges: of those peer reviewers who had no information about the author, almost 50% recommended not publishing the article; among those reviewers who were informed that one of the authors of the article was the junior researcher, this proportion even rose to over 65%; of the reviewers who were informed that one of the authors was the Nobel Prize winner, only 23% recommended rejecting the article.
"Our results clearly show that the different information about the author strongly influences the evaluation of the quality of the research article," says Jürgen Huber, professor at the Institute of Banking and Finance at the University of Innsbruck and co-author of the study. Rudolf Kerschbamer, professor at the Institute of Economic Theory at the University of Innsbruck and also co-author of the study, traces the source of the different evaluations to the so-called "halo effect": "The halo effect is a cognitive bias known from social psychology, whereby evaluators tend to assess actions and works of persons of whom they have a positive impression more favorably than those of unknown persons or persons of whom they have a negative impression." The fact that the Nobel Prize winner has a "Western-sounding" name, while the young researcher has a "foreign-sounding" one, could also have played a role for some reviewers.
The research team sees the results as an important impetus to rethink the peer review process. "As scientists, we are constantly working to improve our methods and processes. Especially in the academic world, the results of our current study have therefore met with great interest", emphasises Christian König-Kersting, research associate at the Institute of Banking and Finance at the University of Innsbruck. On the important question of whether the peer review process should be anonymised or whether the authors' names should be disclosed to the reviewers, these results clearly speak in favour of an anonymised process.
Study in PNAS
Report in Nature
Report in Science
Report in The Economist
Report in Le Monde
Report in Sina.cn
Report inORF
Report in Der Standard
Report in Die Presse
Report in Kurier
Tweet on Twitter
Tweet on Twitter