New method of scientific reviewing developed at the Max-Planck-Institut für Kohlenforschung: “intelligent-crowd-reviewing” accelerates and improves the review process
Prof. List and his Ph.D. student Denis Höfler have developed "intelligent-crowd-reviewing" as a new online-collaboration-tool, which offers many advantages to classical peer reviewing and benefits from referee interaction.
You can find the nature article here:www.nature.com/news/crowd-based-peer-review-can-be-good-and-fast-1.22072
Peer reviewing is at the very foundation of the scientific process. It is the door-keeper of scientific merit. The careers of scientists decisively depend on the work they develop and where it is published. Up to now, scientific journals use peer reviewing, in which a small number of referees from the same research area judge the quality of a manuscript. But such reviews can be vague, sometimes biased, and take time, says Benjamin List, Director at the Max-Planck-Institut für Kohlenforschung and editor-in-chief of Synlett, an international journal on chemical synthesis. In response, List and his Ph.D. student Denis Höfler have developed "intelligent-crowd-reviewing" as a new online-collaboration-tool, which offers many advantages to classical peer reviewing and benefits from referee interaction. After one year of testing, they are convinced that the new method accelerates the review process, brings more valuable comments, and provides a solid basis for the editors’ judgement. The system was also appreciated by the authors, as it helped them improve their manuscripts. Ben List explains: “We uploaded two manuscripts simultaneously and gave the crowd 72 hours to comment. The results were stunning: we received very qualified reviews. Further, the crowd interacted with one another, which is a unique advantage of the system and very valuable for the editor”, Ben List added.
The Mülheim scientists believe that this new method of reviewing could also be interesting for other scientific areas. “Intelligent-crowd-reviewing accelerates and improves scientific reviewing. We are sure that in the future it will become a useful tool for scientific evaluation”, List notes. The magazine “Nature” reports about the new method in its current issue.
Link to the article. 1 JUNE 2017 I VOL. 546 I NATURE
About the testing of ”intelligent-crowd-reviewing”
The platform for “intelligent-crowd-reviewing” was developed by Benjamin List, Denis Höfler, and an IT-Start-up company from Stuttgart. It allows editorial staff to upload submitted manuscripts, which are then commented on anonymously by a diverse crowd of scientists. Referees are even able to respond to comments made by other members of the crowd and at the end of the process, the authors receive a detailed evaluation of the review. For the crowd, more than 100 highly qualified referees were recruited. Most of them are young scientists from both academia and industry and specialize in chemical synthesis. All referees were carefully selected by the editorial board of Synlett. They received access to multiple manuscripts and decided which ones to comment on and to what extent they wanted to comment. In parallel, some of the exact manuscripts were also examined by a classical peer review process. “Intelligent-crowd-reviewing” led to more qualified comments in a shorter period of time and provided a solid basis for an editorial decision. Ben List decided that Synlett will continue working with this new method of “intelligent-crowd-reviewing”.
If you want to know more about the possibilities and challenges of “intelligent-crowd-reviewing”, please contact Professor Benjamin List for any additional information.
Contact:
Max-Planck-Institut für Kohlenforschung
Professor Dr. Benjamin List, Director of the Department of Homogeneous Catalysis
Phone: 0049/208/306-2410, E-Mail: List@mpi-muelheim.mpg.de
Max-Planck-Institut für Kohlenforschung
Isabel Schiffhorst, Press and Public Relations
Phone: 0208/306-2003, E-Mail: schiffhorst@mpi-muelheim.mpg.de