A PEER-TO-PEER REVIEW PROTOCOL

In A Social History of Truth, Steven Shapin describes the original spirit behind peer review. Scientists needed ways to trust other scientists, and, even if rejecting all higher callings, for no other reason than to trust the materials and supplies they are buying to run their own, solitary experiments. Without an ability to trust in another’s work, every scientist would have to start the inquiry all over again.

Modern peer review in its current form is one of the main processes for forming the graph of “officially sanctioned knowledge.” Thus, any flaws in this system have compounding effects on society and the ability of future researchers to publish their findings, especially when these findings run contrary to received wisdom in their field.

While peer review remains an integral part of academic publishing, its flaws have become increasingly apparent. Fame can allow somebody to become an “ultra-reviewer,” while poorly positioned scientists running novel experiments are caught by the wayside.

The grueling and long process of having a paper accepted in a journal is now known not just to academics, but to researchers in various professions who have learned how onerous and slow the process of attempting to publish their work in a “reputable” journal often is, and to young people weighing how they want to spend their careers and to what they want to dedicate their talents.

With new tools for decentralized work, I propose a “peer-to-peer review” research initiative. The goal of this initiative would be to develop a decentralized peer-review protocol that will solve or mitigate several existing problems with peer review:

• Peer review can be an arbitrarily long process
• Peer review as it exists today leads to elitism and cloistering effects
• Emerging fields, or multidisciplinary fields, often do not have ways of using existing peer review processes to garner legitimacy
• Peer review often relies on volunteers

Potential research directions could include:

• Assembling a network of interested researchers to run pilot experiments with
• Checking which blockchain-based incentive structures may encourage researchers in currently underrepresented groups to participate in reviewing others’ work
• Creating a map of intellectual domains that are currently underserved in the credibility marketplace
• Porting experiments in peer-to-peer “rating” dynamics onto the problems in peer review
• Completing a functional analysis of peer review

A peer review system that is not fully trusted can quickly and unexpectedly lead to hard-to-model social problems. Research into alternative models and decentralized modes of peer review can therefore lead to considerable benefits for scientific coordination.