Debunking Post-Publication Peer Review

Jaime A. Teixeira da Silva^*

Miki-Cho Post Office, Kagawa-ken, Japan

Abstract

Science forms a fundamental core of many aspects of society. Many objects, functionalities, systems and processes are based on scientific principles and fundamentals. The latter two are often derived from scientific discovery which is traditionally built upon appropriate hypothesis testing, and exposed through published reports, in the form of scientific papers. Thus, any object or process that relies upon the methodology that underlies a scientific manuscript also relies on its basal premise of veracity. A scientific paper that has a flawed methodology – or errors contained therein – that is transmitted to third party users who do not suspect any flaws or errors can pose a potential threat to the integrity of science, and society. While processing a scientific paper, quality control is usually imposed through peer review, traditionally in a blind or double blind format, but also, in more rare cases, as open peer review, and thus there is shared responsibility by authors, editors, peers and publishers for what is ultimately released into the literature. Post-publication peer review (PPPR) serves to cover gaps inherent to traditional peer review. Where quality control has failed in the latter, the former can serve as an effective tool to cover those gaps. Correcting the literature after it has been published is an integral part of the publishing process and it is incumbent upon editors and publishers to provide the appropriate channels and means to allow for errors, problems and more serious issues related to publishing ethics, such as plagiarism, to be addressed. Scientific activism serves as one vocal tool to bring awareness to the wider scientific community and public about such issues. Literature that is not corrected will remain inherently flawed and corrupted, and will serve as a poor educational tool for young scientists. Poor science and corrupted literature represent a bad business model and can also represent a danger to society. Such issues, discussed here in this opinion piece, must be more openly, widely and publicly debated.

Keywords

Editorial Firewall, Expression of Concern, Peer Review, Plagiarism, Quality Control, Resistance

Received: March 30, 2015 / Accepted: April 20, 2015 / Published online: May 11, 2015

@ 2015 The Authors. Published by American Institute of Science. This Open Access article is under the CC BY-NC license. http://creativecommons.org/licenses/by-nc/4.0/

1. Why Science is Important and How it Fits into the Public Domain

Science and technology are intricately related to society. Given the power of the former, those who control it or basic aspects related to it would theoretically be able to have control over the latter [1]. Thus, the exquisite flavour of the latté that the reader drinks, the novel shape or longevity of the lamp that provides light, or the generic drug that saves money at the hospital or pharmacy register are not random objects that have appeared magically for public consumption. Almost every object or process that we are in contact with day after day, almost every structure that supports any other structure within the public framework that we find ourselves integrated, is based on scientific discovery and development. Therefore, to maintain a solid public framework and society, there is a need to have a robust and trustworthy scientific base.

2. The Role of Post-Publication Peer Review in Scientific Accountability

Traditional (or orthodox or standard) peer review (TPR) has always ultimately been in place to serve one key purpose: to critically evaluate as many aspects of a manuscript with the objective of ensuring that the content of what is ultimately published is as accurate as possible. This process usually involves an implicit guarantee of: a) originality; b) a complete, informative and clear methodology; c) the identification of errors or misrepresentation of the data. The editors assess peer reports, critically assessing these reports and author revisions, ultimately approving a set of findings for publication. These steps constitute scientific rigor and quality control (QC). As McLaughlin et al. [2] state, "a ‘good review’ … builds the reputation of the journal." The converse is thus also true. If QC is compromised through poor or porous peer review, then the journal and publisher’s reputation is also compromised.

When QC is imperfect, then the literature, as a didactic tool, suffers. Several reasons may exist: a rushed or superficial review, an insufficient number of or insufficiently qualified peers, oversight of peer comments, or a fallible submission system. TPR has always been quintessential to the QC of science prior to publication. Yet, it is an imperfect system [3], even with ample checks and balances in place. This reality, i.e., the imperfection, is not a reason for shame or embarrassment, but should not be ignored. When QC-related issues are actively or passively ignored, research integrity can become compromised, as has been shown to occur in China [4]. The correct attitude when encountering any error in the literature of any academic journal would be to issue an erratum. In more acute cases, the claims made in a paper may contradict the findings, or the conclusions based on the data set, invalidating the manuscript’s overall scientific validity and message. In worst-case situations, there may be instances of misconduct including, but not exclusively, self-plagiarism and plagiarism, data duplication, redundancy, or figure manipulation. These represent ethically challenging situations in TPR that need to be dealt with swiftly, and decisively. Usually such concerns are remedied by expressions of concern (EoCs) or retractions. The pride and legend of challenged authors, editors, peers, journals and publishers may be bruised and suffer since errors in the published literature reflect a fallible, or weak, QC.

Specific tools and events have allowed the retroactive scrutiny of the already published literature to be possible, including a greater number of open access texts made possible through more powerful search engines, wider accessibility to a wider scope of texts, more powerful and more refined software to detect text similarity, including plagiarism and self-plagiarism, more liberal web-sites and blogs that allow for a more frank discussion of the issues, and a rapid realization that, following TPR, post-publication peer review (PPPR) is a new reality that is here to stay [5-7]. Yet, as more and more manuscripts come under scrutiny, there is also a rising tide of resistance to and reticence about PPPR, both by defensive authors and by rigid editorial or publishing structures. Such rigidity, mental and structural, makes reform of the publishing framework more difficult to remould.

Prior to PPPR, a paper is in fact published the moment it becomes public, but traditionally, in many scientists’ minds, a published paper is broadly associated with a publication in an academic journal. Such rigid and traditional perceptions can hinder receptivity to the open peer review system. Thus, on the one hand, there is a rigid system with inherent flaws, but which still remains the most widely accepted publishing model, and on the other hand, PPPR is beginning to exist to challenge this rigid framework. PPPR ultimately identifies issues, senso lato, in a published paper. Unlike TPR, where peer review has involved journal-recruited and frequently author-suggested peers, PPPR relies on a frameless, open and public discussion, even anonymously, of research findings. In fact, the concept of PPPR is not novel, or earth-shattering. Many undergraduate and postgraduate programmes in universities and research institutes usually involve journal clubs or slots in academic schedules that are dedicated to the discussion of scientific papers, allowing students and lecturers, professors or other academic staff to interact, share their ideas, criticisms and discussions about a published paper. One could say that the peer pool, of both unqualified and underqualified elements, under the guidance of more qualified elements, is potentially very large. Thus, PPPR has in fact always existed and has always been part of the core fabric of academia. However, with public platforms for discussion, electronic forums or blogs, PPPR has taken on a new digital face that challenges the physical and intellectual limits of traditional journal clubs.

Current platforms that exist for such discussion include (non-exhaustive list) sites like Retraction Watch, PubPeer or PubMed Commons, which allow public debate which exposes errors publicly, for wider discussion in society and among the peer pool without the traditional restricted barrier imposed by, and possible bias inherent in, editorial screening. To date, the discussion has been almost exclusively controlled by the main protagonists (authors, editors, publishers) who have filtered out criticism that could challenge TPR. PPPR eliminated this editorial firewall. Although the use of whistle-blowing and anonymity may invoke negative associations, these are core processes associated with PPPR but still in a young stage of evolution. There is a bottle-neck of decades of controlled TPR and a mass of post-publication evaluation and criticism of the scientific literature that has yet to emerge simply because it has yet to gain traction. As PPPR evolves, its tone, style and content will self-refine, avoiding legal head-on collisions, ultimately leaving a new framework that recognizes that the imperfections of TPR have led to its failure to some extent. PPPR must thus be recognized as an open-ended mechanism to identify errors existent in the already published literature with the ultimate objective of correcting the academic record.

Faced with increasing challenges to the literature, editors and peers are also faced with increasing responsibilities, including receptivity to PPPR reports, due scrutiny and subsequent correction of the published scientific literature. Extreme cases such as misconduct should not be the only spark for change. Any errors in the literature are fair game, and all merit to be corrected. This almost oxymoronic constructively destructive process must clearly distinguish between errors and negative results. In the absence of author accountability, or silence, the editors have the responsibility of correcting the literature even if QC was assured by previous members of an editor board. Most gatekeepers are weakly motivated to expend personal resources, such as time and energy, to improve the QC-related aspects of their journal. In such a situation, it is (unfortunately, and ironically) incumbent upon the general readership to seek the correction of the literature through PPPR. Indifference on their behalf, and their failure to actively address problems and concerns can be referred to as "complicit silence".

3. PPPR is Not Limited to the Published Literature

Integrity in science and science publishing are not limited exclusively to published papers and the publishing process. Publishers as well as peripherally related parties need to be equally critically questioned and examined. Simply because they derive benefit from, in some way or another, the current publishing framework. For example, science activism would demand that tough questions would be asked publicly to and about entities such as Thomson Reuters [8] or the ISSN [9] who lend structural support to journals or publishers, some of which may be failing to implement QC measures, or who are benefitting unfairly (through sales, metrics, or fame) from lax TPR, poor QC and the failure to accommodate PPPR into their socio-publishing and business model. The expanding world of predatory open access (OA) publishing, as exemplified by multiple cases at Beall’s www.scholarlyoa.com, requires the "predatory" nature of such operations and publishers to be quantified (e.g., [10]) to allow truly predatory journals and publishers to be clearly distinguished from on-predatory or borderline cases. One possibility would be the implementation of the Predatory Score [11]. As part of the Predatory Score, which assesses different levels of QC within a journal, editor board, or publisher, there now also needs to be a new aspect factored in: PPPR. When editors, journal or publishers fail to meet their public responsibilities towards QC, i.e., when accountability and transparency are amiss, then they need to be featured as case studies, what some might colloquially refer to as public shaming, to openly alert the wider academic community of the actual or potential risks (e.g., [12]). In this exploratory model of traditional or OA science, technology and medicine (STM) publishing involves, in most cases, the free exploitation of time, expertise and knowledge of editors and peers, without any financial remuneration; moreover, in this same exploratory model, authors are double-taxed and expected to foot the publishing bill [13]. Yet, it is scientists and their institutes who are to blame for the success of this current publishing model, the former because they cannot perceive a system beyond what is fed to them, and the latter, which tends to impose rules and limitations that are based on often non-academic factors such as the impact factor. So, the impetus for a flawed TPR lies far prior to peer review having taken place. In this wider panorama of the purpose of PPPR, the actual responsibilities of the publishing quartet (authors, peers, editors, and publishers) needs to be closely examined, post-publication. The quality of science and the productivity of scientists cannot thus only rely exclusively, or heavily, on metrics that are manipulated and imposed by the core/key STM publishing players, or their associated parties, but must incorporate a wider scope of variables that balances out the equation and dilutes the importance of pseudo-metrics, but that does not exclude them altogether, since they remain informative. One possibility is the Global Science Factor [14].

4. The Path Forward

The most logical and correct procedure moving forward towards a practically useful and implementable PPPR road map requires, at first, cognizance by all key parties involved (authors, editors, publisher), that there are problems and errors, and that these need to be fixed, simply because this forms part of their public responsibility towards science [5]. Problems may reflect more "menial" or "trivial" issues such as the lack of italicization of Latin names, errors in authors’ names (snub publishing; [15,16]), or more complex – and serious – issues such as data, figure or table duplication, self-plagiarism or plagiarism. Moreover, if we observe the overall function of a scientific manuscript as serving as an academic document and a tool used for the education of students, by academic institutes, or by other scientists who then apply those results to further other ideas and expand on their own hypotheses, there is without a doubt that that document must represent a solid, coherent and accurate set of facts, represented in the most rigorous manner possible. Moving full circle to the introductory section of this opinion piece, most scientific papers are meant to serve science itself, but in many cases, those findings lead to discoveries that will benefit society. On this path of discovery and disclosure, there are entities that also benefit, financially or otherwise, from the existence of this never-ending wealth of scientific discovery, one of the most prominent being the for-profit STM publishers, whose acquisition of intellectual copyright represents one of science and society’s greatest conflicts, because it has the potential of being used, or abused. Thus, PPPR serves as a tool to hold all parties more accountable, and to correct what has been published in error, or deliberately falsely.

Where claimed hypotheses are not supported by data sets, where serious contradictory claims are made, or where more serious issues exist, a retraction may most likely be the most appropriate route to follow [17]. In select cases that are not associated with misconduct, authors should be given an opportunity to correct the findings of the manuscript and address the weaknesses, and the revised manuscript should be subjected to a fresh round of peer review, even if TPR. To circumvent the inherent weaknesses of TPR [3], an open peer review system, that identifies the identities of the reviewers [18], following a double-blind peer review, would be advised. In addition, all authors should be listed as corresponding authors, which would allow any, or all of them, to be held publicly accountable for their published work in the future [19]. The implementation of a reformed system that makes authors, editors, peers, journals and publishers accountable for what they have published or approved for publishing in the framework of their publishing models, and by ensuring that they assume the responsibility of correcting any erroneous literature, would go one step forward towards fortifying integrity in all steps of the publishing process [20].

References

1. Teixeira da Silva JA. 2011.Who owns science, owns society.Maejo International Journal of Science and Technology5(1): S1–S10.
2. McLaughlin H, Robbins S, Fogel SF, Busch-Armendariaz N, Wachter K, Pomeroy EC. 2015. Writing a good peer review to improve scholarship: what editors value and authors find helpful. Social Work Education: The International Journal DOI: 10.1080/02615479.2015.1021550
3. Teixeira da Silva JA, Dobránszki J 2015.Problems with traditional science publishing and finding a wider niche for post-publication peer review. Accountability in Research: Policies and Quality Assurance 22(1): 22-40.
4. Qiu J. 2015. Safeguarding research integrity in China. National Science Review 2: 122-125.
5. Teixeira da Silva JA. 2013. Responsibilities and rights of authors, peer reviewers, editors and publishers: a status quo inquiry and assessment. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 6–15.
6. Teixeira da Silva JA. 2013.The need for post-publication peer review in plant science publishing. Frontiers in Plant Science4: Article 485.
7. Teixeira da Silva JA. 2014.Postpublication peer review in plant science. Science Editor (Council of Science Editors) 37(2): 57+59.
8. Teixeira da Silva JA. 2013.The Thomson Reuters Impact Factor: critical questions that scientists should be asking. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 81-83.
9. Teixeira da Silva JA. 2013.The ISSN: critical questions that scientists should be asking. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 76-80.
10. Teixeira da Silva JA. 2014.Global Science Books: a tale from the cuckoo’s nest. How predatory open access publishing can influence the metrics of a traditional scholarly publisher. KOME 2(2): 73-81.
11. Teixeira da Silva JA. 2013.Predatory publishing: a quantitative assessment, the Predatory Score. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 21-34.
12. Teixeira da Silva JA,Dobránszki J. 2013. How not to publish an open access journal: a case study. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 102-110.
13. Teixeira da Silva JA. 2013.Taxing the intellectual base: should authors foot the publishing bill? The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 111-113.
14. Teixeira da Silva JA. 2013.The Global Science Factor v. 1.1: a new system for measuring and quantifying quality in science. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 92-101.
15. Teixeira da Silva JA. 2013. Snub publishing: theory. The Asian and Australasian Journal of Plant Science and Biotechnology7(Special Issue 1): 35-37.
16. Teixeira da Silva JA. 2014.Snub publishing: evidence from the Anthurium literature. Publishing Research Quarterly 30(1): 166-178.
17. Teixeira da Silva JA. 2014.Recent retraction cases in plant science that show why post-publication peer review is essential. Journal of Advancement in Engineering and Technology 1(3): 4 pp (DOI: 10.15297/JAET.V1I3.03).
18. Teixeira da Silva JA. 2013.Should an editor or peer reviewer be openly acknowledged? The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 116-117.
19. Teixeira da Silva JA.Dobránszki, J., Van, P.T., Payne, W.A.2013.Corresponding authors: rules, responsibilities and risks. The Asian and Australasian Journal of Plant Science and Biotechnology 7(Special Issue 1): 16-20.
20. Teixeira da Silva JA. 2013.How to better achieve integrity in science publishing. European Science Editors 39(4): 97-98.