10.5935/rpot/2022.2.editorial EDITORIAL

Peer Review Process in Scientific Journals

A Avaliação por Pares em Periódicos Científicos

La Revisión por Pares en las Revistas Científicas

Roberto Moraes Cruz^I; Jairo Eduardo Borges-Andrade^II; Alexsandro Luiz De Andrade^III; Daniela Campos Bahia Moscon^IV; João Viseu^V; Marcos Ricardo Datti Micheletto^VI; María Elisa Ansoleaga Moreno^VII; Mª Inmaculada López Núñez^VIII; Mussa Abacar^IX; Nádia Kienen^X; Janete Knapik^XI; Simone Cassiano^XII; Daiane Rose Cunha Bentivi^XIII

^IEditor-in-Chief. Federal University of Santa Catarina (UFSC), Brazil
^IISenior Editor. University of Brasília (UnB), Brazil
^IIIAssociate Editor. Federal University of Espírito Santo (UFES), Brazil
^IVAssociate Editor. Federal University of Bahia (UFBA), Brazil
^VAssociate Editor. University of Algarve (UAlg), Portugal
^VIAssociate Editor. São Paulo State University (UNESP), Brazil
^VIIAssociate Editor. Diego Portales University (UDP), Chile
^VIIIAssociate Editor. Complutense University of Madrid (UCM), Spain
^IXAssociate Editor. Rovuma University (UniRovuma), Mozambique
^XAssociate Editor. State University of Londrina (UEL), Brazil
^XIJunior Editor. Federal University of Santa Catarina (UFSC), Brazil
^XIIJunior Editor. University of Brasília (UnB), Brazil
^XIIIPresident. Brazilian Association of Organizational and Work Psychology (SBPOT), Brazil

Peer review is like democracy, the saying goes, despite its flaws, it's the best system we have.

Brian Campbell (2019)

The debate on the quality and integrity of scientific production, which has been accentuated in recent years, has been associated with studies on the quality and relevance of scientific publications. That is, the ability of scientific journals to self-regulate themselves, especially in the conduct of an editorial process that promotes the evaluation of merit, originality, reliability of sources, data, analysis and results, theoretical foundation, and quality of scientific writing of manuscripts submitted to the appreciation of their editorial board and ad hoc reviewers (Cruz et al., 2021).

The increase in the number of theoretical and empirical articles, with emphasis on the scientific, technical, and ethical challenges of scientific communications, has highlighted the role of the peer review process or referee system. Although there are other qualitative and quantitative aspects of scientific publications, this system is considered essential in maintaining the integrity of the editorial process (Horbach & Halffman, 2018). Scientific journals that adopt peer evaluation become the preferred source for reading, citation, publication, and dissemination of knowledge, as well as serving as a reference for the determination of qualifiers of scientific publications, including the Impact Factor (Nassi-Calò, 2015). In journal search portals, the choice of the pre-select criterion is often the selection of only articles that adopt this evaluation is the result of a decision to prioritize what is most likely to represent an article that deserves to be identified, read, and perhaps cited. This can therefore affect the impact of what is published.

Historians of science such as Zuckerman and Merton (1971) suggest that the idea of peer review/evaluation is old, probably originating in the 17th century. It would coincide with the emergence of the academic journal Philosophical Transactions of the Royal Society of London, in 1665, directed by Henry van Oldenburg (1618-1677). This was considered the first publication dedicated to the compilation of scientific essays in the world. Other historians of science identify peer review in the early 18th century in the preface to the first volume of the Royal Society of Edinburgh's Medical Essays and Observations, published in 1731. It stated that the essays sent to the journal would be distributed to members of the institution, according to the subjects in which they were most versed, without their identity being known by the author of the essay (Spier, 2002). This statement suggests that specialized knowledge should be recognized in science, but that the bearers of this knowledge should not be known, when this involves judgments that would affect others.

Peer evaluation gradually gained space in scientific journals, as they developed and strengthened in academic centers, especially in the 20th century. This occurred in the context of the expansion of scientific research, technological development, professional specialties, and demands for publication (Tennant, 2018). Gradually, this evaluation influenced decisions on acceptance or rejection of manuscripts submitted to academic journals that, until then, were usually the responsibilities of a single editor or a select and restricted editorial committee (Horbach & Halffman, 2018). In a way, this expansion represented the democratization of the evaluation process and, on the other hand, it signaled that there would be a diversity of opinions that needed to be taken into account during this process. Peer review-as we know it today-developed sharply during and after World War II, in response to the boom in scientific research and the production of new technologies (Bakanic, McPhail, & Simon, 1987; Ware, 2008).

Since then, peer review has become a predominant method in managing scientific journals. The opinions produced by the evaluators or referees have significantly contributed to the editor's conviction-either by rejecting or accepting the manuscript for publication-which is possible to identify in the editorial flows of national and international scientific journals. This contribution was (or is) greatly intensified, when (or where) there was (or is) the adoption of evaluations in double-blind systems. That is, when the authors were (or are) not identified either. In addition to preserving the identity of the people they rate, these systems can reduce the likelihood that those people will have their appreciative judgments influenced by their knowledge of who produced what they rate.

Most scientific journals adopt similar peer review procedures: for example carrying out a formal and technical preliminary analysis, carried out by the editorial team (desk review), which has the prerogative to refuse manuscripts. They may also employ technical criteria when selecting reviewers; use scientific criteria to assess the format and content of manuscripts; and check reviewers' availability to apply the scientific criteria and issue opinions (Tennant, 2018). Some differences are found in the details of the peer review, in the items included in the opinions, and in the characteristics of the editorial decision; aspects that do not interfere in its adoption, in general.

The Revista Psicologia: Organizações e Trabalho (rPOT) includes peer review as an essential part of the editorial decision (Fig. 1).

There are several types of peer review or evaluation in the academic/scientific community. They are used in the selection and evaluation of members of the university community; in the scientific committees of scientific-professional events; in the appreciation of projects that seek support from institutions that promote teaching, research, and extension; and in the rules for publishing articles in scientific journals. Eventually, peer assessment can be used as a pedagogical tool in teaching-learning processes, providing feedback, critical reflection, and collaborative behavior. Some procedures for peer review are also adopted by non-academic institutions such as national/international agencies supporting policies and actions in different social segments, to assess funding/grant applications or validate the quality of a study, technical report, or intervention carried out.

There are basically three types of peer review. The first-called single-blind review or simple anonymous peer review (single anonymized peer review)-allows the identity of the author to be known by the referees, but the identity of the reviewers is hidden from the authors of the manuscript (and the public). It is the least commonly used method by current scientific journals, but it was adopted by the academic journals Philosophical Transactions of the Royal Society of London, and Royal Society of Edinburgh's Medical Essays and Observations as referred to above. The idea of hiding the identity of the reviewers is to allow them to freely evaluate the manuscripts, without fear of personal damage. Conversely, the simple blind review has historically been criticized for allowing personal criticisms, precisely for allowing the knowledge of the authorship of the manuscript.

In the second type-known as double-blind review (double anonymized peer review)-the identities of authors and reviewers are omitted. It is the type predominantly used in scientific journals, especially those with higher academic/scientific indexing qualifiers (Santos, Morris, Rattan, & Zakrison, 2021). There is the possibility of transferring or using opinions obtained in the peer review and the editorial decision between journals of the same editorial group (portable peer review or cascading), depending on the most suitable for the scope of the target journal. In practice, eventually, the double-blind method may not really mean what constitutes it. In relatively inextensive r fields of research, reviewers may recognize the identities of the authors of the manuscripts based on citations, perspective adopted of the subject, use of specific concepts, or even writing style (Ware, 2011). Even in these cases, opinions of good evaluative quality are obviously expected, which will most likely be verified by the journals' editorial boards.

The third type-called open peer review-proposes to reveal the identities of authors, reviewers, and editors, giving access, even to the community in general, to the evaluations and comments from reviewers and editors. This type of evaluation is associated with the evolution of digital communication processes and the movements called "open research" or "open science", in comparison with what is often seen as the traditional method of peer review (double-blind). The concept of open evaluation is at an embryonic stage and being tested in some publications, with the aim of realigning the original ideals of collaboration and progressive construction of arguments for manuscript improvement (Tennant, 2018). In addition to this open peer review proposal, other innovations are aligned, such as accessible databases, broad community participation in open platforms, availability of manuscripts under review (preprints), and open comments in the final version (Ross-Hellauer 2017).

In any type of peer review, it is expected that reviewers and editors have the freedom to make honest and compatible critiques of the scope and characteristics of the investigation referred to in the manuscript.. The main task of the reviewer is essentially to assist in the editorial decision of the journal-towards the rejection or acceptance (with or without substantive changes) of the manuscript for publication. Each editorial decision, under the scrutiny of reviewers and editors, must also fulfill its educational function of providing high-quality feedback on the merits and quality of manuscripts. It should also suggest possible needs to improve scientific reasoning, the state of the art, the quality of design and procedures used, analyses and interpretations produced, and the scientific writing.

Peer review is a procedure used to independently assess the scientific and technical merit and quality of manuscripts submitted to scientific journals, based on criteria previously determined by editorial rules. Reviewers or referees-"peers"-are researchers and authors of scientific publications, specialists in certain subjects, who are not part of the journal's editorial board (ad hoc). They must provide a conclusive opinion on the evaluated manuscripts, indicating their rejection, total acceptance, or partial acceptance, in the latter case being subject to corrections. In addition, the opinion is expected to provide specific suggestions or recommendations on how the manuscript can be explored, improved, or modified in aspects essential for qualification in the editorial process to which it was submitted, or in another submission.

The peer review process is complex. It involves the need for the collaboration of reviewers in terms of speed, and production of information relevant to the editorial decision regarding the manuscripts. This information is not always satisfactorily obtained (Cruz et al., 2021, 2020). There are other aspects to be considered when reviewing an article, such as the identification of theoretical-conceptual and writing weaknesses, the detection of methodological and statistical errors, and the identification of plagiarism (Botero, 2020; Mulligan, Hall, & Raphael, 2013). On the other hand, there is also the risk of favoring or dis-favoring reflections, perspectives, methods, or the use of specialized literature reviews through the outright rejection of reasonable or high quality manuscripts, combined with the indifferent acceptance of low quality manuscripts (D'Andrea & O'Dwyer, 2017). In any case, editors should be aware of the likelihood of possible biases in the peer review process, especially when there are important disagreements about the merit and quality of scientific research and communication. In these cases, these people need to make a decisive contribution to the verification of controversies and the complementation of information relevant to the editorial decision regarding the manuscript under review.

Peer review is considered the gold standard for safeguarding the quality of published science, determined in large part by the quality of its peer review process and editorial decision (Blockeel et al., 2017). However, the substantial annual increase in manuscripts sent to scientific journals-especially those which are highest evaluated-has caused a certain "crisis" in obtaining responses from reviewers. This has, as a consequence, increased time required for returning responses to authors.

Scientific journals generally do not have the capacity to publish all the manuscripts they receive, due to the limit of volume, number, and pages per year and also due to financial constraints. In a way, peer review can be considered, at the same time, a "filter"-in the sense of debugging acceptable or refusable manuscripts for publication, and a "bottleneck" in the editorial flow-in view of the accentuated volume of manuscripts to be evaluated. The effort of the editorial team to recruit reviewers who provide qualified work free of charge is noticeable, considering the accumulation of tasks and the time restrictions to which all are submitted.

On the other hand, the scarce incentives for peer review work do not reflect its importance in the training of those who research, teach, and write scientific publications. In this direction, it is important to reflect on the role of stricto sensu graduate courses in the development, skills, and competences of graduate students to evaluate manuscripts whilst issuing opinions on their quality, and scientific and professional relevance.

Peer review is firmly established as one of the most important procedures in the editorial flow of scientific journals, supported by a historical experience of about 350 years (Horbach & Halffman, 2018). Despite criticism about its ability to determine the merit and quality of manuscripts submitted to scientific journals, it-especially double-blind-is widely accepted in the scientific community to do so. It also contributes to developing and improving scientific writing and communication skills, increasing the possibilities of relationships within academic communities, and with society in general.

References

Bakanic, V., McPhail, C., & Simon, R. J. (1987). The manuscript review and decision-making process. American Sociological Review, 631-642. https://doi.org/10.2307/2095599

Blockeel, C., Drakopoulos, P., Polyzos, N. P., Tournaye, H., & García-Velasco, J. A. (2017). Review the 'peer review'. Reproductive Biomedicine Online, 35(6),747-749. https://doi.org/10.1016/j.rbmo.2017.08.017

Botero, J. C. R. (2020). La revisión por pares ("peer review") en las revistas científicas: un proceso que requiere intervención. Tempus Psicológico, 3(1),133-155. https://doi.org/10.30554/tempuspsi.3.1.3410.2020

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021). 20 anos da rPOT e o papel dos periódicos científicos na pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(4),1-3. http://doi.org/10.5935/rpot/2021.4.editorial

Cruz, R. M., Borges-Andrade, J. E., Moscon, D. C. B., Micheletto, M. R. D., Sticca, M. G., Carlotto, M. S., ... & Delben, P. B. (2020). Qualidade da redação científica: desafio à formação de pesquisadores e à publicação. Revista Psicologia Organizações e Trabalho, 20(1), I-II. http://doi.org/10.17652/rpot/2020.1.Editorial

D'Andrea, R., & O'Dwyer, J. P. (2017). Can editors save peer review from peer reviewers?. PloS one, 12(10),e0186111. https://doi.org/10.1371/journal.pone.0186111

Horbach, S. P., & Halffman, W. (2018). The changing forms and expectations of peer review. Research integrity and peer review, 3(1),1-15. https://doi.org/10.1186/s41073-018-0051-5

Ladrón de Guevara Cervera, M., Hincapié, J., Jackman, J., Herrera, O., & Caballero Uribe, C. V. (2008). Revisión por pares:¿ Qué es y para qué sirve?. Revista Salud Uninorte, 24(2),258-272. Retrieved from http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0120-55522008000200011&lng=en&tlng=es.

Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1),132-161. https://doi.org/10.1002/asi.22798

Nassi-Calò, L. (2015). A revisão por pares como objeto de estudo. SciELO em Perspectiva, São Paulo. Retrieved from https://blog.scielo.org/blog/2015/04/24/a-revisao-por-pares-como-objeto-de-estudo/#.Yq5GTXbMI2w

Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6. https://doi.org/10.12688/f1000research.11369.2

Santos, A., Morris, D. S., Rattan, R., & Zakrison, T. (2021). Double-blinded manuscript review: Avoiding peer review bias. Journal of Trauma and Acute Care Surgery, 91(1),e39-e42. https://doi.org/10.1097/TA.0000000000003260

Spier, R. (2002). The history of the peer-review process. TRENDS in Biotechnology, 20(8),357-358. https://doi.org/10.1016/S0167-7799(02)01985-6

Tennant, J. P. (2018). The state of the art in peer review. FEMS Microbiology letters, 365(19), fny204. https://doi.org/10.1093/femsle/fny204

Ware, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community-Results from an international study. Information Services & Use, 28(2),109-112. https://doi.org/10.3233/ISU-2008-0568

Ware, M. (2011). Peer Review: recent experience and future directions. New Review of Information Networking, 16(1),23-53. https://doi.org/10.1080/13614576.2011.566812

Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: Institutionalization, structure, and functions of the referee system. Minerva, 66-100. https://doi.org/10.1007/bf01553188

Bakanic, V., McPhail, C., & Simon, R. J. (1987). The manuscript review and decision-making process. American Sociological Review, 631-642. https://doi.org/10.2307/2095599        [ Links ]

Blockeel, C., Drakopoulos, P., Polyzos, N. P., Tournaye, H., & García-Velasco, J. A. (2017). Review the 'peer review'. Reproductive Biomedicine Online, 35(6),747-749. https://doi.org/10.1016/j.rbmo.2017.08.017        [ Links ]

Botero, J. C. R. (2020). La revisión por pares ("peer review") en las revistas científicas: un proceso que requiere intervención. Tempus Psicológico, 3(1),133-155. https://doi.org/10.30554/tempuspsi.3.1.3410.2020        [ Links ]

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021). 20 anos da rPOT e o papel dos periódicos científicos na pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(4),1-3. http://doi.org/10.5935/rpot/2021.4.editorial        [ Links ]

Cruz, R. M., Borges-Andrade, J. E., Moscon, D. C. B., Micheletto, M. R. D., Sticca, M. G., Carlotto, M. S., ... & Delben, P. B. (2020). Qualidade da redação científica: desafio à formação de pesquisadores e à publicação. Revista Psicologia Organizações e Trabalho, 20(1), I-II. http://doi.org/10.17652/rpot/2020.1.Editorial        [ Links ]

D'Andrea, R., & O'Dwyer, J. P. (2017). Can editors save peer review from peer reviewers?. PloS one, 12(10),e0186111. https://doi.org/10.1371/journal.pone.0186111        [ Links ]

Horbach, S. P., & Halffman, W. (2018). The changing forms and expectations of peer review. Research integrity and peer review, 3(1),1-15. https://doi.org/10.1186/s41073-018-0051-5        [ Links ]

Ladrón de Guevara Cervera, M., Hincapié, J., Jackman, J., Herrera, O., & Caballero Uribe, C. V. (2008). Revisión por pares:¿ Qué es y para qué sirve?. Revista Salud Uninorte, 24(2),258-272. Recuperado de http://www.scielo.org.co/scielo.php?script=sci_arttext&pid=S0120-55522008000200011&lng=en&tlng=es.         [ Links ]

Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1),132-161. https://doi.org/10.1002/asi.22798        [ Links ]

Nassi-Calò, L. (2015). A revisão por pares como objeto de estudo. SciELO em Perspectiva, São Paulo. Recuperado de https://blog.scielo.org/blog/2015/04/24/a-revisao-por-pares-como-objeto-de-estudo/#.Yq5GTXbMI2w        [ Links ]

Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6. https://doi.org/10.12688/f1000research.11369.2        [ Links ]

Santos, A., Morris, D. S., Rattan, R., & Zakrison, T. (2021). Double-blinded manuscript review: Avoiding peer review bias. Journal of Trauma and Acute Care Surgery, 91(1),e39-e42. https://doi.org/10.1097/TA.0000000000003260        [ Links ]

Spier, R. (2002). The history of the peer-review process. TRENDS in Biotechnology, 20(8),357-358. https://doi.org/10.1016/S0167-7799(02)01985-6        [ Links ]

Tennant, J. P. (2018). The state of the art in peer review. FEMS Microbiology letters, 365(19), fny204. https://doi.org/10.1093/femsle/fny204        [ Links ]

Ware, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community-Results from an international study. Information Services & Use, 28(2),109-112. https://doi.org/10.3233/ISU-2008-0568        [ Links ]

Ware, M. (2011). Peer Review: recent experience and future directions. New Review of Information Networking, 16(1),23-53. https://doi.org/10.1080/13614576.2011.566812        [ Links ]

Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: Institutionalization, structure, and functions of the referee system. Minerva, 66-100. https://doi.org/10.1007/bf01553188        [ Links ]