10.5935/rpot/2022.1.editorial EDITORIAL

Editorial management and peer review in scientific journals: the challenges of rPOT

Gerenciamento editorial e revisão de pares em periódicos científicos: os desafios da rPOT

Gestión editorial y revisión por pares en revistas científicas: los retos de rPOT

Roberto Moraes Cruz^I; Jairo Eduardo Borges-Andrade^II; Alexsandro Luiz De Andrade^III; Daniela Campos Bahia Moscon^IV; João Viseu^V; Marcos Ricardo Datti Micheletto^VI; María Elisa Ansoleaga Moreno^VII; Mª Inmaculada López Núñez^VIII; Mussa Abacar^IX; Nádia Kienen^X; Janete Knapik^XI; Simone Cassiano^XII; Daiane Rose Cunha Bentivi^XIII

^IEditor-in-Chief, Federal University of Santa Catarina (UFSC), Brazil
^IISenior Editor, University of Brasília (UnB), Brazil
^IIIAssociate Editor, Federal University of Espírito Santo (UFES), Brazil
^IVAssociate Editor, Federal University of Bahia (UFBA), Brazil
^VAssociate Editor, University of Algarve (UAlg), Portugal
^VIAssociate Editor, São Paulo State University (UNESP), Brazil
^VIIAssociate Editor, Diego Portales University (UDP), Chile
^VIIIAssociate Editor, Complutense University of Madrid (UCM), Spain
^IXAssociate Editor, Rovuma University (UniRovuma), Mozambique
^XAssociate Editor, State University of Londrina (UEL), Brazil
^XIJunior Editor, Federal University of Santa Catarina (UFSC), Brazil
^XIIJunior Editor, University of Brasília (UnB), Brazil
^XIIIPresident, Brazilian Association of Organizational and Work Psychology (SBPOT), Brazil

Psychology: Organizations and Work Journal (Revista Psicologia: Organizações e Trabalho - rPOT) has become an important vehicle in its area over the twenty years since it was founded, for the dissemination of scientific knowledge in its scope. The history of rPOT reflects the development of the training of its editors, assistants and reviewers, in technical findings, improvement of theories, methods and techniques, and promotes critical reflection on n professional training and updating (Cruz et al., 2021a).

Fulfilling this role requires editorial efforts, prerequisites for manuscript evaluation time, and the need for financial resources for producing and editing a high-level quarterly scientific journal (Cruz et al., 2021b; Cruz et al., 2019). In this direction, it is important to highlight the role of the Brazilian Association of Organizational and Work Psychology (Associação Brasileira de Psicologia Organizacional e do Trabalho - SBPOT) in promoting the sustainability of rPOT throughout its trajectory.

The preparation of articles for publication in scientific journals is an important aspect of the research process, the dissemination of findings, and eventually, the proposition or improvement of theoretical perspectives, methods, techniques and instruments (Kazdin, 1995; Nickerson, 2005). Authors who submit their manuscripts to scientific journals generally expect an agile process of peer review and evaluation, and a quick feedback on the editorial decision-favorable or not-for their publication. For the journals' editorial teams, publishing manuscripts competes with the caution required to mobilize reviewers and associate editors in a timely manner whilst guaranteeing the quality of the scientific production made available to the community.

The management of the editorial process for manuscripts submitted to rPOT depends on a set of variables to be successful. For example the volume of submissions; verification by desk review; acceptance/refusal of reviewers' suggestions, turnaround time and quality of reviews; communication between editors, authors and reviewers; feedback on suggested changes in manuscripts, editorial decisions throughout the process of (re)evaluation; and review of submissions and the technical and computerized processing of manuscripts definitively accepted for publication. All these aspects influence the quality of the editorial process, the quantity of articles published, and especially, the time between submission and publication of a manuscript.

The publishing of scientific journals faces paradoxical challenges: the search for speed and quantity of publications as well as easy access, whilst maintaining transparency, scientific quality, and the rapid return of manuscript evaluations (Campos & Candido, 2021). All these aspects are part of the constant challenges which scientific journals face, in an attempt to guarantee a constant flow of annual volumes, supported by minimal financial resources, a qualified editorial team, and the essential support of ad hoc reviewers.

A brief assessment of the processing of manuscripts (theoretical and empirical articles, experience reports, reviews) published in the last three years (2019 to 2021) reveals an increase in the number of submissions and publications (Figure 1). Among the articles evaluated between 2019 and 2021, the average rejection rate in the final editorial decision was approximately 68%.

The desk review phase is an important strategy in the initial processing of a manuscript. It constitutes a preliminary analysis carried out by the editorial team, which has the prerogative to refuse manuscripts without sending them for peer review. In the cases examined, manuscripts that did not meet the requirements for submission to rPOT were found which i) did not fit into one of the modalities accepted by rPOT (empirical research, literature review, professional experience report, theoretical essay or review; ii) presented incomplete data or lack of documents; iii) did not comply with APA Publication Manual; iv) lacked a formal logical structure (from the abstract to the conclusion) and contained low quality of writing (grammatical and scientific writing); v) did not show adherence to any of the themes that compose the production of knowledge in organizational and work psychology or clear contributions to the field. In the latter case, specifically, the editorial understanding of which other journals could potentially be targets for manuscript submission is reported to the authors.

One of the most sensitive aspects faced by scientific journals-and at rPOT it is no different-lies in the identification of qualified reviewers who can collaborate in the evaluation of manuscripts and in the reception of opinions that express the expectation of a detailed evaluation and that objectively help in the editorial decision. The opinions of rPOT deal with an assessment of the quality of some essential items, arranged in the form of a questionnaire: a) Title (conciseness and focus); b) Abstract (structure, quality, keywords); c) Introduction (research problem, state of the art and theoretical arguments that justify the relevance of the study); d) Method (study design, characterization of participants, context, instruments, data collection and analysis procedures and ethical care adopted); f) Results (organization, description, analysis and consistency with the adopted procedures); g) Discussion (interpretation of results, supported by relevant and updated scientific literature, and consistent with the problem and objectives of the study); f) Writing and scientific writing (grammatical correction and consistency, use of scientific language, logical-argumentative consistency and compliance with APA rules).

In the questionnaire sent to reviewers, there is, therefore, a set of essential elements for the evaluation of manuscripts that can provide relevant observations for editorial decisions. rPOT has a robust list of reviewers (over 1,200), built throughout its trajectory , which in practice does not mean a quick response. In fact, the number of reviewers who refuse or simply do not respond to rPOT requests is unfortunately high: above 50%. There are reviewers who agree to collaborate in carrying out the opinions, but exceed the stipulated deadlines and, still, there are those who send opinions lacking justification (for example, only with a "yes"/"adequate" or "no"/inadequate). This makes their use in the evaluation process unfeasible.

In many cases, opinions are requested from more than a dozen people for the same article, which further contributes to the extension of deadlines for evaluation and sending responses to authors. All these aspects, which are certainly not exclusive to the editorial process of rPOT, emphasize the importance of the reviewer's role in the evaluation of manuscripts, but also the difficulty of managing this process by scientific journals. It is also important to point out the work overload of the reviewers, who tend to work with several journals and offer their services without charge.

In addition to the problems and challenges faced in the production of technical opinions of manuscripts and its importance in the flow of editorial decisions on acceptance (with major or minor reformulations) and rejection, the pedagogical role of peer review in the editorial process is very relevant. In this sense, the analyses, arguments, and contributions offered by qualified opinions help, not only in the eventual technical and scientific improvement of the manuscripts, but also in the training of researchers, and therefore in the improvement of scientific production and dissemination.

The peer review system adopted in scientific journals will soon go into crisis, due to the number of manuscripts submitted, given that the demand for reviews is outstripping the supply, as emphasized by Fox and Petchey (2010). Peer review, although relatively effective for improving the evaluated manuscripts, is not always effective in dealing with plagiarism and substantive improvements in the interpretation of research findings in the reviewed texts (Ware, 2011). Assigning responsibility for integrity to the peer review system is relatively recent and remains controversial. However, for at least the last 20 years, there is an evidence base on the validity and degree of reliability of peer review, little known or ignored by journal editors, according to Horbach and Halffman (2018).

Based on a sample of 31 opinions issued between 2019 and 2021 by rPOT reviewers, and considered at a level of excellence, we identified errors or content problems, which indicate a loss in the quality of scientific writing and in the theoretical/technical consistency of the manuscripts. All these opinions led to the rejection of the manuscripts or to substantive changes in the text. To systematize this data, among the 80 items identified in the opinions, 17 categories were created. Table 1 shows a compilation of the categories of errors or content problems most commonly cited in these opinions, in a cumulative and comparative way, based on the frequency hierarchy, in percentage terms.

In general, among the errors and content problems most commonly mentioned by reviewers, those that directly interfere with the quality of a scientific publication stand out: insufficient and/or outdated theoretical and methodological aspects, inconsistent and/or insufficient discussions/interpretations and inaccurate and/or poorly written summaries. These aspects were identified in two thirds of the evaluated manuscripts. One of the main aspects to be observed in peer review lies in the theoretical contribution of a manuscript, in addition to its methodological rigor, says Maner (2014).

Other aspects that significantly interfere with the quality of manuscripts submitted to rPOT concern the presence of grammatical and typing errors, indicating less commitment to reviewing the manuscript before submission. In these cases, we can raise the hypothesis of "premature submissions", or rather, those in which authors should review the text before sending it. These aspects are salient in at least 65% of the evaluated manuscripts. These problems are associated with lower quality in the treatment and analysis of data-qualitative or quantitative-often "dislocated" from the objectives, assumptions, or hypotheses of the study. An imprecise description of the study or with missing information complements this previous point. These last two issues were present in more than 50% of the evaluated materials.

The other aspects categorized in Table 1 reflect errors or content problems related to the design of the article: title, objectives, hypotheses, assumptions; and the quality of scientific writing. They also reveal the use of common sense language, undefined concepts and poorly designed tables/figures, inconsistent or repetitive text, lack of logical sequence in the arguments, and difficulties in systematizing the findings, contributions and limitations of the research. All these aspects, in general, accentuate the problem of learning scientific writing: in the definition/delimitation of the scope of the research problem and in its form of operationalization in the text to be published, as well as in the compilation, treatment and analysis of theoretical scientific works and empirical data.

The current debate about the credibility and integrity of scientific research, and their respective forms of dissemination, emphasize the concern with the search for transparency and determination in the constant maintenance and improvement of criteria for submission, evaluation, and publication of research in scientific journals. The evaluation system based on ad hoc opinions, which subsidize editorial decisions, remains a central aspect for self-regulation of the quality of publications. Improving it and considering it as a necessary filter for the reliability of rPOT, remains a current and future challenge.

References

Campos, L. A., & Candido, M. R. (2021). Transparência em DADOS: submissões, pareceristas e diversidade no fluxo editorial dos últimos anos. Editorial. Dados rev. ciênc. Sociais, 65(1). https://doi.org/10.1590/dados.2022.65.1.000

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021a). 20 anos da rPOT e o papel dos periódicos científicos na pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(4),1-3. https://doi.org/10.5935/rpot/2021.4.editorial

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021b). Produção e divulgação de conhecimentos científicos em tempos de pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(1), I-II. https://doi.org/10.5935/rpot/2021.1.editorial

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Micheletto, M. R. Sticca, M. G., ... & Barros-Delben, P. (2019). Repercussões e desafios à produção da Revista Psicologia: Organizações e Trabalho. Revista Psicologia Organizações e Trabalho, 19(3), I-II. http://doi.org/10.17652/rpot/2019.3.Editorial

Fox, J., & Petchey, O. L. (2010). Pubcreds: fixing the peer review process by "privatizing" the reviewer commons. The Bulletin of the Ecological Society of America, 91(3),325-333. https://doi.org/10.1890/0012-9623-91.3.325

Horbach, S. P., & Halffman, W. (2018). The changing forms and expectations of peer review. Research integrity and peer review, 3(1),1-15. https://doi.org/10.1186/s41073-018-0051-5

Kazdin, A. E. (1995). Preparing and evaluating research reports. Psychological assessment, 7(3),228. https://doi.org/10.1037/1040-3590.7.3.228

Maner, J. K. (2014). Let's put our money where our mouth is: If authors are to change their ways, reviewers (and editors) must change with them. Perspectives on Psychological Science, 9(3),343-351.

Nickerson, R. S. (2005). What authors want from journal reviewers and editors. American Psychologist, 60(6),661-662. https://doi.org/10.1037/0003-066X.60.6.661

Ware, M. (2011). Peer Review: recent experience and future directions. New Review of Information Networking, 16(1),23-53. https://doi.org/10.1080/13614576.2011.566812

Campos, L. A., & Candido, M. R. (2021). Transparência em DADOS: submissões, pareceristas e diversidade no fluxo editorial dos últimos anos. Editorial. Dados rev. ciênc. Sociais, 65(1). https://doi.org/10.1590/dados.2022.65.1.000         [ Links ]

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021a). 20 anos da rPOT e o papel dos periódicos científicos na pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(4),1-3. https://doi.org/10.5935/rpot/2021.4.editorial        [ Links ]

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Viseu, J., Micheletto, M. R. D., ... & Carvalho-Freitas, M. N. D. (2021b). Produção e divulgação de conhecimentos científicos em tempos de pandemia da COVID-19. Revista Psicologia Organizações e Trabalho, 21(1), I-II. https://doi.org/10.5935/rpot/2021.1.editorial        [ Links ]

Cruz, R. M., Borges-Andrade, J. E., Andrade, A. L. D., Moscon, D. C. B., Micheletto, M. R. Sticca, M. G., ... & Barros-Delben, P. (2019). Repercussões e desafios à produção da Revista Psicologia: Organizações e Trabalho. Revista Psicologia Organizações e Trabalho, 19(3), I-II. http://doi.org/10.17652/rpot/2019.3.Editorial        [ Links ]

Fox, J., & Petchey, O. L. (2010). Pubcreds: fixing the peer review process by "privatizing" the reviewer commons. The Bulletin of the Ecological Society of America, 91(3),325-333. https://doi.org/10.1890/0012-9623-91.3.325        [ Links ]

Horbach, S. P., & Halffman, W. (2018). The changing forms and expectations of peer review. Research integrity and peer review, 3(1),1-15. https://doi.org/10.1186/s41073-018-0051-5        [ Links ]

Kazdin, A. E. (1995). Preparing and evaluating research reports. Psychological assessment, 7(3),228. https://doi.org/10.1037/1040-3590.7.3.228        [ Links ]

Maner, J. K. (2014). Let's put our money where our mouth is: If authors are to change their ways, reviewers (and editors) must change with them. Perspectives on Psychological Science, 9(3),343-351.         [ Links ]

Nickerson, R. S. (2005). What authors want from journal reviewers and editors. American Psychologist, 60(6),661-662. https://doi.org/10.1037/0003-066X.60.6.661         [ Links ]

Ware, M. (2011). Peer Review: recent experience and future directions. New Review of Information Networking, 16(1),23-53. https://doi.org/10.1080/13614576.2011.566812        [ Links ]