Keywords

What This Chapter Is About

As an early career researcher, you will probably not be extensively involved in reviewing journal articles or research proposals, or editing scientific journals. However, reviewing, evaluating and editing are important aspects of research. As an early career researcher, especially after getting a doctoral degree, you may be invited by a journal to serve as a peer reviewer, or may edit or work in a scientific peer review journal. It is important that you understand what to expect from a responsible review of your work – when you submit a manuscript to a journal or a grant proposal. In this chapter, we will look at different types of journal peer review. We will address the responsibilities of peer reviewers toward the authors and editor, including confidentiality, objectivity, and competing interests. We will focus on journal peer review, because this is something that you will certainly experience from the author’s side, and possibly as a reviewer. The principles of professional and responsible peer review also apply to other types of peer review, such as for grants of academic/research advancement.

Case Scenario: Peer Review Misuse

This hypothetical scenario was adapted from a narrative concerning the links between research environments and research integrity. The original case scenario is developed by the Members of The Embassy of Good Science and is available at the Embassy of Good Science. The case below is published under Creative Commons Attribution-ShareAlike license, version 4.0 (CC BY-SA 4.0).

Professor Daniels is a well-known sociologist in the faculty of social sciences at a public university. Currently, the research group is working on an interdisciplinary project investigating innovative empirical methods that rely on the use of social media platforms for data collection. With her team of master and doctoral students, she plans to submit a paper on this topic. She receives an email from a new academic to review a manuscript academic journal. After reading the abstract of the manuscript, which seems to have significant overlaps with the research topics of their own manuscript draft, she accepts the journal’s invitation to review. Due to the lack of time, Prof. Daniels asks her doctoral student to read the manuscript and develop the review. When the review is ready, Prof. Daniels sends it without reading it to the journal. The submitted review is very critical and recommends major revisions, including additional reference from Prof. Daniels.

Very soon, Prof. Daniels and her team submit their paper to a prestigious journal. The paper receives extremely positive reviews and is outright accepted and published online only 4 weeks after submission. A couple of weeks after the publication of the article by Prof. Daniels and her team, the editor of the journal receives a complaint from the corresponding author of the article where Prof. Daniels was a reviewer, claiming that Daniels’ article contains one of the innovative methodological models developed by their group and described in their own article. It is also claimed that the table in Daniels’ article presents the main features of their model from the other manuscript. The authors ask for an urgent investigation into the peer review process for Prof. Daniels’ article, which they state was compromised. Two weeks have passed and the corresponding author of that article has still not received an adequate response from the editorial office. Because Prof. Daniels is the first author of the paper that contains the suspicious content, the corresponding author of the other article decides to send their complaint to the research integrity office at Prof. Daniels’ institution.

Questions for You

  1. 1.

    What do you think about the practice that professors give doctoral students articles for review, which they received from journals?

  2. 2.

    What should have Prof. Daniels done when she received the manuscript for review, which significantly overlaps with her research?

  3. 3.

    What steps could have been taken to increase the transparency of this particular peer review process?

  4. 4.

    What do you think about the practice for a peer reviewer to use ideas that they have identified when reviewing the work of other researchers? If not, what are your reasons? If it is acceptable, what conditions must be met in order for a peer reviewer to employ these ideas in their own work?

What Is Peer Review?

Peer review is the evaluation of someone’s work (manuscript submitted to a journal, research proposal submitted to a funding body, research/academic promotion assessment) by peers – experts with similar competencies (research, professional, academic).

Peer review is mostly viewed as an evaluation of a suitability of a submitted manuscript for publication in a journal. Although the first two scientific journals were started in 1665, the first formal peer review was introduced almost a century later, in 1731, when the Royal Society of Edinburgh introduced peer review by society members. Over time, peer review evolved into an assessment external to the journal and became a standard practice only in the twentieth century. There is a famous story of Albert Einstein complaining to the editor of the Physics Review about reviewing his article and refusing to address the comments on an anonymous expert. The journal Nature introduced peer review only in 1967.

You are probably familiar with the most common types of peer review (Table 8.1), which have evolved in different research fields. If you come from biomedical and health research, you are used to the single blind peer review, in which you do not know the identity of the reviewer, who reviews a manuscript with the full information about authors. If you come from social sciences, you are probably used to the double-blind peer review, where neither you nor the reviewers of your manuscript are aware of each other’s identity.

Table 8.1 Types of journal peer review
Full size table

New types of peer review are being developed and tested in different journals (Table 8.1), with the aim to reduce the bias in making decisions on the quality and suitability of manuscripts for publication, promoting transparency and replication, and generally preventing waste in research. The future may bring more novel approaches to peer review and discussions in the scientific community.

Similar to scientific research, peer review also raises many ethical issues and problems, and these issues may sometimes be complex and serious. Unfortunately, there is no clear right or wrong way or easy decisions in such cases, so you have to get familiar with the complexity of and expectations from peer review.

Good research practice from the European Code of Conduct for Research Integrity:

  • Researchers take seriously their commitment to the research community by participating in refereeing, reviewing and evaluation.

If you are invited to review a manuscript for a journal, the invitation will come together with the abstract of the manuscript. Keep in mind these questions before you accept the invitation.

Do I Have the Expertise to Review the Manuscript?

Journal editors often ask experts from different research areas to review an article, in order to get a comprehensive opinion on research presented in the manuscript. You do not need to have a high level of expertise in the topic of the article, but you should have sufficient knowledge to be able to provide an objective and professional assessment. Sometimes it is difficult to make this judgement based on the abstract – the basic rule for you may be that the topic of the abstract is close to the general topic of your research (doctoral dissertation).

What Type of Peer Review Is Used by the Journal?

As an early career researcher, who has to build their own research career and is dependent on senior researchers, you may not be comfortable to take part in a fully open peer review process. You may feel more secure to provide critical comments about work of senior researchers if you participate in a blinded (i.e. masked) review, where your identity will not be disclosed to the authors. We know from research that authors are not good at identifying the reviewers of their journal manuscripts (although they often think they know who reviewed it). We also know that it is difficult to fully anonymise a manuscript so that the authors cannot be identified – this means that it is difficult to ensure double blind review, especially in very specific, small research fields. Some journals offer their peer reviewers to disclose their names to the reviewers, and this is fully optional. So, check what type of peer review is used by the journal and accept to review it if you feel comfortable with it.

Do I Have Time for the Review?

Be aware that peer review is a serious work and takes time. Studies and surveys of researchers show that the time for peer review varies, from 2 to 12 hours or more. Make an honest judgement about your workload and see whether you will have time to do the review. The time will depend on the complexity of research presented in the manuscript, methodological approaches and statistical methods.

Can I Meet the Review Deadline?

Journals usually give 2 to 3 weeks to their reviewers to complete the review and submit it to the journal. Some journals publish so-called “fast-track” articles, where they expect reviewers to complete the review within 24 or 48 hours. Do not accept review tasks if you know that you cannot meet the deadline because of your other obligations (planned research experiments, deadlines for your thesis or manuscripts). If you accept to review the manuscript in good faith but then face a conflicting task or activity, contact the journal editor and ask for an extension of the deadline. Be honest and transparent.

Do I Have Competing Interests (Activities and Relations)?

We will discuss this issue in more detail later on in the chapter. Carefully assess whether you have activities and relationships that may affect your objectivity or increase your bias in reviewing someone else’s work. Check carefully the journal’s policy on competing interests. If you are not sure, do not be afraid to contact the editor and ask. Honesty and transparency is always the best way to address any dilemmas you may have.

When you agree to review a manuscript, you enter into a contract with the journal to become its consultant and to adhere to the journal’s policies and guidelines for the review of manuscripts.

If you have questions or doubts about your ability to review the manuscript, contact the editor and discuss the issues that you identified. It is better to prevent the problem then to try to solve it when it emerges later on.

Good research practice from the European Code of Conduct for Research Integrity:

  • Researchers review and evaluate submissions for publication, funding, appointment, promotion or reward in a transparent and justifiable manner.

When you accept to review a manuscript, the journal will ask you to provide your personal and professional information in the online submission system. Be careful to provide and accurate and true representation of your expertise.

In the case scenario from the beginning of this chapter, a senior researcher asks a PhD student to review a manuscript and submits it under her own name. This is considered a serious misconduct, as it constitutes impersonation of another individual during the review process.

When you accept to review a manuscript and receive it, you first have to see whether you can perform the review professionally, transparently and responsibly. Maybe the abstract did not provide a full description of the study and you discover that the manuscript is actually outside your expertise. Or you may discover that you have a conflict of interest. You should contact the editor and resolve these issues.

When you have the manuscript in front of you and are ready to assess the manuscript, ask yourself the following questions. First ask yourself if the research described in the manuscript is clear to you – are the aims and methods clearly explained and presented? The next question is about the quality of the research presented – are the conclusions justified by the data and are the methods valid? You do not have to be a statistical expert to assess study design and methodological approaches. Finally, make a judgement about the importance and interest of the results – are they relevant for the journal in question?

This means that you have to read the manuscript carefully before considering the review, as well as journal’s guidance to authors and to reviewers, so that you can get familiar with the journal’s scope and what is expected from authors to submit to the journal (supplementary information, checklists, permissions, etc.).

Be also aware that peer review has its biases, which may introduce systematic judgement errors. Research has shown that both the editors and reviewers may have bias towards positive results – they get published more often than negative results. Peer review may also be prone against new ideas and results (this is confirmatory bias) and against novel methods.

After you carefully read the manuscript and decide that it is within the scope of your expertise, it is time to write a review report. The purpose of the peer review report is to help editors decide on whether to publish the manuscript, but also to help the authors to improve the presentation of their work. However, keep in mind that your role is advise the journal and not help the authors publish their article. It is not a responsible practice to let a flawed article be published - peer review is considered to be a scientific stamp of approval of the article and its contents.

The format of a peer review report may differ from journal to journal – from a free-text commentary to a checklist with tick-boxes. The following is a guidance to write a comprehensive review report:

  1. 1.

    It is good to start with a very brief outline of the paper, which will show your understanding of the paper.

  2. 2.

    Number your comments – this will make it easier for authors to address them in their response and adequately revise the manuscript.

  3. 3.

    Highlight not only the weaker points of the paper, but its strengths, too, so that you provide a balanced assessment.

  4. 4.

    For each criticism that you have, clearly explain the reasons for it, and also indicate how critical your comments are to the assessment of the manuscript. You can, for example, indicate major and minor points of your critique, or indicate which comments must be addressed and which are optional.

  5. 5.

    Responsible peer review means that you stay with your expertise and not provide comments on the aspect of the manuscript or research presented in it. For example, if you are not an expert in statistics, you do not have to comment on statistical analysis (even if the journal asks for it). The responsible and transparent way is to state that you do not have sufficient expertise to cover that aspect of the review. In some journals, particularly in medical journals, special statistical reviewers assess the statistics in submitted manuscripts.

  6. 6.

    Check if the manuscript included important references and whether some important literature sources have been missed by the authors. List them in your review report – it will help the authors to improve the manuscript. Be careful about suggesting your own publications, as this may be perceived as self-promotion and a way to artificially increase the number of bibliographical citations to your work. Suggest your articles only if they are really relevant for the manuscript. You may also check whether all references are mentioned in the manuscript and that they are written correctly and consistently.

  7. 7.

    You are not expected to provide language editing of the manuscript. Your task is to assess the quality of research presented in the manuscript and not to improve its language and style. However, indicate when sentences are not clear so that research is unclear, or when the language and style of the manuscript require editing assistance.

  8. 8.

    Write clearly, in a neutral tone, but be decisive and give clear comments and suggestions. Do not write long reviews, be concise. Do not push your own opinions and hypotheses.

  9. 9.

    Do not use hostile or inflammatory language or make libellous or derogatory personal comments. Be aware that some journals have editorial policies to edit such language in review reports, or decline such reviews.

If you follow this guidance, you will grow into a high-quality reviewer (Box 8.1), who will never write poor review reports.

Box 8.1 Responsible Reviewer

  • has expertise in the research field of the submitted manuscript.

  • does not work in a competitor research group.

  • is familiar with research methods presented in the manuscript.

  • is able to assess the quality of data and methods.

  • is able to assess the validity of the conclusions.

  • is able to assess the significance of presented research.

  • writes professional, constructive and polite review comments.

Good research practice from the European Code of Conduct for Research Integrity:

  • Reviewers or editors with a conflict of interest withdraw from involvement in decisions on publication, funding, appointment, promotion or reward.

One of the questions that you have to ask yourself when you get the invitation to review is whether you have any conflict of interest in relation to the authors and the research of the manuscript.

Just as authors are asked to disclose their competing interests, the reviewers also have to declare their own either real or apparent relationships and activities that may influence their judgement (see Chap. 7). Competing interests may stem from institutional or collaborative relationships, personal relationships (family, friends), or financial relationships (funds and moneys received personally or by your organisation). Intellectual passion and personal beliefs may create a conflict of interest.

In the case scenario from this chapter, the researcher did not disclose her conflict of interest stemming from the closeness and competitiveness of her own research in relation to the reviewed work.

If you are not sure about whether your relationships or activities constitute a conflict of interest, contact the journal so that they can provide guidance on how to handle a potential conflict of interest. If you have a clear conflict, recuse yourself from the review.

Good research practice from the European Code of Conduct for Research Integrity:

  • Reviewers maintain confidentiality unless there is prior approval for disclosure.

Traditionally, confidentiality was at the core of the peer review process. With the move towards open peer review, where the identities of authors and reviewers are revealed to them and/or to the public (Box 8.2), confidentiality may not always be vrequired. However, an early-career researcher may welcome blinded reviews because it may allow them to be more honest in their comments and feel more protected in expressing their professional opinion.

Box 8.2 Types of Open Peer Review

  1. 1.

    Reviewers’ names are disclosed to authors together with review reports, but reviewers’ names are not publicly disclosed (published with the varticle).

  2. 2.

    Reviewers’ names and their reports are disclosed to authors during review; reports are published with the article, but without names.

  3. 3.

    Reviewers’ names and their reports are disclosed to authors during review; names of the reviewers are published with the articles, but not their reports.

  4. 4.

    Reviewers’ names and their reports are disclosed to authors during review; they are both published with the articles as a publication history.

You should carefully check the journal’s policy and adhere to confidentiality requirements.

Generally, you should consider that the manuscript you are reviewing is privileged information, i.e. authors’ confidential, private property. In some cases, for example when the results presented in the article have commercial potential, disclosure of such information may harm the authors intellectually and financially.

As a reviewer, you must not publicly discuss the work in the manuscript under review or reveal the identity of the authors.

Your review is also a confidential document, which should not be shared with other people. This means that you cannot seek help from other people with your review. You should keep it in a secure place on your computer. If you print a copy of the article, keep it locked – do not leave it around on your desk. When you complete and submit the review, delete the files or destroy the paper copy.

You should also not contact the author about the work in the manuscript under review.

In the case scenario from this chapter, the researcher who delegated the review to a doctoral student and submitted it in her own name has not behaved responsibly. This behaviour can be considered as a serious form of research misconduct. The proper conduct for a reviewer who is too busy to do the review is to decline it, or suggest another person (an early career researcher, for example) as a reviewer. This increases the transparency and gives proper credit to the person who actually reviewed the manuscript.

If you have any questions or dilemmas about confidentiality issues related to the manuscript you are reviewing, contact the editor and ask for guidance.

Good research practice from the European Code of Conduct for Research Integrity:

  • Reviewers and editors respect the rights of authors and applicants, and seek permission to make use of the ideas, data or interpretations presented.

Researchers value their role of peer reviewers because reviewing articles for journals gives them access to the latest developments in their field and may give them new ideas and approaches to their own research.

In the case scenario from this chapter, the reviewer and her research group used the new methodology for their own research and published it without reference to the manuscript they reviewed. Such conduct is unacceptable because reviewers should not use what they learned from their review for their own benefit before the reviewed work is published.

When can you use the knowledge you get from peer review? The answer is – only after the publication of the article. Then you can use it in your own research and cite it in your own articles.

What to Do If You Think There Are Integrity and/or Ethical Problems with the Manuscript?

As a reviewer, your primary task is to check the quality and relevance of research presented in the manuscript. However, you should comment on ethics or integrity issues.

Check whether the manuscript addressed ethics approvals for research with human participants or for research on animals.

You may be aware of competing interests that were undisclosed by the authors.

As a reviewer, you are best placed to suspect potential plagiarism or duplicate publication, problems with the integrity of data (falsification or fabrication), or problems with the integrity of analyses or conclusions. You can also suspect that the authors have failed on purpose to acknowledge evidence in the manuscript, which contradicts their results or views.

If you discover such integrity problems, contact the editor. Allegations of research misconduct are a serious issue and you should be able to provide relevant documentation to support your suspicion. Be aware that journal editors will follow established guidelines and protocols, which may include reporting the allegations to the authors’ institution(s), which may undertake a formal research integrity investigation. In such cases, journals will wait for the results of institutional investigation before there is action in the journal, such as a correction or retraction of the published article. In some cases, journal editors will publish and expression of concern, informing readers about the problems with the published article. At the end of the investigation, the outcome will also be published and the published record updated.