The Ethics of Peer Review

Dr. Alana Kirby:
So today we're going to be talking about ethics in the peer review process. Our listeners are mainly junior members who don't have a lot of experience in peer review and have a lot of questions about how to approach peer review in an ethical manner. So I think the first question that comes up is really: What are some examples of conflict of interest that should make a potential reviewer decline the invitation to review?
 

Dr. Claudia Trenkwalder:
The main conflict of interest is if you are doing almost the same or even exactly the same project, and maybe you are even going to publish it at the same time, or you have already submitted a publication either to the same or to a different journal. And this does happen sometimes because you are an expert in the area. And so it's very likely that you get papers for reviewing, which are very close to your own project, or maybe you are in competition with the authors for performing the same trial or study. I think this is a major conflict.

The other conflict is that, you know, the authors very well. And this Is a delicate one, as sometimes you still can review it If you feel you can objectively look at that paper. But if you have already a close contact or have published a lot together with the authors and similar work, I think it's more ethical then to decline.
 

Dr. Alana Kirby:
Is it ever appropriate to ask the editor who is inviting you to give their opinion on whether or not a conflict of interest is sufficient for you to decline?
 

Dr. Claudia Trenkwalder:
I think if you have doubts if you should decline or not, a short email to the editor declaring what kind of conflict you have may solve that. And then the editor can decide. I already have the situation that I sent the editor. We are doing the same project, but we, we are not yet as far.

And, but I feel there's a competition. And then the editor told me, this is not a conflict at all. We have three reviewers. You just you are an expert. Send me your opinion. And then I think it's fine. So it's then up to the editor to estimate and to evaluate your review in the light of a possible conflict.
 

Dr. Alana Kirby:
I've heard other editors that I've spoken to before that providing that kind of information allows them to take your review in context. It's not that your review is the absolute truth that can not be interpreted in any way. It's that it needs to be weighed in the sum of all the other information that they get about the paper.
 

Dr. Claudia Trenkwalder:
But sometimes, you know, as you are really inside that field and inside a similar project, you can also be a very good reviewer because you know, all the problems that may arise with this kind of studies that others do not expect because they are not performing it.

Dr. Alana Kirby:
So once you do accept, what are your duties of confidentiality? Can you ever mention that you were a reviewer on a paper?
 

Dr. Claudia Trenkwalder:
I think you should not. I know that sometimes people are doing this, but as long as the paper is not published, you can not do Do it at all. So I think this is an absolute. It's absolutely mandatory. So you keep the confidentiality, you will not contact the authors or you do not give any hints, let's say, at when meeting the authors by chance or at a Congress.

I know there are some temptation to say, "Oh, I did review your paper. This was a wonderful paper." But I would not do it. Wait until the paper is published and then maybe after publication sometime then you can say, and by the way, I also saw your paper quite early or something like this. I think this would not violate the ethics. But during the publication process, it should be confidential.
 

Dr. Alana Kirby:
This leads into kind of a hot topic in the field, is that of open reviewing. There are some journals these days that are publishing the identities of the reviewers when they publish the paper. And I think there are pros and cons to the closed and the open approach. I was wondering what your thoughts were about that?
 

Dr. Claudia Trenkwalder:
Yeah, I think I have mixed feelings here. As you say, there are pros and cons. I think the pro is that first on your side, you will think very carefully what will you write as your name is disclosed. Maybe this is also an advantage. And you also get credit for your work because sometimes it's a lot of work to review a paper. And also then the authors can finally, after publication contact you and they are, "Thank you. This was a helpful review," or whatever. So these are the pros.

The cons are that even if you... if you will give some critique, this can be perceived as some hostile action. So you never know what will happen. And so, on the long term, the collaboration with the authors can be disturbed. And as you never know what will finally happen with this, because then sometimes maybe the editor will declined the paper. So it's not your fault, but the paper is declined. And then the author said, "Oh, this was because of that review."

I think altogether I am more cautious than positive for open review. It may be different with review papers or meta analysis compared to original.
 

Dr. Alana Kirby:
Yeah. Whenever I have a paper myself, it's always useful to get a stringent critique from a colleague, but it is a different situation to get that from a reviewer who you don't know,, or who may be in competition with you, than it is to get it from someone who you, trust and solicit.

 So many junior members get their start in peer review by working with a mentor who has been asked to provide a review. And then they participate in that review process. When you have a mentee who is helping you with a review, what do you do to allow them to participate in the process with the consent of the journal?
 

Dr. Claudia Trenkwalder:
I'm always sending them the name to the editor. And I have the experience that this is very helpful because the editors really appreciate, because they may get new reviewers for the next paper. And this is exactly how it works. I knew this at least from my colleagues then after some weeks or months — sometimes only weeks later — they are coming to me, "Oh, what should I do? I got an invitation from this esteemed editor. And I told them, "Yes, see, because I mentioned your name and you helped me with the other review." So usually the editors have no problems. If one openly declares a young member helping with the review and and they will send it to the young member the next time.
 

Dr. Alana Kirby:
So it's a, it's a career building opportunity.
 

Dr. Claudia Trenkwalder:
Right.
 

Dr. Alana Kirby:
So a reviewer has a significant amount of power that accompanies the responsibility of reviewing. And I think that, you know, we've all kind of heard horror stories about reviewers misusing that power. Is there anything that you think of as an absolute, "No, this is should not be done. This is academic misconduct," in terms of actions that a reviewer might take?
 

Dr. Claudia Trenkwalder:
Yeah, I think sometimes you realize only after you have a agreed to review the paper, that there is some competition with your own work or that you have a conflict here. I think this is a difficult situation and this may lead to this misconduct then, because then the paper was still being reviewed. And there will be some misconduct by, for example, slowing down the process because you want to send off paper. This all arises from competition, especially if grants are involved here. And I think a reviewer should not do any review, you not slowing down, not accepting, whatever, if there is a competition with his own work. Because there are some subtle actions that are taking place here.

And the best thing is just to stop and to decline.
 

Dr. Alana Kirby:
Basically to, if there's any potential situation where, either intentionally or unintentionally, you might have benefit yourself at the expense of the person who is reviewing, then just don't put yourself in that situation where that may arise.
 

Dr. Claudia Trenkwalder:
Yeah. And don't put yourself in that situation when you always have to decide, do I do this because of some competition, or do I do this because I have lots of work to do and cannot do it faster. So before asking yourself, these questions just declined the review.
 

Dr. Alana Kirby:
You're so clear about what you should and should not do that it's hard to even come up with some of these bad viewers might do. Hopefully I don't think that our listeners will be doing those things either. Is there anything else that you would want our listeners to know about the ethics of peer review?
 

Dr. Claudia Trenkwalder:
Yeah, I have sometimes the situation that you are receiving an invitation to review, and then there is only the abstract at the end. So just from the abstract, you cannot perceive how big the work is or what the paper really is. Then I agree to review. Then I download the PDF or whatever, and then I see, oh, there are seven supplementary file. And this is a manuscript with 50 pages and 10 figures. And that it's, it's a large amount of work. And then I realize, even looking at the statistics, that I'm not able with my expertise really to review this sufficiently. I think this is a problem. After agreeing, you realize this is different as what you have expected this paper to be.

And in this case, you have to take decisions. Either You say, I'm sorry, this is too much work. I cannot do this right now, or I need more time. Or I need another one who helps me with that review. And I have written sometimes to the editor, I can give you some clinical review on that paper, but I cannot evaluate the statistics and I cannot correctly evaluate some experimental procedures that are within that paper that I didn't know before.

So I think it's always better than to concentrate on what you can do is just to send a review saying this is a fantastic paper. This is easier than to say, but finally, it's not really helpful. So it's often as a surprise, what you get after you have sent your agreement.
 

Dr. Alana Kirby:
I think that goes along with the duty of timeliness. And when it turns out that it's a large amount of work, that that may be something that is not possible. But I'm, I'm interested in this other idea that you brought up, which is that, when you think about ways to review a paper well, of course it is bad to be overly critical of a work in a way that is unfair to the authors and unfair to the work, but it is also not good to not put in as much time and be overly praiseworthy because you have not spent the time to really dig in and see the flaws or the things that can be improved in the paper.
 

Dr. Claudia Trenkwalder:
And I realized sometimes then if I get the reviews of other reviewers, this is quite interesting, that you have three reviewers, for example, and two have put a lot of work and have highlighted several flaws of the paper. And some of them are in agreement with the other reviewers, others are not.

And then there's the third reviewer oh, this is a great paper. You can accept it. And I thought, oh, this wasn't easy work to do. I think this is also unfair for the authors, because this is not a good review for them.
 

Dr. Alana Kirby:
Yeah. The peer review process really should be about making the papers as good as they can be.
 

Dr. Claudia Trenkwalder:
One issue that comes up more frequently now that authors contact you before submission, and you know that the journals want names for possible reviewers sending by the author. So they contact you upfront saying, would you be able to review my paper? And I'm not so sure what is the official conduct here? If this is a topic that I know well, I would say yes, I can do it. But this often implies that some good colleagues mean that you definitely should positively review them. So, this is also an ethical problem if they contact you upfront. And because then they may know that the editor will send the paper to you. But I think this is something that should be discussed among the editors once, if they always ask for names for reviewers, how these reviewers should finally behave. Should they disclose that they have been contacted upfront or should they not say anything? So I think there is undefined zone here.
 

Dr. Alana Kirby:
I would imagine this is an issue currently because my understanding in talking to colleagues is that it can be quite hard to find reviewers these days. And so I have known a couple of colleagues whose papers reviews have been delayed because of the difficulty for the editors to find reviewers.
 

Dr. Claudia Trenkwalder:
Yeah.
 

Dr. Alana Kirby:
And so of course the authors want to try and minimize that delay so that their paper can be evaluated and hopefully published as soon as possible. But it does remove perhaps some of this anonymity that we think of as being important to unbiased peer review.
 

Dr. Claudia Trenkwalder:
There is some bias then, in this process, but I can understand the editor because they really have a problem to find reviewers with the, really, the many publications also with the online journals that are just circulating. But it affects the unbiased review.
 

Dr. Alana Kirby:
Hmm, I guess we will have to, we'll have to think about that one. Maybe the default could be, if something is questionable to you, then ask the editor who is asking you to review.
 

Dr. Claudia Trenkwalder:
I think this is the best message. So if you have doubts or if something is questionable, just send an email to the editor. And in my experience, the editor is always responsive and gets back to you.
 

Dr. Alana Kirby:
Well, thank you so much for sharing all of your thoughts and your experience. It's been a wonderful conversation.

That completes episode eight of the peer review podcast. In this episode, Dr. Trenkwalder shared her thoughts about the ethics of peer review. Next week, we will tackle an interesting topic: What not to do in peer review. Thank you so much for listening.