Notes from the AE desk (2): How I choose reviewers

Jun 27 2012 Published by under [Education&Careers]

I've now been an associate editor for roughly 6 months and am starting to settle in. Things are good and I feel confident that the journal is making solid and progressive changes. I've learned a lot (including why I am handling 3x more manuscripts that I was told I would) and enjoyed parts of the experience.

One thing that can suck, however, is finding reviewers. People are busy, I get it. I'm familiar with the concept. Most people are over committed and the thought of adding one more straw to their camel's back is not very exciting. The journal I work with has about a 50% return rate for reviews, which isn't bad. My own personal rate is a bit better, but I do have manuscripts that I have needed to ask 6-8 reviewers just to get 2 to say yes.

Identifying reviewers is not all that different from filling a seminar series in that you need to invite enough people to fill all the slots without having to wait for everyone to respond before checking with the next person. I don't want to waste time picking reviewers because the journal is stressing turn-around. So, I often will invite 4 or 5 people right away, hoping to get three. I have never had all 5 say yes, but have gotten 4.

Usually the authors provide a few suggestions for reviewers and I will typically pick one to invite. Checking the history of these suggestions is key, however. You may or not be surprised by how many authors suggest reviewers who would fall under the COI definition of NSF. After picking a suggested reviewer, I search the journal database for key words. That means, for those of you new people wanting more reviews, get your keywords into your account at the journals you are active with. I find that those who have a history with the journal are more apt to review for it. Barring enough suitable candidates, I start searching Web of Science (more inclusive than PubMed for us non-metazoan peeps).

Of course, it's not as straight-forward as a couple of searches and emails. At some point it comes down to making judgment calls based your knowledge of the people in your field. There are people I would never send a review request to, no matter the circumstances. I am also more likely to send a request to someone I recognize as doing good work than someone I don't know. My goal is not just to get decent reviews back, but also to make my job easier when sorting through them.

Finally, there is a Naughty List kept by journals / publishers that notes all those times you were 87 days late on a review or never responded to a request. You people get flagged in the system to be avoided (which is one way to reduce your requests).

Once all that information is sorted, the emails go out and I wait to see if I need to break out the reserve list or go with the starters.

8 responses so far

  • DrugMonkey says:

    So...why was the load 3X higher???

  • Though it sounds like you are not 100% in Pubmed world, a really useful tool for finding potential reviewers (as well as many other things) is JANE - the Journal/Author Name Estimator (http://www.biosemantics.org/jane/). You can pop in an abstract/full-text (be sure to remove the references though), hit "scramble" for security, and then click "authors" for a sorted list of authors who work in an area close to the text you are evaluating. It's fast and easy to use, and worth adding to the AE toolkit (for papers that are in the PuBmed-related scope)

  • Melany Gail Bushn says:

    I have kept my own journals forever. I don't know whether this would qualify me as a reviewer. However, if I commit to something I follow-up and meet deadlines.

  • proflikesubstance says:

    So...why was the load 3X higher???

    I didn't realize that the journal was "turning over" the AEs in my section. For about 3 months I was the only AE handling papers in two fairly broad topics for the journal. And when I didn't push back for a bit they just kept them coming. Eventually I had to tell them to cut it back and they did. With more AEs on board now, the load has reduced.

  • Mark P says:

    "Usually the authors provide a few suggestions for reviewers and I will typically pick one to invite. Checking the history of these suggestions is key, however. You may or not be surprised by how many authors suggest reviewers who would fall under the COI definition of NSF. "

    As an editor, if I eliminated anyone within this broad definition (especially the "co-atuhored paper rule", in some cases NO one qualified would be left. I think everyone has bias, and most of us try our best to rein them in when reviewing.

  • deevybee says:

    Sorry, but what is "the COI definition of NSF"?

  • proflikesubstance says:

    NSF defines Conflict of Interest as:
    PhD advisor or advisee (lifetime conflict)
    Postdoc advisor or advisee (5 years)
    Co-author (4 years) - although this has been softened to be only those co-authors with him you had substantiative contact.

    I have, on more than one occasion, had authors suggest their former advisees as reviewers.

  • anon says:

    I moved from a NSF-oriented lab to a NIH-oriented lab, and the COI definitions seem to be very different. Lots of reviewing former advisees' and advisers' papers at the latter. It has happened for some pretty big area journals.

Leave a Reply