Can n00bs be trusted to review?

Jul 22 2011 Published by under [Education&Careers]

As many readers know, the bulk of my research falls under the purview of NSF, but I have begun to shop some of what we do to NIH. As an initial foray into NIHland, we've actually done respectably considering I thought there was little to no chance I could make the case that NIH should fund our particular brand of science. I'm not prepping an R01 for October and looking to take a full swing at the NIH pinniata.

But in a lot of ways, NIH remains a bit of a mystery to me because it is set up in a very different way than NSF at almost every level, from day-to-day business to the philosophy that drives how the money gets distributed. Perhaps because I don't run in the circles that counts NIH as the only funding game in town, it all has a foreign and unwelcoming feel, even if that sounds ridiculous.

But part of that philosophy is what has Drugmonkey all giddy today. Apparently NIH is considering actually allowing n00bs into the inner review sanctum.... sort of. I guess that participation of Assistant Professors in review has been something that NIH has recently been trying to reduce even further than the <10% of reviews they have gotten from that population in the past. In an effort to change that trend this year NIH is now saying that they will bring in as study section reviewers, investigators who have not yet attained NIH funding. BUT, there shalt only be one per study section, they shalt not take on more than 2 reviews and will not be given primary status on even that review load!

NIH gives n00b reviewers their first chance to get on the review highway.

While I will admit that getting the experience of being in the room is a valuable thing, WTF is that? In my NSF panel experience, POSTDOCS had more review responsibility than that and there were two of those on the panel. I know that NIH loves to trot out the mantra (as quoted in DM's post) "And even though we generally use more senior and experienced reviewers on our panel because they have the depth and breadth of expertise that allows them to give a more knowledgeable assessment of the applications", but this is a bit fucking ridiculous.

People who do not have NIH funding might certainly need some assistance when it comes to things like feasibility in the time frame or maybe even broader picture significance, but is it really the case that they can't even be trusted to review the science of more than two proposals as a non-primary reviewer? Puleeez! It's not hard to understand why NIH has so much trouble with discrimination against the ESI crowd when it is actively propagated from the top down.

17 responses so far

  • Dude, giving the n00bs only two applications and making them tertiary benefits *them*. They get to attend the full study section, participate in discussion of every application under review, but their workload is dramatically less than for a regular reviewer. I'd *love* to be able to attend study section but only have this workload!

  • Arlenna says:

    I gotta agree with CPP there--as a noob who has gotten to review for NIH, JUST 2 would be awesome. My last experience involved NINE, which was just too many. Even though they were for a shorter format type of grant, i.e. 6 pages not 12--by the 6th-7th one I was having a really hard time keeping my head from feeling scrambled and making sure I didn't bias against those ones just because I was getting burned out.

    But I totally agree with you PLS about the "aww widdle baby reviewers, we'll pat them on the head and MAYBE listen to what they have to say..." attitude. That is irritating--but I'd still take the opportunity to see and do review over not being allowed at all, even if it meant dealing with some condescension.

  • Drugmnky says:

    I think it is silly to make them tertiary....I'd go for one primary and one secondary, personally. Most SROs are going to make sure the secondary is one of their more experienced folks on the primary-noob one anyway.

    A light load is a good idea but it makes it unlikely that those applications will be discussed. Ideally you'd want the new folks on two *discussed* applications. So I might go for a 4 application load, I guess.

  • Ink says:

    That bike is a thing of beauty. Dora rules!

  • proflikesubstance says:

    While I think having a lighter load is not a bad idea, being tertiary on 2 is ridiculous. As DM points out, the chances of actually getting into the thick of things that way is slim. Plus, one of the biggest lessons I learned from being on a panel was how to write for a reviewer with a full workload - a realization you might not appreciate with what amounts to fewer grants than I review per round as an ad hoc for NSF. Yes, you see how it all works, but in the same way that a player whose team goes to the Super Bowl, but who never gets off the bench. Sure, you were there, but....

  • [...] This is fantastic. ...CSR is piloting a new program that we call the early career reviewer, where we will take complete novice reviewers, people who have not reviewed for NIH before, very early in their career, probably new investigators. More thoughts on the matter from Your Humble Narrator and Prof-Like Substance. [...]

  • It is stupid to be outraged by this vast improvement on how things have been.

  • proflikesubstance says:

    I'm hardly outraged, nor do I think it is a good idea to take scraps and cherish them.

  • Eli Rabett says:

    Are you aware of this Great idea, examples of winning proposals.

  • drugmonkey says:

    oh yeah, GREAT idea. in fact if everyone writes like Striepen they will be in like Flynn. That Innovation section is a marvel!

  • ecogeofemme says:

    It's amazing what these "philosophical" differences between NIH and NSF do to the culture of the fields they fund. I've shifted from an area with no NIH support to one with lots of it. The new one is so hierarchical in comparison. It's not all because of how NIH does things, but I think it's a big part of it.

  • Are you aware of this Great idea, examples of winning proposals.

    AHAHAHAHAHAHAH!!!!!!!!!!! HAHAHAH!!!!!!!!! AHAHAH!!!!!!!!!!!!!!!!!!!

  • drugmonkey says:

    What does the NIH source of funding have to do with whether a job place is "hierarchical" or not, echogeofemme? Particularly as contrasted with NIH support?

  • Bee says:

    I know of at least two people who have served as full members on an NIH panel who did not have funding (one of whom was a fairly new prof and had no funding at all yet). It must depend at least in part on the particular panel.

  • Eli Rabett says:

    Yeah, you clowns read a couple of hundred proposals over the years like Eli and you would be amazed at how many of you clowns leave out the obvious and screw up the simple stuff like the budgets and budget justifications, time lines, etc.

  • femalephysioprof says:

    if i lived on the east coast, it would be great to have the lighter-than-average load i was given at my first meeting. i learned a lot and met up many interesting people. but, i most emphatically did not miss giving up three days of my life for the privilege of having exactly zero impact on the review process. it might almost have been ok (or at least tolerable) if the PO had told me in advance that i would be warming the bench at my first meeting and would gradually earn the right (and responsibility) of playing. didn't happen.

  • drugmonkey says:

    Just because the review of an application doesn't go your way, doesn't mean you had zero impact, IMO. Even for triage, your critique could have given the applicant hope to try again.

Leave a Reply