Panelized

On top of several other things I have going on these days, I'm also reviewing a stack of grants for an NSF panel. I have done a good amount of ad hoc reviews for NSF and other funding agencies before, so I didn't think this would be all that much different. Oh, but it is.

To start with, some of these proposals are pretty far outside of my expertise. When you get an ad hoc review it often (though not always) falls within your close field, which is why they sent it to you. When you have been assigned 12 reviews out of ~100, the odds are against even half of those being related to your work.

Another issue is the sheer volume. Proposals are dense and it takes a bit to get through on, summarize your thoughts on their system and approach and comment intelligently on the whole thing. Doing that 4 or 5 times takes a lot of time. 12 times for grants that are mostly outside of work? That's a whole new ball game. I would be lying if I didn't tell you that I am about 3/4 through and feeling like my brain has turned to oatmeal. Remember that I am also teaching, dealing with everything going on in my lab and trying to write a new grant for the Oct NIH deadline.

Finally, the feedback is giving me a bit of stage fright. When you do ad hoc reviews you submit them and go about your daily business. When you are a panelist, you instantly have access to all of the other reviews on the proposals you are reviewing, as soon as yours is submitted. As soon as the submit button is hit, you can see how your impression measures up against all the other reviewers, and I admit that has made me a little more vigilant in making sure I can back everything I say up with data. It's not that I did crappy reviews in the past, but the instant comparison between my opinion and that of Big Name X is slightly intimidating as I hit the submit button. In some cases I am right in line with others and in some cases I have a very different opinion of the proposal. In the latter cases, it means that I need to do more reading to make sure I can defend my position when it comes to the panel, which BTW, is yet another unknown for me.

I am glad I am doing this and looking forward to the experience, but the process has been more draining than I thought it would be and finding time to get to all of them has really cut into the time I spend with my family. I'll be happy when it is over.

9 responses so far

  • Namnezia says:

    How do you work around your teaching when you go to the review panel? Last year I was asked almost at the last minute to join an NSF panel, but I had to decline since I would have missed two days of teaching, and it is difficult to arrange guest lectures at the last minute, plus it would totally throw off the course.

  • proflikesubstance says:

    Carefully. It's easier with more time to plan and my course this semester has some flexibility in it. If it were next semester I would have a much more difficult time pulling it off.

  • Odyssey says:

    Welcome to the other side. 🙂

    This is a great illustration of just how important it is to write proposals that can be understood by non-experts. I know from experience that it doesn't matter how good the ad hoc reviews are, if the reviewers on the panel can't understand what you want to do and why, you won't be funded.

  • drugmonkey says:

    dude, you are giving me flashbacks to the weeks before my first NIH study section!

  • ianqui says:

    Yes, yes, yes. I found all of that to be true for me too. I was so worried that I wouldn't find major flaws that would be pointed out by others. But I was also surprised at how valued my opinion was by the panel, and in the end, I rarely feel like I don't stack up (occasionally it does happen). I'm on my 3rd one now (of 6 in the 3 year cycle), and despite the gigantic amount of work, I do enjoy it a lot.

  • This is so timely! I just agreed to be on my first panel in November, and I am freaking out a little about all the work, and about whether I am qualified.

    @Namnezia
    I have one flexible teaching semester (this one, since I teach and undergrad class that is offered both semesters. Me and the other prof have an arrangement to sub for each other in situations like this one) and one less flexible. I "schedule" in one class session to cancel ahead of time, figuring something will come up (I will be sick, need to travel, etc), so I have a bonus day just in case.

  • GMP says:

    @Namnezia: About flexibility in teaching: I like to teach MWF late-ish in the afternoon so I schedule 3x a 75 min slot. That means that for a 3 credit course I usually teach MWF 50 min each time, but when I have to travel that week I switch to 2x 75 min on the days I am there, or if I need to make up some time we just stay longer on a few other days.

    This has done wonders in the past and works great with grads, with undergrads so-so, as they have tons of labs in the afternoon. But you do what you must...

  • Eli Rabett says:

    What you will find is that your opinion matters most if you are the first reviewer and that the panel summary is not necessarily going to be related to the various individual evaluations. Everyone expects that you will not know about everything. Discussion is important.

    The first rule of panels is that you can push at most one proposal to the top. You cannot be for everything. If one of your twelve, MUST be funded, concentrate on how important that one is and let the rest fall where they do. If you have a bunch of good, but not earth shattering proposals, listen to the discussion and see if there is a great one in the ones that others have reviewed. Eli has been on panels where two out of thirty proposals will hit the funding line.

  • [...] been asked to do another NSF review panel this spring. Given that I did one in the fall of 2010, a second one pre-tenure might be a little excessive. However, with the new submission policies in [...]

Leave a Reply