REPOST: How I prepare for an NSF review panel

Mar 19 2013 Published by under [Education&Careers]

I posted this last year before heading to a preproposal panel. With the bulk of the DEB/IOS panels on the horizon, I thought it would be a good time to repost it.

The first time I went to NSF for a panel I wasn't quiet sure what I needed to do to be prepared. I read like crazy and showed up fairly apprehensive about whether or not I would be prepared to do a good job. Facing my second panel in a few weeks, I have a much better feel for preparing this time around. Below are some suggestions if you are making a trip to NSF in the future.

- Get you reviews in be the requested week before... or close. This isn't just for the benefit of your PO, it is for you. NSF has a couple of different choices behind the panelist sign-in:

You reviews all get filed under the Panel Review System tab, but once you have submitted a review for a proposal you can go check out the other reviews under the Interactive Panel System tab. This is important. Not only because you'll want to know if your assessment is radically different from the other panelists (and ad hocs if you are on a full proposal panel) so that you can reread it, but for the reasons below.

-Decide which proposals you are going to fight for and find out who you will be fighting. You will be going into the discussion with a short list of proposals you really liked. By looking up the other reviews ahead of time you can figure out how much of a fight you have on your hands. What did the other panelists think of your Good List? Go look those panelists up and find out what they do. Is their expertise strong in the proposal topic? People may write opinionated reviews, in either direction, before the panel discussion and change their minds during the discussion. This seems especially true if they were out of their element a little when writing and didn't "get" the proposal. Don't be scared off of fighting for something that got hammered by another panelist, but figure out where they are coming from.

-Identify your panel enemy. Alright, maybe "enemy" is a little strong but in both of my experiences I have found that I share the majority of the proposals I am responsible for with a small group (3-5) of other people. Among that group, there seems to be one person who thinks every idea put forth is doomed for failure. Their reviews often focus on poking holes in any perceived methodological flaws, as if the PI(s) have no recourse but to follow exactly what is written without modification. This is the person you will need to convince that the proposals you are fighting for will work. Read their review, know what they work on and be prepared to spar with them.

-Get a sense of the panel's mood. After reading the reviews you have access to, are they generally positive or negative? Is this going to be a panel that spends most of its time finding reasons to whack a proposal or finding reasons to support them. There is always a mix of both, but there can be a very different feel to panels that go one way or the other.

-Figure out if anyone else shares expertise in your area. Scan the list of names on the panel. Know anyone? Read anyone's work? I wouldn't go e-stalking anyone or reading people's stuff just for kicks, but I find it useful to have a little idea what other people do and what they may weigh in on.

That's what I do in the lead up to a panel. I'm sure some of the more experienced Peeps out there may have other suggestions as well. Above all, remember that you are there to talk science and participate in the process of getting people funding. As a group you'll have to make some hard decisions and it's easy to walk out of one of these feeling a little depressed by the amount of good science that doesn't make the cut. But do what you can, learn what you can and get to know the other panelists - there's a decent chance they will be reviewing one of your grants some day.

7 responses so far

  • Excellent post. The picking your battles part is extremely important. If you fight for everything, then the value of what you say will be discounted.

    Be careful in your critiques - don't make yourself into someone else's "enemy" early on.

    During breaks, lunches, etc, take time to meet your fellow panelists - not just cause they will be reviewing your grants, but because they quite likely have something worthwhile to say to you. Its like meeting networking, but with less performance pressure.

    Finally, enjoy yourself. Learn something new. Understand that you *are* making a difference.

  • I always have the moment of baited breath right before seeing what the other reviewers thought....did I miss something cool about the proposal I just have a poor to? Was I the only one who loved it? Interestingly, I have found that in 80-90% of cases, the panelists are pretty similar in their view of a proposal. It's the 10-20% where there are strong differences where things get....interesting.

  • proflikesubstance says:

    Yeah, I have already submitted a few of my reviews and was disappointed that no other ones were in yet for me to compare to.

  • AnonPI says:

    PLS, I think you and I might be on the same panel, if it is next week and includes the word "hybrid."

  • proflikesubstance says:

    Nope, not my panel.

  • Emilio Bruna says:

    This is my first panel under the pre-proposal system, and one of the things I've been struck by is that 5 pages really is enough to convince me ideas are interesting and that I do/don't want to read a full proposal. The new format puts a real premium on being able to write well and getting feedback from colleagues.

    As for preparing, I also try to scan the ones I haven't been assigned to see if there's one or two I might be interested in. There's no pressure to prepare a review, but you can still participate in the discussion and provide insights when people disagree.

  • Emilio I was struck by that too on my panel. The shorter format also let me dive into the literature to see what the state of the art was for a particular question so I could figure out for myself whether the proposal would be filling a clear need inthe field.

Leave a Reply