Panel decisions for this year's preproposals are staring to come in. I got my first response last night and was lucky enough to get invited* for a full proposal for this one. Sure, I was in bed and turning off my phone when the email came in, but I had to go check out the reviews (not a recommended pre-bed activity, btw). A few things jumped out at me right away:
- Clearly there is lots of variation among panels with their approach to preproposals. The panel I applied to had considerably more applications than the one I served on. So many, in fact, that the had to deal with them in two panels. So, not only is there the complication of dealing with different audiences and expectations between the pre- and full proposal evaluation, but we also have multiple preproposal panels for the same application set. I don't really see that as a major issue, but I would be curious to hear from the POs whether there was consistency or differences.
- The invite rate was <20% for this panel. They were tougher than my experience, but we used high and low priority invite categories without knowing whether any of the proposals in the LPI group were going to get invited. If they don't, then the numbers will be close to what my panel did.
- Though there has been some debate about whether there should be a preliminary data requirement for the preproposals or if they should be just Big Ideas, I packed mine full of preliminary data and the reviewers responded favorably to it.
- Because all reviews were from panelists, they were short. This is not unusual and often a way you can tell which reviewers were panelists on your full proposals. If you reading 26 of these things, you're not spending a lot of time writing all your thoughts out when you can just discuss it at panel.
- Based on the reviews and my experience, I can tell two things:
1) My proposal went in decently rated (except one person who "didn't get it") but benefited from the discussion. I watched this happen to a couple of proposal in my panel, where the shortcomings seen by one person are explained away by another and soon the proposal looks better to everyone.
2) I had an advocate at the table. The shortest review seemed the most excited by the work and that person clearly influenced the discussion heavily. If you don't have someone like that in the room for your proposal, you're screwed. Thank you to whoever that was.
So now the big questions: How much flexibility do we have to respond to the panel comments? How much should we?
Much was made of the possibility that someone could make major changes between their preproposal and their full proposal because there would be no one in common reviewing both to keep them honest. The POs in my panel said that they would be watching for that and some mechanisms were suggested to make sure this didn't happen, but no one asked the real question: why change the proposal that got you there?
In my case I am seeing some familiar criticisms in the preproposal panel summary that I feel I should address. My lab has been working hard towards dealing with these issues since we've heard them before. Now we can, but it means changing at least one aim substantially. Do we go for it, realizing that it may bring a new round of critique? The current proposal was enough to get us invited to the dance, is it worth trying to switch dates while there? Is it okay by the POs to make those kinds of changes or will they cry foul? All of this is up in the air.
So, I'm going to call my PO and see what they have to say. All of this is new territory for everyone, so I imagine that any reasonable modifications will not be denied. Interesting times ahead.
*Don't get all panicky if you haven't heard back yet, different panels are handling notification in different ways and times.