It was my intention last week to blog a bit about the NSF Bio Preproposals while I was in DC, but that just didn't happen. What did I learn? Well, I can tell you what happened on one panel, but I got the impression that there is some decent variability in the system right now. Below are a few observations.
- The Big Idea is no more important than before. In talking to people leading up to writing the preproposals, there was a lot of emphasis on selling your Big Idea. Theoretically there was going to be less emphasis on methods and more on potential. Well, that kinda turned out to be bullshit. They were essentially judged like mini-proposals.
Cut rate. Our target cut was supposed to be 80%, with 15% landing in the category of "High Priority Invite" and 5% in the "Low Priority Invite". In the end we were closer to 20% HPI and 15% LPI, with everything else landing in "Do Not Invite". How many from the LPI category will actually get invited was not clear.
Two flavors. A lot of good science went into the DNI category. If you get a panel summary you will know why: The DNIs with a summary were discussed, the rest were triaged.
Triage. Anything that got less than three ratings of "good" in the preproposal stage was not discussed. Slapped a boiler-plate panel summary on those and moved on. Roughly 25% of the initial pool went undiscussed.
- BI still matters. Thought you could short the Broader Impacts section just because it was a preproposal? Wrong. A crappy BI section bumped several proposals to DNI.
- Small proposals get killed. For a long time there has always been the party line at NSF that there was no reason for a small grant mechanism because you could always send in a small proposal. Well, guess what happens when you remove the budget and measure all proposals with the same stick? Yeah.
- NSF is worried about new PIs too. Much was made of the concern for the N00bs, and NSF is watching this closely. If new PIs get disproportionately whacked, there will be a correction (This goes for RUI as well). BTW, current average to grant is three years.
- Possible funding from preproposals. One possibility that came up was that the very top preproposals might, in future years, just get funded without a full proposal. Things are still in flux, but my panel felt it could make some awards now.
- Incorporation of preproposal panel members for the full proposals is being discussed. There was some concern by panelists over having two very different hoops for the PIs to jump through, between the preproposals and the full proposals. One way to keep some continuity could be if preproposal panelists agreed to serve as ad hocs for the proposals that got invited and for which they were the primary reviewer in the first round. Everyone started with eight primaries, but I don't think anyone had more than two or three make it to the invite stage, which would be a manageable load to ad hoc.
- Shrinking everything. The full proposal may be going on a diet soon, too.
- Three's often enough. Most proposals got a pretty fair treatment with three people having read them and no ad hocs. There may have been some that could have benefited or been hurt, but overall a panel-only review didn't seem to be an issue.
- The long month. The time line to notifications is roughly a month for those getting an invite, longer for those that got bumped.
So, do I think this is going to be an effective process? It's too early to tell. Just based on my preferences as a PI, I think I would prefer something more along the lines of the 8 month cycle that MCB has gone to, so that you can more effectively manage one's grant load. As it stands now, anyone who can apply to DEB and IOS could have 4 preproposals going in every January. The 8 month cycle with 1 proposal per round would spare people the year between submissions and spread the load out a bit.
But we will have to see how it all goes. One thing was clear: no one, at NSF or as PIs, really know how this is all going to play out.