Where NSF preproposals are a Good Thing

Feb 24 2014 Published by under [Education&Careers]

I'm on record as not particularly liking the preproposal system that NSF Bio has been essentially forced to moved to. However, I will admit that some of the advantages of the system do indeed accomplish what they were set in place to do. Namely, the short format means there is a low activation energy for getting in the game.

Why is that important? For reasons outlined in Michael Tomasson's latest post about dealing with triaged proposals. Essentially, every PI needs to have a few ideas in the mix and getting feedback on them quickly is key. If your proposal gets trashed in review, then it's time to rethink it and move on with something else. The preproposal system allows you to do this, but providing feedback on a general idea (and in 4 pages, that's about all you can do) for relatively little effort.

The downside, of course, is that you have 8 months before you can resubmit. This is why it's important to be cycling several preproposals at the same time. With the 4 page format you can afford to do it, even if you're juggling it with teaching.

If you're tenure is largely dependent on bringing in NSF funding, you need to avoid being stuck with a single proposal you keep trying in slightly different ways and be willing to walk away from an idea that gets poor feedback at panel. By at least year three you should be maxing out the 2 preproposal limit for your favorite division and probably looking at ways to hit other calls as well. There are plenty of "cross-cutting" mechanisms to explore, as well as the CAREER awards (which can also be a landing spot for re-thought non-invited preproposals).

The most successful PIs I know have also been the most flexible in their funding search. Preliminary data can be used for multiple different proposals with different foci, as long as you can be creative in the questions you want to explore.

12 responses so far

  • anon says:

    I think you should be more up front about your perception of this being based on anecdotes. When talking to POs at NSF, they frequently mention that the PIs most likely to be funded are those that focus on one project - their very best - rather than play the strategy you mention. I guess one size does not fit all.

  • proflikesubstance says:

    It's also worth considering the different wants of different groups. A PO wants your best proposal in it's best crafting and does not want the system "bogged down" with additional proposals. PIs want to be able to do their science.

    Yes, many labs do fine of basically banging away at one topic and trying to string one grant along after another. But given the current funding climate, that's a high risk strategy, IMO. When you look at how much good science is probably left on the table, it's hard to justify the hole-in-one strategy.

    What do you consider a successful lab? What does the funding strategy of that lab look like? Do they have basically one research track or is the lab diverse in its topics? I think you'll find that this strategy isn't anecdotal.

  • proflikesubstance says:

    Related: Your most fundible idea and your favorite idea may not be the same thing. Obsessing on the latter is why labs fail.

  • Jim Woodgett says:

    The classic pedestrian and driver dichotomy: we are both. As applicants, we want to ensure the best possible funding chance and so throw more material at the wall (of decreasing quality as we are increasingly tapped out for ideas). As reviewers, we don't want to be working through every weekend and evening drowning in grant applications (and agencies hate additional applications as they reduce the "success rate"). Applicant and reviewer burdens are circular and fuelled by over-production of scientists. There needs to be some over-production so that competition is real and effective. Science should abhor resting on laurels. It's when that over-production is allowed (or encouraged through bizarre incentives) to increase to more than three fold over availability that major problems arise. Peer review is lousy under stress and becomes even lousier as system stress increases. As funding pressures increase, our ability to deal with it decreases. The "Pull Up!" cockpit alarm has been triggered but we're at stalling speed.

  • proflikesubstance says:

    I have yet to meet the altruist PI more interested in reducing the review burden than sending in ideas. The preproposal stage is a good opportunity to test which ideas are ready for prime time.

  • I dislike the NSF turn-around schedule (essentially yearly). This is way too long for jr. faculty that need funding for continued life. This problem also touches on another hot button. One of the benefits of the NIH one-resubmit policy is that it does force you to move along, and do something new. When I've reviewed NSF (bio) grants, there are ones that are back for the 5th or 6th year. Still trying. This has two problems - the PI not moving on, and the pipeline clogged up (only sr. people can afford to do this) making it that much harder for young folks.

  • Joshua King says:

    Don't forget the other pressure that will encourage you to submit more: the somewhat unreasonable expectations of Deans and tenure committees. Since I arrived at my new school (3 years ago) I've been getting pressure to bring in a second NSF standard grant or something even bigger before I go up for tenure to "demonstrate I can bring in money here, not just where you were before." The result: I've applied for all sorts of grants and written proposals for all kinds of ideas that have little or nothing to do with my "bread and butter" research. The end result: my bread and butter is still the only funded line, and the last round of preproposals suggest that the bread and butter is most likely to be funded in the next year or so. I've enjoyed exploring all the other lines of research and it has resulted in some excellent, likely long-lasting, collaborations, but the trade-off between exploring new territory and building on established lines can be a tough call. Proposing broadly probably isn't a bad strategy, but I'd really love to see data on the "payoffs" for how often it pays off, so to speak.

  • proflikesubstance says:

    PT, In full disclosure here, I did have a proposal that I spent way too many drafts sending in, but did eventually get it funded. It probably could have benefited from me taking a cycle off to concentrate on the generation and analysis of supporting data, but hindsight and all. It helped that we already had a funded grant in the lab so a little pressure was off, but I was certainly risking going too far down the rabbit hole with that one.

    JK, when I'm talking about proposing broadly it doesn't necessarily have to be using a variety of different systems (although it certainly can). I am more referring to exploring new ways to interrogate a set of questions. If what you do fits nicely into DEB, is there an interaction in your system that could be pitched to IOS? If it complements your current stuff, all the better, but I think you need some preliminary data to make a go of it. So, you can't just fire off proposals in all directions. For me it's about being creative about how we attack a central question.

  • Jim Woodgett says:

    proflikesubstance - it isn't altruism - it's reality. The more stuff gets thrown at the wall, the less effective the peer review. Someone has to do it and that someone is us, the applicants. I agree that pre-proposals are a good idea, but I worry that the quality of assessment at this stage (little detail, not well rounded ideas, etc) is relatively low. The risk of false negatives (where good ideas are blown up) increases as the burden increases.

    In Canada, we are moving to a 5 review per application scheme (and multiple phases) where all holders of funding (CIHR) are obligated to review. Do the math and you quickly see that stick will get chewed down very quickly.

  • proflikesubstance says:

    Jim, who is going to be the lab that waits a round or two to submit that second application just to keep the system healthy? Where is the benefit to the individual for doing that? I don't see it.

    I have sat on two preproposal panels in different divisions and I don't think that the "false negative" problem is a big deal. First, panelists do a pretty solid job of vetting the bullshit. It may not be perfect, but when only 20% are making it through, you can bet that the panel looked pretty hard at them. Second, even if a few worm their way through, it'll be exposed in the full proposal round, where the funding rate is only 30ish% of what jumps through the first hoop.

  • DrugMonkey says:

    POs are not the best analysts of sound strategy from the perspective of the applicant. Take their advice as part of a much greater whole.

  • Michelle Elekonich says:

    Also know that different POs from different programs may give you different advice based on their program rather than the whole.

Leave a Reply