Should NSF become NIH Light?

I was going to explain this in the Strassmann blog discussion, but since even G-rated comments about people writing like they are building a medieval labyrinth seem to have a way of disappearing over there, I'll post it here. Ya' know, because we have to keep this discussion civil.

If you have been following the discussion over at the Strassmann blog regarding changes to NSF, you'll notice a few familiar themes. One of the most popular is comparison to the Canadian system and a call for smaller grants to more people. This debate was a particular hobby horse of the now tumbleweeded NSF is Broken forum. Suffice to say it's not a good solution without overhead reform, another re-occurring topic. Unfortunately, NSF's hands are tied on that front as well unless NIH wants to play along. So what to do?

IMO, the calls for a hybrid system make the most sense and we need not even leave the DC metro area to find a model for how to make it work. Regular readers will know that I have some limited experience applying to NIH. Whereas NIH has it's issues with success rates, one of the things they have right is the variety of mechanisms for funding. If we think of the traditional NSF grants as an R01-like mechanisms (the big individual PI grants), then I think NSF would be serving it's constituents well by instituting an R15 and R21-like mechanism of limited budget funding.

For those unfamiliar with NIH, the R15 awards are limited budget ($300K/3yrs in direct costs) awards aimed specifically at institutions that are not in the top XX% of NIH money getters. This mechanism would not only give NSF PIs a limited budget option, but also deal with the "the top schools take all the money and leave none for the little guy" concern, because the top schools would not be eligible. This has been a very popular program at NIH.

The R21s are also limited in their budget ($275K/3yrs2yrs), but are for exploratory research. Another major (and valid) criticism of the NSF process is that the bar for "preliminary" data is so high that you need 50% of the work done already. The R21 mechanism would solve this issue and provide an alternative to NSF's seemingly arbitrary EAGAR program by providing a source of funding to get a line of research off the ground.

If you want to get crazy, we could throw in an R03-like mechanism, but I feel less strongly about this.

But obviously the money for these programs can't come out of thin air. Instituting the change would require cutting some money from the regular programs, so how would that work? I don't know the numbers, meaning I'm just tossing out ideas here, but I would support something like a 5% cut to all existing grants (yes, mine included) and a slight reduction of the pot for the next round. While people will howl about further reductions to the pot given the current funding rates, I think the potential success of these programs will make up for it. In addition to that, I think panels should be empowered to tell investigators that if they want to resubmit a certain "regular" project as it is, it should come back as another (smaller) grant type.

So far, though, we have not dealt with the review load problem. For that issue I would support both a shorter grant application (8-10 pages) and a deadline set up like MCB has gone to - two deadlines a year, but an 8 month cycle of review with a limit of two proposals per PI per year. I think the 4 page preproposal is relatively useless, particularly when there are different people reviewing the preproposals and the full proposals.

Perhaps this doesn't fix all the problems instantly, but I think it goes a long way towards a good compromise. There may be good reasons why NSF is uninterested in the smaller grant mechanisms, but the argument that one can always submit a smaller budget doesn't fly because the science is judged first. There needs to be a way to separate out these smaller proposals so they can be judged against one another. I could see budgets for each of the smaller mechanisms limited to something in the $100K-$150k range in direct costs over a three year proposal. We could even shorten the length of the proposal for these grants to 6 pages if you want.

There's my suggestion, feel free to make your own.

22 responses so far

  • Isis the Scientist says:

    The R21s are also limited in their budget ($275K/3yrs), but are for exploratory research. Another major (and valid) criticism of the NSF process is that the bar for "preliminary" data is so high that you need 50% of the work done already.

    I'm not so sure this has worked out so well for the R21.

  • proflikesubstance says:

    Just because it hasn't uniformly worked as currently implemented, that doesn't mean it can't.

  • Kelli says:

    R21 is limited to 2 years with a total direct of $275,000.

  • These are interesting ideas. I have found the judging process at NSF to be much more open than the NIH one. Exactly how proposals are evaluated is another topic. Could it be better at NIH, or NSF?

    I do not think it is fair to imply that I randomly delete comments at, since I only deleted two one sentence comments of no substance. I have put them back up, so you can see for yourself. The statement at the beginning of this piece implies I delete comments if I do not agree with them, and I don't. I think this has been a useful discussion but feel a little doubtful that we can have a great outcome until we can figure out how to convince everyone that DEB and IOS at NSF deserve more funding because of the importance of the things we discover.

  • Alex says:

    Isn't the NSF EAGER mechanism in the same general category as the R21? I don't claim that they're exactly analogous in implementation, but they are for exploratory research.

    And NSF has the RUI mechanism for undergraduate institutions. Since many (though not all) R15 recipients are undergrad institutions, there's similarity.

    If you want to talk about doing these things differently or better, great, but I don't know that we can claim that NSF is just completely ignoring these categories.

  • Odyssey says:

    As Alex said, there is a grant mechanism at the NSF for PUI's.

    One correction to what you have written, MCB does not necessarily have two deadlines a year anymore. The eight month cycle means every second year, with 2012 being one of them, there is only one deadline for research grants. Personally I would prefer they go back to the six month cycle.

  • proflikesubstance says:

    Kelli, thanks for the clarification. It's a mechanism I haven't tried for yet so I'm less familiar with it.

    Joan, I think NIH and NSF each have their good and bad points in evaluation. No system is ever perfect. I think arguing the subtleties of proposal evaluation should come after getting better mechanisms in place to categorize them prior to evaluation.

    WRT comment deletion, my feeling is simple. If you delete comments based simply on whether you judge them to have worth, it is a form of censoring. While I can suport comment removal for vicious personal attacks or anything involving bigotry, making a call on the value of non-offensive comments is a slippery slope. It sends the message to commenters that they could be censored at any time and stunts conversation. I have little interest in rehashing comment moderation policies and their effect (the bloggosphere is awash in posts on the topic) here in this thread, suffice to say I felt more comfortable moving this conversation here.

    The problem with saying that DEB and IOS need more $$ is that everyone feels that way about their subject. Are the topics covered under this programs yielding more important results than from other programs? Who's to say? I think we need to make proposals that can be viable within the current financial constraints. If the work now they will likely work better if the situation improves.

    Alex, the EAGER mechanism is not peer-reviewed, but money decided on entirely by a PO. That is not even close to the R21. The RUI designation is not a mechanism at all, simply a classification of a regular proposal. And it is false that R15 money goes primarily to RUIs. I believe what I have put forward above does do things better, though I never claimed that NSF ignores these categories.

  • proflikesubstance says:

    Odyssey I was under the impression that the deadlines remained the same (Jan/Jul), but that they were extending the time to eliminate proposals being submitted twice in a year. This would be what I would advocate for. I'll have to go read the MCB RFP closer.

  • drugmonkey says:

    Joan- just don't delete quotidian, Internet typical comments, period. It makes you look like a humorless censoring monarchist. An extra bonus is that if someone really is being a jerk, they make the case against themselves far better than you can with either response or petulant deletion. And that is always entertaining.

  • What the fucke comments did the idiot delete???

  • Oh, I just went over there and looked. Bradny!!! FTMFW!!!

    BTW, I thought "sociobiology" was like some kind of bullshitte like phrenology? That shitte is a reall field now??

  • CoR says:

    Joan is not an idiot. I have reviewed work from her lab. It is awesome.

  • Who the fucke is "Joan"?

  • proflikesubstance says:

    Joan Strassmann is the author of the Strassmann blog. I don't know about her blogging history, but her science is good. She has admitted that she is new to dealing with comments and is looking for advice. If this is the biggest mistake she makes as a blogger, she'll be ahead of most of us.

  • drugmonkey says:

    There is a drive going on at NIH to drop the R21. Some of the ICs have already done so formally. And the R21 hate comes right from the head of OER with a couple of comments about higher paylines to discourage applicants.

    I think that's a disaster and what they should have done was fix the review of these allegedly Exploratory and/or Developmental proposals.

  • proflikesubstance says:

    Couldn't agree more, I think the R21 is an extremely useful mechanism if administered as originally intended. It would be really interesting to see the proportion of R21s that went on to serve as preliminary data for successful R01s.

  • drugmonkey says:

    Yes....except that can't be the simple analysis expecting all to turn into R01s. As I've opined before R21s that "fail" are actually a GoodThing.

  • Assoc Prof says:

    Prof-Like Substance, I just read over Joan Strassmann's blog and the comments that she restored. I think she initially did you a favor by deleting your comments.

  • proflikesubstance says:

    Yeah, AP, I would hate to be seen as someone who uses humor in a conversation when it gets absurd! Glad I have senior people around to save me from myself and keep me in my place.

  • Joe says:

    Why get rid of the R21? A lot of them I have seen propose work that might make up an R01 (or a decent chunk of one) but with a lot less fluff. Thus it seems a bargain for the NIH. The "no prelim data required" aspect is not handled as well as it should be - though you can't criticize an R21 proposal for lack of prelim data, you can still praise the ones that have nice prelim data, so it's still an important aspect. As with other proposals, the expectations for R21s have been ratcheting up, and they are very competitive.

  • [...] Strassmann's thoughts on preproposals at the IOS and DEB divisions and the resulting comments, and Prof-like Substance's thoughts on potential changes at the NSF. Lots of good fodder for discussion at both [...]

  • [...] Cath, over at VWXYNot, has a post up describing the changes being proposed by CIHR (the Canadian NIH) to revamp the review process there and alter their funding structure. These changes are being fueled by some of the same pressures that influenced NSF Bio to undergo recent changes. [...]

Leave a Reply