Today, guest blogger and NSF PO, Michelle Elekonich, writes about what is going on at NSF between and after panels. What has to happen for your proposal to be picked up? Where does the money come from and how are panels put together. If you missed them, Part 1 and Part 2 of the series were posted earlier. With this last installment posted, I want to thank Michelle for taking the time to write this, get it approved by NSF and answer questions that have come up.
Since I arrived before the preliminary proposal system was instituted you might wonder what has changed at NSF with the new preliminary proposals? When I arrived I asked my program’s staff to graph the proposal load over the last 5 years- it had almost doubled in that time. A good chunk (about 20% I would learn) came back each time from the previous round, often with few changes as there had probably been less than a month since the prior version was declined. One of the first things I had to do when I arrived was build a panel and get outside reviews for the proposals that had just come in. I quickly learned how difficult it was to get outside reviewers and got new appreciation for program directors and editors of journals. My own record was 13 requests for 2 outside reviews- my co-program director’s record was 17! Mostly I ended up asking 6-8 people to get 2 reviews. With 120 proposals that was a lot of requests between us for that first panel and it only grew from there….it was quickly clear that this was problematic and it ended up being one of the major drivers behind the new process.
That was the first of many panels I would put together-it is really an art form. There are a lot of requirements for the composition and the division director has to approve the final panel list. Panels must have some new people but not too many; some people from primarily undergraduate institutions (Yes! NSF wants you!) and some from state schools and some from the big research universities; in IOS we are not allowed to have more than two from the same state without getting special permission from the division director; we must balance the number of men and women and include members of underrepresented groups and of course most importantly panelists have to have the expertise to cover the proposals submitted. Phew! Flexibility to change relative to submissions and the ability to have more people participate over time are two of the reasons IOS doesn’t use standing panels.
Now, with the preliminary proposal system program directors are asking for fewer reviews. But the workload at NSF hasn’t really decreased overall, and program directors and the administrative staff are working at top capacity throughout the spring. But, the new review system has provided a couple months in the summer for analyzing the funding trends, doing outreach and going to one’s lab for a longer period. It is just as much work, but feels saner, because now there is a break where there was none before. The first year and a half I was here we would just finish one round, be having phone consultations with declined PIs and (patiently) answering questions from people getting ready to submit all at the same time and then start all over again when the next batch of proposals arrived. If there was any break at all it was just a couple days.
One thing that has not changed and that surprised me when I came to NSF is the amount of work the program directors do after the panel. In preparation to make the invitations or the new recommendation decisions program directors have to analyze the distribution of the current awards relative to topics, states, broadening participation and other division and directorate priorities. This is what program directors are referring to when they talk about portfolio balance.
Most often there is far more good science than the money available can fund, so hard decisions have to be made. Even in high priority not every proposal may be recommended. This is where the portfolio analysis comes in- the NSF mission is to move science forward so we don’t necessarily want to keep funding the same things that we already have funded and of course the panel doesn’t know what is in the entire portfolio. So the program directors evaluate the science relative to the portfolio, and the panel’s input about strengths and weaknesses plus any knowledge of their own to come up with a potential list to recommend. Within some clusters there is one program director for each program or for a subset of the program and they make their own recommendation decisions. For example, in IOS, the Processeses, Structures and Integrity Program has a plant panel and an animal panel and each has its own program director who has a specific part of the money to use. Other programs, like Animal Behavior, have multiple program directors working from one program budget so they all have to agree on what to recommend. A lot of intense discussion goes on. Once they have a list it gets presented to the division director, deputy division director and in IOS the science advisor at a meeting a few weeks after the panel. Only then can the program directors start calling people.
With the new preliminary proposal system, in the spring the invites are processed first in IOS so that PIs will have as much time as possible to write the full proposal. In the fall, the declines are processed first so PIs have time to revise for the next preliminary proposal deadline. Because the panels are spread across a two month period, the post panel meetings are spread out and some clusters finish, write their analyses and get division director approval sooner and thus contact their PIs before others. The shorter preliminary proposals gave the reviewers less to read but did not really change the NSF process – in fact the larger number of them made it just as much work here. The only real work savings was in the fall full proposal panels by having to make fewer ad hoc review requests. But, the new process let us work smarter- we developed tools to automate some parts of the processing of the streamlined preliminary proposals so we could handle the larger numbers, by knowing what would come in as full proposals we could build panels sooner so people were less booked up when contacted, and the smaller number of full proposals let us put more energy into getting better reviews for each one and this helped to maintain the quality of the review.
Another aspect of the post panel processing involves writing a review analysis that documents the program director’s assessment of the review and reasons he/she made the decision. Each proposal submitted through Fastlane (no matter what type – preliminary proposal, full proposal, supplement, EAGER etc -or what the decision is) has to have one. Once this is done, all the documents from the panel and the review analysis go into the electronic file and it is checked by the staff to make sure nothing is missing and then each decision is approved by the division director with the help of the deputy division director and science advisor.
Throughout all this, the program directors are also looking for co-funding to stretch their program dollars. If a program director can get money to fund part of one proposal then it means he/she could fund an additional proposal or perhaps cut less money from some other proposals. I knew that there was some negotiating for co-funding that went on before I came to NSF, but I did not know that there are actually a lot of different programs at NSF that a program director could get co-funding from and that it can take months! For example, the deadline for applications for EPSCoR or other co-funding might be a couple months after the panel and it can take another month to hear back. Generally these NSF wide programs require a written document to apply for the co-funding. I find it ironic to have to write a mini-proposal to get co-funding for a proposal! Other times program directors from two different programs will decide to share the funding (often as the result of a co-review) and that is faster and easier.
Even when all is said and done in IOS, awards still have to be processed through the Division of Grants and Agreements (DGA)- while program directors can recommend, the folks in DGA are the ones that actually give the money and create the contract with your university. They make sure that all the federal rules for using the money are followed by the universities and that the contracts are complete. All in all, it’s a pretty complex process.