From multiple fronts I have recently been made aware of an effort that started at the Ecology Society of America to formally protest the new NSF Bio regulations on grant submissions. For those not aware, the short version of the changes includes a move from two annual cycles to one, the institution of a preproposal stage and a limit of two grants per PI in any capacity.
These changes were met with resistance and the general feeling that early career people were going to take it in the teeth. Only the DEB and IOS programs in Bio initially made these changes, but MCB recently let it be known that they were going to do the same after a year utilizing a different approach.
The ESA letter focuses on the concern that the new policy is going to significantly slow the pace of science in the Bio Directorate, an opinion I have written in the past. I'll deal with each of the points of the letter separately:
The process creates an exceedingly long lag between the time when ideas are first proposed and when funding becomes available to investigators. Even if the two preproposals allowed per Investigator per year are successful, it takes over one year from submission to funding (as opposed to 6-9 months in the former system). This lag time increases to over two years or longer if a preproposal is unsuccessful. The increased lag period comes at a time when the rapid pace of environmental change requires science-based solutions to address societal needs. It also hinders the development and deployment of new tools and technologies (e.g., molecular and informatic) that inform solutions that address such rapid environmental change. The long lag between idea generation and funding is particularly hard on junior scientists who are establishing their research programs, but also hinders progress of more senior scientists seeking to sustain active research programs and to educate the next generation of scientists.
It's hard to argue with this one, which basically summarizes several points that have become familiar refrain. When I served on the preproposal panel last spring, these exact points came up. The NSF party line* is basically that the extra time now built into the process is critical for PIs to incorporate the feedback they received from the previous round. The claim is that internal studies demonstrated that proposals that were turned around from one round to the next did not fair as well as those that took a round off. Unfortunately, there is no NSF equivalent of Rock Talk to show these data, so we can't pick it apart. We're left wondering whether these data are a game of averages or represent real trends. Nevertheless, from the perspective of a PI, the loss of one round per year is perceived as a significant development, no matter how NSF wants to spin it.
The process limits the scope of science by (1) selecting against complex, interdisciplinary science that cannot be convincingly described in four pages and (2) hindering collaboration among scientists (by limiting the number of submissions per investigator per year) at a time when research programs and teams need to be increasingly multifaceted, innovative, and interdisciplinary to address complex issues.
This is where I'm a little less enthusiastic about the approach. The first argument about complexity that can't be described in four pages is, to me, a little weak. Is it hard to encapsulate a complex project in four pages? Yes. But claiming my science is SO complex you just don't get it because I can't tell you all about it is not exactly winning me over. Make it work.
The second point is stronger. For a decade plus we have been told that interdisciplinary is the way to get things done. Multi-PI projects have been encouraged at every level, from federal to institutional, and suddenly it is a liability. Now we have to pick and choose what we can contribute to in order to stay within an arbitrary limit of proposals. Okaaaaaay.
The process limits feedback to scientists, slowing the pace at which creative ideas advance during the iterative submission-resubmission process, because of the lack of ad hoc reviews for proposals. Although investigators faced low rates of proposal success with the former process, it at least offered comprehensive feedback and allowed for relatively quick resubmission, increasing the chances for success with future submissions.
This is another point that I don't see eye to eye with. If you have ever served on a panel, you know that the opinions that really matter are in the room. Whereas the ad hoc reviews can be very informative to the panelists, they can just as easily be almost ignored. The POs will ask the lead reviewer to comment on the ad hocs, especially if they are at odds with the panelist opinions, but they do not carry the weight of the panelist reviews. Additionally, my experience in the preproposal panel suggested that NSF can get pretty good coverage of the vast majority of the proposals with the panelists, not unlike NIH's study sections. Besides, the PI still gets at least three reviews back and if it didn't get trashed in those reviews, a panel summary as well. Are the additional couple of ad hoc reviews really that important?
The delays, the reduced opportunities for collaborative proposals, and the more limited feedback are likely to have a disproportionate effect on young scientists and members of groups who are not yet well represented in our science. We fear that this new process will result in the loss of some very promising people from the pipeline who are already discouraged by bleak prospects for funding research.
This is a concern shared by NSF. When the Director of Bio came to speak to our panel in the spring, this was something that he cited as being at the top of the list of things they are monitoring. I have to say that most panelists were not necessarily comforted that the situation was being monitored, but it was made clear that the POs were being directed to keep their portfolios balanced as before. How the preproposal process is going to affect that is not entirely clear.
One rumor I have heard repeatedly, however, is that only established labs made it through the preproposal process. I can say from my experience as both a panelist and a preproposal PI, this is not true. I can understand the perception that those with more proposal writing experience were able to navigate the new process better, but we pushed many early career preproposals through in my panel and I had one accepted from my own lab as the sole PI. I don't buy the fear mongering.
The letter concludes with the following:
We are optimistic that thoughtful modifications of the new preproposal process, made in consultation with the ecological and environmental sciences community, will ensure that science progresses as rapidly as possible given the level of funds available, thereby providing maximum benefit to society. In any such modifications, we believe it is essential (1) to ease current restrictions that limit collaboration and the pursuit of high-risk, high- reward ideas and (2) to provide two deadlines per year, even if that requires taking other measures, such as reducing the number of ad hoc reviews or reducing proposal length, to ensure reasonable workloads for NSF staff and the reviewer community.
Here's my biggest problem with this letter. Where is the solution? It is very easy to say that the current system is not working and we need to make changes, but don't place that burden right back in the lap of NSF if you want to make changes! Make a PROPOSAL. I've made some very clear suggestions geared towards solving these exact issues, but I have seen almost no other concrete proposals made. I think this is where we, as a community, are doing ourselves the most harm. We need to hammer out what we want and how to implement it if we want to gain any traction, otherwise NSF can throw up their collective hands and say "this is the best we can do."
So will I be signing this petition? No. As much as it summarizes some of the concerns I share, I don't think it has much value without suggestions for real change. I think NSF was fully aware they would get push back from the community based on these changes and this type of letter is going to fall right into the "expected whining" camp. Until the community can present a meaningful document with realistic changes, there is no impetuous for NSF to do anything.
I see this as a labor negotiation. NSF changed the rules of "payment" and has left their "payees" upset with the new environment. Unless we can make a convincing case (including some concessions**) for change, we're going to have to live by the new rules.
*Make no mistake, there are wildly different opinions on the changes among the Program Officers.
**Sorry folks, but NSF believed strongly that the system was nonsustainable. If you want to convince them to change you need to make concessions that address the issues.