Archive for: August, 2012

Thoughts on the ESA letter to NSF Bio

From multiple fronts I have recently been made aware of an effort that started at the Ecology Society of America to formally protest the new NSF Bio regulations on grant submissions. For those not aware, the short version of the changes includes a move from two annual cycles to one, the institution of a preproposal stage and a limit of two grants per PI in any capacity.

These changes were met with resistance and the general feeling that early career people were going to take it in the teeth. Only the DEB and IOS programs in Bio initially made these changes, but MCB recently let it be known that they were going to do the same after a year utilizing a different approach.

The ESA letter focuses on the concern that the new policy is going to significantly slow the pace of science in the Bio Directorate, an opinion I have written in the past. I'll deal with each of the points of the letter separately:

The process creates an exceedingly long lag between the time when ideas are first proposed and when funding becomes available to investigators. Even if the two preproposals allowed per Investigator per year are successful, it takes over one year from submission to funding (as opposed to 6-9 months in the former system). This lag time increases to over two years or longer if a preproposal is unsuccessful. The increased lag period comes at a time when the rapid pace of environmental change requires science-based solutions to address societal needs. It also hinders the development and deployment of new tools and technologies (e.g., molecular and informatic) that inform solutions that address such rapid environmental change. The long lag between idea generation and funding is particularly hard on junior scientists who are establishing their research programs, but also hinders progress of more senior scientists seeking to sustain active research programs and to educate the next generation of scientists.

It's hard to argue with this one, which basically summarizes several points that have become familiar refrain. When I served on the preproposal panel last spring, these exact points came up. The NSF party line* is basically that the extra time now built into the process is critical for PIs to incorporate the feedback they received from the previous round. The claim is that internal studies demonstrated that proposals that were turned around from one round to the next did not fair as well as those that took a round off. Unfortunately, there is no NSF equivalent of Rock Talk to show these data, so we can't pick it apart. We're left wondering whether these data are a game of averages or represent real trends. Nevertheless, from the perspective of a PI, the loss of one round per year is perceived as a significant development, no matter how NSF wants to spin it.

The process limits the scope of science by (1) selecting against complex, interdisciplinary science that cannot be convincingly described in four pages and (2) hindering collaboration among scientists (by limiting the number of submissions per investigator per year) at a time when research programs and teams need to be increasingly multifaceted, innovative, and interdisciplinary to address complex issues.

This is where I'm a little less enthusiastic about the approach. The first argument about complexity that can't be described in four pages is, to me, a little weak. Is it hard to encapsulate a complex project in four pages? Yes. But claiming my science is SO complex you just don't get it because I can't tell you all about it is not exactly winning me over. Make it work.

The second point is stronger. For a decade plus we have been told that interdisciplinary is the way to get things done. Multi-PI projects have been encouraged at every level, from federal to institutional, and suddenly it is a liability. Now we have to pick and choose what we can contribute to in order to stay within an arbitrary limit of proposals. Okaaaaaay.

The process limits feedback to scientists, slowing the pace at which creative ideas advance during the iterative submission-resubmission process, because of the lack of ad hoc reviews for proposals. Although investigators faced low rates of proposal success with the former process, it at least offered comprehensive feedback and allowed for relatively quick resubmission, increasing the chances for success with future submissions.

This is another point that I don't see eye to eye with. If you have ever served on a panel, you know that the opinions that really matter are in the room. Whereas the ad hoc reviews can be very informative to the panelists, they can just as easily be almost ignored. The POs will ask the lead reviewer to comment on the ad hocs, especially if they are at odds with the panelist opinions, but they do not carry the weight of the panelist reviews. Additionally, my experience in the preproposal panel suggested that NSF can get pretty good coverage of the vast majority of the proposals with the panelists, not unlike NIH's study sections. Besides, the PI still gets at least three reviews back and if it didn't get trashed in those reviews, a panel summary as well. Are the additional couple of ad hoc reviews really that important?

The delays, the reduced opportunities for collaborative proposals, and the more limited feedback are likely to have a disproportionate effect on young scientists and members of groups who are not yet well represented in our science. We fear that this new process will result in the loss of some very promising people from the pipeline who are already discouraged by bleak prospects for funding research.

This is a concern shared by NSF. When the Director of Bio came to speak to our panel in the spring, this was something that he cited as being at the top of the list of things they are monitoring. I have to say that most panelists were not necessarily comforted that the situation was being monitored, but it was made clear that the POs were being directed to keep their portfolios balanced as before. How the preproposal process is going to affect that is not entirely clear.

One rumor I have heard repeatedly, however, is that only established labs made it through the preproposal process. I can say from my experience as both a panelist and a preproposal PI, this is not true. I can understand the perception that those with more proposal writing experience were able to navigate the new process better, but we pushed many early career preproposals through in my panel and I had one accepted from my own lab as the sole PI. I don't buy the fear mongering.

The letter concludes with the following:

We are optimistic that thoughtful modifications of the new preproposal process, made in consultation with the ecological and environmental sciences community, will ensure that science progresses as rapidly as possible given the level of funds available, thereby providing maximum benefit to society. In any such modifications, we believe it is essential (1) to ease current restrictions that limit collaboration and the pursuit of high-risk, high- reward ideas and (2) to provide two deadlines per year, even if that requires taking other measures, such as reducing the number of ad hoc reviews or reducing proposal length, to ensure reasonable workloads for NSF staff and the reviewer community.

Here's my biggest problem with this letter. Where is the solution? It is very easy to say that the current system is not working and we need to make changes, but don't place that burden right back in the lap of NSF if you want to make changes! Make a PROPOSAL. I've made some very clear suggestions geared towards solving these exact issues, but I have seen almost no other concrete proposals made. I think this is where we, as a community, are doing ourselves the most harm. We need to hammer out what we want and how to implement it if we want to gain any traction, otherwise NSF can throw up their collective hands and say "this is the best we can do."

So will I be signing this petition? No. As much as it summarizes some of the concerns I share, I don't think it has much value without suggestions for real change. I think NSF was fully aware they would get push back from the community based on these changes and this type of letter is going to fall right into the "expected whining" camp. Until the community can present a meaningful document with realistic changes, there is no impetuous for NSF to do anything.

I see this as a labor negotiation. NSF changed the rules of "payment" and has left their "payees" upset with the new environment. Unless we can make a convincing case (including some concessions**) for change, we're going to have to live by the new rules.

*Make no mistake, there are wildly different opinions on the changes among the Program Officers.

**Sorry folks, but NSF believed strongly that the system was nonsustainable. If you want to convince them to change you need to make concessions that address the issues.

16 responses so far

Open thread on preproposal grumblings

Aug 23 2012 Published by under Uncategorized

I'm working on a post discussing the letter that is being circulated for signatures about the NSF preproposal process, but I'm in the field this week so it'll take a day or two. The letter appears to have sprung out of the recent Ecology Society of America meeting and makes some valid points. In the mean time, now that we have gone through the preproposals and the full submission deadline, I'll leave this thread open for people's thoughts on the new NSF Bio proposal process. What worked? What didn't? Let it all out.

5 responses so far

Becoming a grandparent apparently makes you lose you damn mind

Aug 16 2012 Published by under [Et Al], [Life Trajectories]

Parenting is an odd thing. You spend a lot of your time trying to do your best, surviving when your best isn't good enough and generally trying not to fuck your kids up. There's millions of books on the topic if you care to read them, but in the end you default to emulating what you think are the best parts of what you know from growing up, fused with what you think are improvements to your upbringing. It's wonderful, infuriating, thrilling and miserable all at the same time. Above all, it gives you a new respect for your parents.

Armed with this new respect, you assume that you can hand your kids off to your parents and, with their honed parental instincts that produced the amazing person that you are, they will deliver your child back unto you in at least the same condition they were when they departed.

But that's not what happens. At all.

What's they very first thing your parents do when they get ahold of your children? Let the children decide everything. It's like those years between their own last child leaving the house and the arrival of grandchildren has wiped every ounce of parenting common sense out of their system. Suddenly the inmates are running the asylum.

My mother is very into health. She's been eating organic foods since before you could find them in regular stores. I can recall the week she decided that she was getting rid of all the non-organic stuff in the house, which has remained that way ever since. Based on this, one would assume that she would feed her grandchildren in a similar way that she fed her own family for decades.

Instead, my mother has introduced the following items to my kids:

McDonald's (a place she hadn't been in >20 years)
gum
soda
hot dogs
Chocolate ice cream for breakfast
Cheetos (quite possibly the antithesis of her diet)
And probably a dozen more things I haven't heard about

It's not that I mind that the kids eat these things - it is only that we don't usually eat these things ourselves that prevents the kids from being served them - but the fact that my mother was the vehicle of delivery for these foods absolutely floors me. I'm willing to bet that she had to look up where the nearest McDonald's to their house was.

And that's just the tip of the iceberg. When I mentioned to my father on the phone some of the issues we were having with the Wee One recently, he said "Hmm, she doesn't push against the rules at our house." It wasn't until I pointed out that you can't push against something that isn't there, that he laughed and agreed. Bed time? whatevs! Naps? Who needs them? Don't want to brush your teeth? Those teeth are gonna fall out anyway!

I get it, it's not fun to enforce rules with kids and grandparents want to spend all of their grandkid time having fun. They want the grandkids to beg to go back and in the end they don't have to deal with the repercussions of utterly exhausted children returned home. They can drive away just as their parents did after dropping my exhausted sibling and I off after a few days of sugar fueled madness.

And I'm sure they chuckle all the way home.

29 responses so far

Repost: The grad student totem pole

Aug 14 2012 Published by under [Education&Careers]

It's that time of year again. Out with the old, in with the new. From 2010.

There are certain things about grad school that you kinda forget about once you move on. It's been roughly 6 years since left my grad lab and a lot has happened since then. With the new grad students filtering in around now, I was reminded of one absolute: new students get what's left.

For whatever reason, there is always a known ranking of resources in a grad office. A best desk space, best chair, best coffee cup, best whatever, and every other similar item fits into the hierarchy.  Grad student B has the third best chair, for instance. At times of personnel flux, the perturbation sets off a wave of resource shuffle.

Senior grad students get first choice and will scarf up anything that is higher in the ranking than they already have. The next in line than devour their refuse and so the grad student food chain re-arranges itself with the n00bs getting the scraps.

This, of course, is unbeknownst to the noob, because they are just happy to be starting something new and everything takes some adjusting to. Research, all the reading, the other students, a new place, new surroundings. While they find their bearings they don't notice that their chair has one arm duct taped on and their mouse only scrolls in one direction. The other students, of course, assure them that the chair is the most comfortable one and the loose arm staves off carpel-tunnel. Besides, who ever scrolls up in a document or webpage? All the good stuff is down!

Before long, the noob realizes the deal and starts counting down to the next resource shuffle.

2 responses so far

Year 4 in review: getting warmer

Aug 09 2012 Published by under [Education&Careers]

Hard to believe it, but I'm coming up on the end of year 4 here. Years one and two were full of the typical bumps and bruises that any new lab goes through, but they were really the honeymoon phase. Year three was the year that things started to come apart a little bit and I was really starting to get concerned about how things were going. While I alluded to it in this space, that was really not a good time. I should say that I appreciate this community for keeping me motivated. Despite how things looked early on, the year did improve and we got some decent funding rolling. We're now a year into that proposal and it's going well.

Since then a few things have happened. We added the Weer One to the family, which has changed a great number of things. I've also taken steps to make sure there is no Weest One. Not sure I could handle the sleep deprivation again.

We've seen NSF make some major changes in the way they review proposals and I went to DC to see how this would work, first hand. After much debate over this process, I offered up a proposal for fixing some of the issues that led to the preproposal system in the first place. No calls from DC yet.

I've taken some additional professional steps, including becoming an AE for a journal in my field and becoming far more comfortable in my role as a teacher (but not with grade inflation). There has been considerable discussion of work life "balance" and semicolons.

Overall, year four was a good year. We have two proposals pending with different agencies, a few lab alumni, some papers out and a bunch about to go, and a LOT of data. I am far from ready for the semester to begin, but when it does, we'll be starting in good shape. The lab is beginning to resemble what I had pictured when I started out. I just didn't know it was going to take this long to get here.

8 responses so far

Resident alien

Aug 07 2012 Published by under [Education&Careers], [Et Al]

Like most people in academia, I have moved around a decent amount. I've lived in multiple states and countries throughout my career, and may someday move again. The semi-nomadic lifestyle has some perks, particularly as it relates to gaining experience with different cultures and norms. I have made friends in far reaching places, especially as they spread to new places from where we met. My eldest daughter has two passports and I hope to bring her back to where she was born in the next couple of years. I see these as good things that have helped me develop my sense of values and perspective.

There are things, however, that rolling stones miss out on. I am once again teaching teachers for a few days this week and many of them have lived entire lives within a few mile radius of where they grew up. I've found myself lost in conversations about places and things (some still present, some long gone) that are familiar to them all. Nearly every one of them knows somebody who is a friend of someone else in the room.

The level of connection is astounding to someone who has spent much of their life disconnected from the permanent residents based on my transience. A few years here, a couple more there, a handful over yonder.

In about a week I will have been here for 4 years. That's roughly as long as the previous 3 places I have lived. This time is different, because there is potential that I will stay here for a considerable length of time. However, it still feels the same at this point and I wonder whether I'll settle into to my current environment or always remain a foreigner. Time will tell.

10 responses so far