Your DEB panel summary will look a bit different this year

(by proflikesubstance) Nov 04 2015

Tis the time of year for full proposal panels in NSF's IOS and DEB and there's a few new twists for those in DEB, including the panel summary. If you were fortunate enough to have a full proposal in panel this round, you may be slightly surprised at the feedback you get. There's a longer explanation at the DEB blog, but I was initially surprised by the new template, which looks like this:


There's a fair bit of information here that was previously not really included in the panel summary, which focused almost exclusively on the strengths and weaknesses of the IM and BI sections. So what do that mean to you as a grant writer? Two things in particular:

1) There is now a heavier emphasis on evaluating prior support. Were you productive? Did you actually do more than training for your broader impacts? These will now be specifically commented on. Make sure you choose to highlight the recent grant (you only need to write about ONE) where you can flex your publication and BI muscles a bit.

2) The data management plan is becoming more of a focus. Don't be surprised if this ends up as a field in the reviewing score sheet soon. Those who have been mailing it in for this section (and there are plenty) are going to have increasing difficulty convincing reviewers they can deal with the project data. I don't think the postdoc mentoring plan is far off from showing up here either, BTW.

Whereas these new sections of focus don't have the same weight as the others, DEB is certainly shining a light on sections that were often glossed over before. Be prepared if you get a shot at a full proposal next year, because you don't want to waste those opportunities.

No responses yet

What you learn from reviewing a batch of proposals

(by proflikesubstance) Oct 06 2015

It is full proposals review season for NSF's IOS and DEB panels. Theoretically, what we are reviewing is the cream skimmed from the top of the preproposal pile (+ CAREER proposals). The reality, of course, is that we have the resulting proposals from those who were convincing at the preproposal stage. Subtle, but very different.

One thing that has really jumped out at me this year is the number of proposals that are clearly stretching to make the 15 pages. There's lots of tricks to do this, some more obvious than others, but it's still clear if you add a superfluous table that spans two pages or use five pages for your Broader Impacts that could be summarized in two. I don't remember seeing that in previous full proposal panels, but perhaps I was less attuned to it before. Indeed, some people this round aren't even bothering to fill the full 15 pages, ending their proposals a page or more early. My sample size is too small to know whether this is A Thing, or if I'm just getting an unusual number of these proposals.

I honestly also don't know whether this is a consequence of the preproposals stage or not. I mentioned this on twitter and got different reactions, but the general feeling was there was an influence of how we select preproposals.

As we've discussed before, pre- and full proposals are different documents with different goals. It is entirely possible to sell an exciting preproposal that doesn't hold water as a full proposal. I don't think that's in inherent flaw, just a new feature of the system.

But I digress.

Another obvious issue with some proposals is their lack of balance between writing for the ad hocs and the panelists. What's the difference? The ad hoc reviewers will be people in your field who know the ins and outs of the system. They will be the ones to spot a flaw that doesn't take into account some recent literature in your field. They will be the ones questioning part of your specific methodology. The panelists will likely be evaluating the goals of the project at a different level. Beyond the minutiae of your system, panelists will be determining whether or not your over-arching questions appeal broadly. Is there more than just answering a subfield question here?

Unlike the preproposals that do not get reviewed outside of the panel, every full proposal walks the tightrope between these two audiences, attempting to please them both. The key to doing this is structuring the proposal so that the broadest questions and approaches are front and center, with the gritty details towards the end and well labeled. Mixing and matching the two works for some, but often makes a mess in less experienced hands. When reading a number of proposals at once, this is very clear.

As a panelist I often struggle with how much grant writing advice I should include in my reviews. I mean, that's not technically the job, but that feedback might be just as important as the scientific feedback. Not everyone out there has mentors willing or able to provide critical feedback. Unlike NIH, where the review is written specifically to the study section, NSF reviews are written more with the applicant in mind. I admit that I've found more of this type of critique slipping into my reviews than it once did. I'll be curious to see how that is handled at panel.

Of course, the biggest trick of all is ensuring one takes one's own advice when it comes time to write your own proposal.

8 responses so far

Arming professors: A terrible, no good, very bad idea

(by proflikesubstance) Oct 02 2015

Deterrence. This argument often surfaces in the aftermath of school shootings, which have become almost commonplace in the US. If we simply armed teachers, it would keep would-be school shooters from carrying out their plans. It's an opinion that is gaining traction nationally. I even had this conversation yesterday:

You can follow the thread, but there's several major flaws in this argument. First off, the data just flat out don't support that claim. Even the casual observation that the US has by far the most guns in the hands of citizens, yet the highest national gun violence stats, should tell you all you need to know. There is NO QUESTION that more guns = more gun violence.

But but but, we just need the right people to have the guns! Well, this is also a favorite NRA argument. Spoiler: it doesn't hold weight, either. Check out this 20/20 episode where they trained students well beyond the normal training for concealed carry, told them they would have to fend off a shooter and not one managed to pull their weapon when shot got real and they all got shot.

Now we want to argue for putting guns in the hands of professors. Okaaaaaay, how's that gonna go? Well, to start with, the vast majority of profs I know would refuse. Why? there's probably an infinite number of reasons, but they would start with the fact that even trained professionals often make mistakes in live shooter situations and kill innocent people. Now you want Dr. Smith to seamlessly transition from their economics lecture to gunning down some dude who bursts into a classroom with an assault rifle? Yeah. Ok. How many of the professors you have met would you trust to react to a live shooter incident, fire a weapon accurately at a distance and not accidentally hit an innocent student in the mayhem?

Even if we only secretly hire new professors with Navy Seal training, what would be the effect of having professors toting guns to class? You think your professor is unapproachable now? I'm sure a loaded .38 will help. "He seems much more nurturing now that I know he could gun me down at any moment!" The reality of any open carry movement is that the primary use of visible guns is intimidation. Any professor that would volunteer to carry a gun to class is almost certainly not one I would want to carry a gun to class.

Now, would armed professors make any difference? Doubtful. If someone wants to shoot up a school, is the presence armed professors going to make them change their plans? "I was going to go on a rampage with my assault rifle, but the thought of Dr. Ratcliff's .22 made me reconsider and channel my efforts to community service!" Yeah, no. Mentally stable, rational people don't gun down innocent people on a whim. Assuming that they are taking risk into consideration is absurd when almost every school shooting has ended with the death of the shooter. They know they are not making it out alive, so the idea of armed professors would mean nothing.

Knee jerk reaction aside, there is no evidence at all that would suggest that arming professors would curb the violence we have seen. More guns are not the solution, no matter what the NRA has conditioned us to believe. We need gun laws that actually make sense (see: almost every other first world nation) and the citizenry to actually give two fucks about curbing gun violence.

Or maybe we just charge $5k for bullets:

One response so far

Bio Jobs Wiki

(by proflikesubstance) Aug 20 2015

As a follow on to the conversation from yesterday, I wanted to post the Bio jobs wiki here so that people could keep track of the progress of various searches. This wiki kept me semi-sane during my search.

It's just getting rolling for the year, so there aren't many comments yet, but it is a way to keep up to date on when short lists have been formed and when interviews are being scheduled. In years past people also included start-up numbers. Worth a bookmark if you're on the market.

2 responses so far

WTH hiring committees?

(by proflikesubstance) Aug 19 2015

Interviewing people for tenure track positions is hard work. There's a lot of time spent weeding through applications, checking references (for the short list, dammit!) and coordinating visits. During the interviews it's even more work, with considerable time spent both with the candidates and discussing them after the fact. But no matter how exhausted the committee is by the process, this is criminal:

The list of on campus interviews is usually about 4 candidates. Presumably the committee ends up hiring one of those candidates, leaving three flapping in the wind. Three people, who made your list after an intense winnowing process and two day interviews.... and you never contact them again?

That's just not how you treat people. Generally. But in this specific case, the committee is doing an even greater disservice by putting a very bad taste in the mouth of someone who might end up being a colleague! If they were good enough for you search, chances are they have other interviews. If they get hired somewhere else, do you think they will have anything good to say about your dept/university or committee members, personally?

There is just NO REASON for hiring committees to be so cavalier with candidates and treat them like they are just another folder in the pile. It's dehumanizing and just further pushes the narrative of the cold elitist ivory tower. If you can't remember what it was like to go through the interview process as a postdoc and wait those excruciating weeks as they dragged on, sometimes into months, then you need to at least try to put yourself in those shoes.

Treat people like you would want to be treated in their situation. Don't be an asshole.

29 responses so far

Your online advice is rubbish if I can't read your CV!!!

(by proflikesubstance) Jul 29 2015

As we have covered before (Here and Here) there are many reasons why people blog under a name that can't be directly tied to their professional life. You can check the links if you want my extended thoughts on that and links to posts elsewhere that are more eloquent, but it's why the following exchange bothers me:

Academia is a very hierarchical enterprise. Trainees slot in at different, defined, levels. PIs have defined structure to advancement. There's university rankings, journal rankings, h-index, and on and on. We really love to define people by a variety of metrics and context.

All of these pieces of information provide a matrix where we, consciously or unconsciously, can form an opinion on the authority or each individual. As reviewers of proposals and manuscripts and as humans interacting with, and through, the literature or at meetings - whether you are "known" caries significant weight.

And that's where social media goes and screws it all up.

Giving people the option to post comments and interact without those all important identifiers takes the context that many academics rely on IRL, away. And that has opened the door to provide a voice to those who might not be heard 10 or 20 years ago. Ironically, this gets to the point:

Though this was said tongue-in-cheek, the reality is that what is a social norm for some is another's shut door. Having people entrenched in the upper tier culture view voices they can't place in their social context as not worthy of attention only re-enforces the echo chamber and excludes the same people who have been excluded for decades. I'm not here to chastise anyone, but I think it's important that we recognize that this is exactly the attitude that has gotten us to the point of being a largely white and male dominated profession. In the vast majority of cases, good science flows more from having the opportunity ($$$) to do science, rather than the individual brilliance of those doing it. Recognition and opportunity are inextricably tied together, and systematic exclusion (conscious or unconscious) has consequences.

But advice abounds everywhere, good bad and otherwise. To pretend like there is a universal correlation between the source and the value of the advice is ridiculous. I have gotten both good and bad advice from people I respect. Same goes for those whom I would not normally seek out. If you want examples of well-established PIs doling out terrible advice, just pick up Science Careers at some point and flip through. Or maybe you have a senior colleague in your department who got a job out of grad school 30 years ago and has renewed the same R01 for all that time. I'm sure he's got valuable advice for the grant scene these days.

We all get loads of advice throughout our careers, and whether the source is someone you know or someone writing under a pseud online, you need to evaluate it and decide whether it works for you. No one is out there faking their status to try and feed out bad advice, but your situation might not suit what they have to sell. We do this all the time IRL, so making the arbitrary distinction online is something that is curious, at best.

29 responses so far

My love/hate relationship with NSF preproposals

(by proflikesubstance) Jul 22 2015

I have no idea how many posts I've dedicated to NSF BIO's preproposals and I'm too lazy to check, TBH. However, they have significant impact on my community and many of the scientists I know and work with. They are the Gate Keepers to getting a shot at that golden ring. Successfully navigating this stage of the process is critical to having the chance to even apply for money.

As a reviewer I generally like preproposals. I feel like I can get a decent sense of what the PI(s) are trying to get at and I can decide whether I believe they can make it happen, for the most part. I like reviewing the compact format - I often find I am interrupted too much when reviewing full proposals if I don't retreat to solitude. I can get through preproposals in between obligations. However, you fill in a lot of gaps as a reviewer in the preproposal format. It's also a different type of writing with a different target audience and I think it lends itself a bit more to over-promising with the intention of generating that last bit of data before you really have to prove it in the full proposal. The fact that the membership on the pre- vs full panel does not completely overlap has always left me uncomfortable for exactly this reason.

As an applicant to NSF BIO I'm concerned that preproposals have significantly increased the number of applications both DEB and IOS, mostly from new applicants. There's two ways to look at this: 1) By lowering the activation energy, NSF is now able to capture a broader diversity of science that was not previously being proposed. 2) Alternatively, there's more noise in the system as people throw applications in so that they can report them in their annual review. Unfortunately, this is A Thing in some places and there are universities that even pay PIs per submitted application. Reality is likely somewhere in between, but the competition at the preproposal stage is fierce and not getting better.

As a preproposal writer, the format creates problems. Foremost, the limit of 2 proposals per division means that I often have to choose between a promising collaboration and a core project my lab is invested in. Considering the short turn around of NSF grants, it seems like I've always got a renewal and a new core project going in. One strategy is to, say, re-pitch a DEB proposal for IOS, but that never works out as well as it seems it might. Whereas the short format makes it easy to propose new ideas, the limits on numbers kills a lot of potential collaborations. Obviously there has to be some way to control the number of applications that are now easier to package up and send off, but damn, I wish I could fit in some of these cool side projects with collaborators.

Unfortunately, none of this is getting better without more money in the system. However, I would honestly prefer a compromise where we funded shorter proposals (say, 8 pages), did away with preproposals and limited people to 3 total aps during a year with 2 cycles. I fully recognize that this solution works best in my world, whereas someone at a PUI might think this is a terrible change. I don't know, but I can't help but think there's a better solution that splits the difference between the old and new system. And yes, I realize I just recently wrote about the futility of making tweaks to the proposal system.

5 responses so far

Tilting the NSF odds in your favor

(by proflikesubstance) Jun 23 2015

A lot of advice gets thrown about the interwebs about being a scientist in our current (North American) climate. One of the biggest topics is naturally, funding. How many applications? To which agencies? To which review groups? All of these are good topics and we've touched on them over the years.

We just landed our 3rd major NSF award in my 7 years on the job, and based on the DEB stats released for the last 9 years, that's something I'm proud of. It doesn't make me a good scientist or good PI, but it suggests my proposal strategy has been decently effective.

My strategy is probably only going to work in a place where research is a primary tenure requirement and teaching loads aren't overwhelming. When people ask me how to attack NSF funding, this is what I tell them:

1) Develop a stable of proposals. This isn't something you can do in the first couple of years, but by year 3 or 4 you should have a few major proposals that you are actively working. Some will fall by the wayside as new projects come up, but keep a few in the air at all times. The more proposals you write the faster they come together, as you should be able to share text among some of them.

2) You'll need to make a leap of faith more than once. If you're gonna float all those proposals, you'll need the "preliminary" data (i.e. half the project) to make them viable. Find small pots of money to generate those data (yes, that means more proposals). Concentrate your start-up on the core central needs, but finance other projects as well via internal or state grants. Some will never be fully funded, so this gets tricky with grad student effort, so have fall back plans.

3) Stay the course if you have conviction. Listen, some projects are going to get roughed up. Even if they are good science, you may not be communicating it the way a review panel hears it well. For those projects you know are going to score, don't drop them after a bad review or two. Read the reviews and read them again. Have others suggest how they might interpret them. Talk to your PO. Understand where you're going wrong, as best you can.

4) Have a short memory when it comes to rejects. Learn from them, but don't dwell on them. Fix the problem and try again.

5) Don't miss deadlines. Yeah, that's a shitty part, but there really isn't a time when you can sit back. I have never had a proposal get funded the first time it is reviewed. Considering both DEB and IOS are down to a single deadline a year, that means you need to be testing proposals years before you're looking at a funding gap. Do not let current funding get you comfortable. If the POs don't want to fund you because you currently are in good shape, let them turn you away this year and get that solid proposal back in next year.

6) Diversify. This is always a point of contention, but I am not a person who is going to flog a single system all my days. The advantage of this is being able to apply to a variety of funding sources, both within NSF and elsewhere. Sometimes it means I end up in too many collaborations and I need to pump the breaks on certain things. It's okay, everything is fluid. I haven't made any enemies yet by saying "I'm overcommitted right now and can't contribute what I originally thought I could." But spacing out deadlines across the year makes things more manageable and less desperate in January.

7) Serve on panels. It's a lot of work and often at bad times of the year, but panel service is SO key to understanding the review process The down side of submitting a lot of proposals is that you are excluded from a lot of panels, but get your service in.

I'm sure others will bring up suggestions I forgot for whatever reason, but the moral of the story is that proposal writing is a full time job right now. There's no free lunch and there's no gaming the system. There are many successful strategies and this is the one I've chosen. It may not work for everyone, but there's no one out there having an easy go of NSF funding. So get writing.

11 responses so far

For the Canadians suffering through the CIHR mess

(by proflikesubstance) Jun 09 2015


No responses yet

How do we make NSF science more sustainable?

(by proflikesubstance) Jun 05 2015

Yesterday I talked about the futility of making nano-scale adjustments to the process of review and funding at NSF. The take home is basically no system is perfect, but NSF is the best we've got right now.

The real issue is growth can't occur unless the system is fed, and Congress is starving science. For the enterprise to remain viable in the coming years, there needs to be money put into the system. However, the federal government being what it is, I think NSF needs to be thinking about how to change how we do science in the US so that we will be better buffered against congressional whims going forward.

My thoughts on how we do that aren't new. Many of these topics have been batted around in different blogs over the years, but some are unique to NSF and others aren't. These proposed solutions ALL require that congress view science as worth funding so that we can move forward and not continue to regress.

1) Extend the average time on proposals from three years to four. At least. Three years seems like along time only at the time of award. The second that clock starts ticking, that's no longer the case. Also, 4 years is a reasonable time frame to fund a student, whereas 3 years requires overlap with another grant (and that's unlikely) or institutional support, which may or may not be possible. Three years puts students and PIs in more of a bind and forces additional grant churn. This will cause overall awards to be more expensive, but the benefits are worth it.

2) Focus more on staff and less on students. I imagine this will be controversial, but I think we over-produce PhDs in some fields. Yes, the data say having a PhD is better for your job prospects than not, but it also says that the opportunity cost of entering the workforce much latter than peers is significant. Most current budgets can barely fit a student and one staff member (postdoc or tech). A cheaper option is to do two students. If NSF said it will only support one graduate student per grant and more critically evaluate whether some projects need a technician or a postdoc, I think we would see a growth in technical staff as a career path. This will likely also require some long-term commitment from universities to cover gaps for staff, but we need to shift to a model where there is less reliance on transient cheap labor and more on full time staff. Longer grants will also aid this.

Neither one of these changes fundamentally shifts science in the US, but neither is going to happen on it's own. The focus here is shifting to a system that produces slightly fewer PhDs and employs more. Obviously it's a non-starter with congressional action to care about science in this country, but stability is important as well. We can't do our best science while constantly writing grants and worrying that the coffers will be empty in 6 months. Is grinding and demoralizing, not to mention incredibly time consuming.

I would be interested to hear other suggestions.

24 responses so far

Older posts »