Archive for: December, 2012

NSF DEB data from 2012 shows distrubing trends

Dec 21 2012 Published by under [Education&Careers]

Fresh off the data release by IOS, DEB has sent out a similar (though slightly harder to decipher) dataset. Let's break down the data!

Preliminary Proposals submitted: 1624 across 4 clusters

Panel recommendation for invitation: 395

Preliminary proposals invited: 380

Overall Invitation rate: 23.4%

So, the invite rate for DEB was slightly lower than the 30% invited by IOS. What about the early career and PUI peeps?

Groups of concern:

Early Career Investigators

Preliminary proposals submitted: 401

Preliminary proposals invited: 82

Invitation rate: 20.4 %

Primarily Undergraduate Institutions

Preliminary proposals submitted: 287

Preliminary proposals invited: 47

Invitation rate: 16.4%

Okay, the Early Career people at DEB seem to be getting similar invite rates as those in IOS, but those from PUIs are having more success in DEB at jumping the preproposal hoop. What about the full proposals?

Full Proposal Panels – Fall 2012:

Full Proposals submitted: 380

Panel Recommended for Funding: 259

Panel Recommendation Rate: 68%

Anticipated Overall Funding Rate: 22%

Here's where comparisons to the IOS data get murky. The DEB data are reported as "Recommended for funding" include proposals "across three categories, High Priority, Medium Priority, and Low Priority for Funding". Also, these numbers are based on the budget under the Continuing Resolution, which is 80% of last year's budget. The IOS numbers were all based on "High Priority" status, so we're looking at apples and oranges here. We can tease a bit out of the concern categories, however.

Early Career Investigators

Full Proposals Submitted: 82

Recommended for funding: 29

Success Rate: 35%

If we use the numbers from the section on EC preproposals (401 total), we get a success rate of only 7% for the Early Career people in the new system. That rates is less than half the 16% from last year and well below the ~12% success rate in the two previous years. How about the PUIs?

Primarily Undergraduate Institutions

Full Proposals submitted: 47

Recommended for funding: 18

Success Rate: 38.3%

Calculating from the 287 preproposals submitted, we see a success rate of 6.3%. Again, we see over a 50% drop in success rate from last year, and the previous 5 years where rates have averaged around 12%. Despite the high "success rates" listed in these sections, the total funded need to be compared to the total submitted, not just those that jumped the preproposal bar.

What does it all mean? The numbers make it look like DEB does have significant reason to be worried about its Early Career and PUI categories of investigators in the new system. Both groups have taken a hit in their ability to compete, as was the initial concern. So far, the IOS numbers do not show the same trend, so we should be asking what the two divisions are doing so differently.

Food for thought this holiday season.

11 responses so far

Christmas music that doesn't suck

Dec 21 2012 Published by under [Et Al]

Tired of 1950's Christmas jingles assaulting your ears at every public venue? Here is the antidote.

No responses yet

TT Phone interviews: Don't talk yourself off a short list

Dec 18 2012 Published by under [Education&Careers]

I never did a phone interview for my interviews when on the TT job hunt and so I balked when my search committee colleagues suggested we do a round. My though was simply that we would have a bunch of superficial conversations with people that wouldn't change our perception.

I was wrong.

I was quite surprised at the variation in how people handled the phone interviews. I fully recognize that candidate anxiety is high this time of year, as examplified by Reaction Norm's posts on phone interviews and some of the questions. It's a tense time and I remember it well - sitting in the kitchen with my very pregnant wife trying to figure out if I needed to look for another postdoc or whether the job deities would smile favor on me.

But luck favors the prepared and it's unwise to go into a phone interview without having prepared. There is a lot of interview advice on Dr. Becca's job advice aggregator, but fresh from the search committee (SC) side of phone interviewing I've posted some suggestions here for candidates.

1) Do your homework. I don't think it is at all necessary to read papers from people on the SC or anything like that, but you should look into the department, college and university. Know a little about the size of the school and the size of the department. Identify resources, like centers, that may be helpful to you.

2) In a similar vein, find a few people who you might like to talk to should you be invited for a campus interview. The web makes this easy, so have a feel for who is in the department and what research overlap there may be.

3) Be organized in your answers. If you got the questions ahead of time make sure you have some talking points and don't ramble for days. If you didn't get the questions, consider the question and then get to the point. A SC doesn't need to hear a dissertation-level answer and you want to leave time at the end for your own questions. Common questions to prepare for:

-Why is this university a good fit for you (Reaction Norm's fav). Don't go on about "the international reputation of your prestigious university" like you are reading from one of those "Dear Esteemed Professor, I want to work in your lab on *cut and paste topic* using my background in *completely unrelated field*" emails. Identify resources that would be helpful. Proximity to field sites? Potential colleagues? Whatever, but give a couple specifics.

-What could you see teaching here? Demonstrate that you have thought about this. You don't need to see what classes need to be offered (that's for later), but the committee wants to know you have fired a couple neurons on the topic. What level course would it be? What's the topic? Have you considered the general outline? Would it have a lab? Field component? Etc....

-Describe your experience/plans in applying for extramural funding. Have you been involved in submitting grants? Have you submitted something yourself? This your opportunity to bring up unsuccessful proposals that don't show up on your CV. Also, have a feel for WHERE you would apply in the future. And not "I would apply to NSF", but how about "I would probably target the Evolutionary Processes cluster in NSF's DEB".

4) Have questions. Don't be surprised at the opportunity to ask questions you might have, remember that the SC is ALSO trying to recruit you. They'll want you to have a chance to figure out if this is good for you too. Some questions were aggregated by Reaction Norm, but I think a lot of those are more on campus type questions. For the phone interview I would stick to something like:

-What are the teaching expectations and are the different for pretenure vs. tenured faculty?

-Is there lab space available or will space have to be renovated?

-Is available as a shared resource?

-How are grad students brought into the program (rotation or not) and how are they supported?

-If your work might cross departments or colleges: Is there a history of collaboration between Search Department and XXXX Dept?

-What is the time line for the interview process?

5) Finally, be yourself. I know that is cliché, but you can't guess what the department is looking for, so you need to just let them know about you. Be organized in the talking points you want to get across, but don't read a script. Try to roll with the tone of the committee and just let them know who you are. If you make it clear that you are interested in their department and excited about the opportunity to come discuss in person, you've done your job.

7 responses so far

Life of an NSF Program Officer Part 3: The process of funding

Dec 14 2012 Published by under [Education&Careers]

Today, guest blogger and NSF PO, Michelle Elekonich, writes about what is going on at NSF between and after panels. What has to happen for your proposal to be picked up? Where does the money come from and how are panels put together. If you missed them, Part 1 and Part 2 of the series were posted earlier. With this last installment posted, I want to thank Michelle for taking the time to write this, get it approved by NSF and answer questions that have come up.

Since I arrived before the preliminary proposal system was instituted you might wonder what has changed at NSF with the new preliminary proposals? When I arrived I asked my program’s staff to graph the proposal load over the last 5 years- it had almost doubled in that time. A good chunk (about 20% I would learn) came back each time from the previous round, often with few changes as there had probably been less than a month since the prior version was declined. One of the first things I had to do when I arrived was build a panel and get outside reviews for the proposals that had just come in. I quickly learned how difficult it was to get outside reviewers and got new appreciation for program directors and editors of journals. My own record was 13 requests for 2 outside reviews- my co-program director’s record was 17! Mostly I ended up asking 6-8 people to get 2 reviews. With 120 proposals that was a lot of requests between us for that first panel and it only grew from there….it was quickly clear that this was problematic and it ended up being one of the major drivers behind the new process.

That was the first of many panels I would put together-it is really an art form. There are a lot of requirements for the composition and the division director has to approve the final panel list. Panels must have some new people but not too many; some people from primarily undergraduate institutions (Yes! NSF wants you!) and some from state schools and some from the big research universities; in IOS we are not allowed to have more than two from the same state without getting special permission from the division director; we must balance the number of men and women and include members of underrepresented groups and of course most importantly panelists have to have the expertise to cover the proposals submitted. Phew! Flexibility to change relative to submissions and the ability to have more people participate over time are two of the reasons IOS doesn’t use standing panels.

Now, with the preliminary proposal system program directors are asking for fewer reviews. But the workload at NSF hasn’t really decreased overall, and program directors and the administrative staff are working at top capacity throughout the spring. But, the new review system has provided a couple months in the summer for analyzing the funding trends, doing outreach and going to one’s lab for a longer period. It is just as much work, but feels saner, because now there is a break where there was none before. The first year and a half I was here we would just finish one round, be having phone consultations with declined PIs and (patiently) answering questions from people getting ready to submit all at the same time and then start all over again when the next batch of proposals arrived. If there was any break at all it was just a couple days.

One thing that has not changed and that surprised me when I came to NSF is the amount of work the program directors do after the panel. In preparation to make the invitations or the new recommendation decisions program directors have to analyze the distribution of the current awards relative to topics, states, broadening participation and other division and directorate priorities. This is what program directors are referring to when they talk about portfolio balance.

Most often there is far more good science than the money available can fund, so hard decisions have to be made. Even in high priority not every proposal may be recommended. This is where the portfolio analysis comes in- the NSF mission is to move science forward so we don’t necessarily want to keep funding the same things that we already have funded and of course the panel doesn’t know what is in the entire portfolio. So the program directors evaluate the science relative to the portfolio, and the panel’s input about strengths and weaknesses plus any knowledge of their own to come up with a potential list to recommend. Within some clusters there is one program director for each program or for a subset of the program and they make their own recommendation decisions. For example, in IOS, the Processeses, Structures and Integrity Program has a plant panel and an animal panel and each has its own program director who has a specific part of the money to use. Other programs, like Animal Behavior, have multiple program directors working from one program budget so they all have to agree on what to recommend. A lot of intense discussion goes on. Once they have a list it gets presented to the division director, deputy division director and in IOS the science advisor at a meeting a few weeks after the panel. Only then can the program directors start calling people.

With the new preliminary proposal system, in the spring the invites are processed first in IOS so that PIs will have as much time as possible to write the full proposal. In the fall, the declines are processed first so PIs have time to revise for the next preliminary proposal deadline. Because the panels are spread across a two month period, the post panel meetings are spread out and some clusters finish, write their analyses and get division director approval sooner and thus contact their PIs before others. The shorter preliminary proposals gave the reviewers less to read but did not really change the NSF process – in fact the larger number of them made it just as much work here. The only real work savings was in the fall full proposal panels by having to make fewer ad hoc review requests. But, the new process let us work smarter- we developed tools to automate some parts of the processing of the streamlined preliminary proposals so we could handle the larger numbers, by knowing what would come in as full proposals we could build panels sooner so people were less booked up when contacted, and the smaller number of full proposals let us put more energy into getting better reviews for each one and this helped to maintain the quality of the review.

Another aspect of the post panel processing involves writing a review analysis that documents the program director’s assessment of the review and reasons he/she made the decision. Each proposal submitted through Fastlane (no matter what type – preliminary proposal, full proposal, supplement, EAGER etc -or what the decision is) has to have one. Once this is done, all the documents from the panel and the review analysis go into the electronic file and it is checked by the staff to make sure nothing is missing and then each decision is approved by the division director with the help of the deputy division director and science advisor.

Throughout all this, the program directors are also looking for co-funding to stretch their program dollars. If a program director can get money to fund part of one proposal then it means he/she could fund an additional proposal or perhaps cut less money from some other proposals. I knew that there was some negotiating for co-funding that went on before I came to NSF, but I did not know that there are actually a lot of different programs at NSF that a program director could get co-funding from and that it can take months! For example, the deadline for applications for EPSCoR or other co-funding might be a couple months after the panel and it can take another month to hear back. Generally these NSF wide programs require a written document to apply for the co-funding. I find it ironic to have to write a mini-proposal to get co-funding for a proposal! Other times program directors from two different programs will decide to share the funding (often as the result of a co-review) and that is faster and easier.

Even when all is said and done in IOS, awards still have to be processed through the Division of Grants and Agreements (DGA)- while program directors can recommend, the folks in DGA are the ones that actually give the money and create the contract with your university. They make sure that all the federal rules for using the money are followed by the universities and that the contracts are complete. All in all, it’s a pretty complex process.

6 responses so far

The Life of an NSF Program Officer (Part 2): What is it like to work at NSF?

Dec 13 2012 Published by under [Education&Careers]

Today's post is the second of three (part 1 here, part 3 here) Guest post installments about life as an NSF PO. Michelle Elekonich has been kind enough to write her thoughts down about her experiences at NSF.

What is it like to work at NSF? Much more fast paced and deadline driven than being an academic and constrained in different ways. It is a highly regulatory environment and the decisions have far reaching implications – even beyond the lives and careers of the PIs involved. Decisions made this year can affect what can be funded next year, and new programs may compete for funding with more established ones. Because about half the scientific staff are rotators it sometimes reminds me of being in grad school where everyone is in the same boat trying to figure things out. Once a new rotator gets to NSF they start learning all the software that is on the other side of Fastlane and go to what NSF calls “The Program Management Seminar”, fondly known as “Bootcamp”. It is a 4 day seminar where you learn all about the NSF’s history, working in the federal context, what the various service units at NSF (like the Office of the Inspector General, Travel, the Office of Legislative and Public Affairs etc) do relative to the review process and your duties and how to handle problematic situations- like being called to testify before congress or having one of the awards you manage be featured in Senator Coburn’s Wastebook (this one actually happened to me). One of the most important things I learned was that almost all of the review process varies around the foundation- some directorates or divisions within a directorate use preliminary proposals (BIO was not the first to do so), others don’t; some have one deadline a year others don’t ; some use only external reviews, some use only panel reviews and some use both; some have panels made up of program directors and some have outside PIs run the panel and the program directors observe, while others like BIO have panels run by program directors but made up of PIs.

Bootcamp is where I first learned about the Hatch Act which prohibits federal employees from lobbying congress. This is why NSF is not working the Hill to get more money. In fact, the only time NSF gets input into the budget is in the request to the Office of Science and Technology Policy as the budget is being developed – typically a year and half ahead of the actual start of that budget. Because the congress almost never passes the budget on time, all of the government starts most fiscal years on a continuing resolution- that puts NSF programs typically at 80% of their prior year’s budget. This could last a month or many months. It wasn’t until I came to NSF that I realized why it mattered to me in my lab back at UNLV that the government was on a continuing resolution (CR in acronym speak- ☺), it limits the number of awards a program officer can make right away with the fall round of proposals. Fortunately, usually the CR doesn’t last and more awards can be made later. With the new preliminary proposal/full proposal system it means that the funding rates for the core programs won’t be known until late in the fiscal year.

13 responses so far

The life of an NSF Program Officer (Part 1 of 3)

Dec 12 2012 Published by under Uncategorized

I'm very excited this week to have a guest blogger to talk about the experience of an NSF program officer, from the inside. Rotating PO, Michelle Elekonich, has agreed to provide some perspective on how people become NSF program officers and what that decision entails. Over the next two days I'll have two more posts in this series. (Part 2, Part 3)

Let me introduce myself- I’m Michelle Elekonich. I am an associate professor at the University of Nevada Las Vegas (UNLV). My honey bee research focuses on cellular and physiological mechanisms of behavior, stress and aging (NSF funded) and also on the honey bee disease American Foulbrood (USDA funded). I came to NSF in August 2010 to be a program director in the Behavioral Systems Cluster in IOS and am currently the science advisor for IOS, part of the division leadership with the Division Director and Deputy Division Director. Now the disclaimer: these are my experiences and not an official NSF announcement or policy statement. Your mileage may vary.

You might be wondering how someone becomes a program director….I had been an external reviewer and panelist for NSF for a few years when one of my colleagues from UNLV who was rotator at the time called and said he wanted to nominate me as a rotator for the Animal Behavior Program in IOS. (Little did I know then each rotator is expected to help find his/her replacement). I agreed and sent him my CV. I figured an interview would be a chance to find out more and then I could decide …. it was fun to talk about my NSF funded work at NSF and meet all the program directors and administrative staff.

Then I waited …I interviewed in March, they called in June and I arrived in August. NSF spent July negotiating with my university – depending on how a rotator is appointed NSF returns different amounts of the salary, health benefits and retirement to the university. I was appointed as a Visiting Scientist, Engineer or Educator (VSEE – everything at NSF has an acronym) and my university used the salary savings to hire a temporary instructor. Some of my NSF rotator colleagues got some of the salary savings back to pay a tech or postdoc to run their lab while they were gone. I wish I had thought to ask for that! Since the colleague that recruited me had not gotten any accommodations, it did not occur to me that I would be able to either.

So how could I leave my lab on such seemingly short notice and come to NSF? About 8 mos. before I got that fateful phone call, my close collaborator left UNLV for a better position (good for him - we continue to collaborate), both post docs got jobs (Yay!) and my technician joined another lab as my grant was winding down (ya gotta do what ya gotta do). I was post-tenure and really feeling like I needed to get some perspective and I wanted to do something that mattered…since I could not afford to do a sabbatical on the half salary that my university would give me and I really enjoyed being a panelist – I thought why not- what better way to get a really BIG perspective on science than go to NSF? It could be an “alternative” sabbatical, sort of like doing an alternative spring break with Habitat for Humanity. I had just taken two new grad students, but they were deep in course work and I figured that it would likely be a year before they got most of their courses out of the way. NSF has a internal program called Independent Research and Development which would pay for my travel back and forth to my lab (I go for a week or so about 4 times a year and have lab meeting every week by Skype) and pay for me to attend meetings where my students and I are presenting – which also happen to be meetings that NSF wants representation at anyhow. You can read a little about my students’ reactions in this article in the Chronicle of Higher Education. I’ve ended up staying longer than I expected, but I think that has helped them be more independent.

The biggest determinate for people deciding to be a rotator at NSF seems to be timing – professional and personal. I’ve just described the timing professionally, personally the timing was also good. My stepdaughter lived with her mom and was finishing high school, so I did not have any children to worry about. Due to the recession, my husband’s business closed, so he had no reason to stay in Las Vegas and could join me in Arlington. Both of those things made the family issues easier for me than they might have been - but I know multiple rotators that have brought their families and enrolled their kids in new schools locally. For really little kids, NSF even has a day care on site.

10 responses so far

How did NSF beginning investigators fare in the new system?

Dec 10 2012 Published by under Uncategorized

There's a webinar today for the IOS Division of NSF to discuss the current state of NSF affairs. Based on the slides sent out, there is a lot of general info about the process that will likely be serious rehash for anyone who has been paying attention to all the changes. However, there is some interesting data on how reviewers felt about the process and how Beginning Investigators (those with no previous federal funding) faired.

General:

1) Ad hoc reviews were cut from 14,000 in the old system to 2,500 in the new.

2) The 30% full proposal target was hit the year.

3) IOS reports that 80% of PIs used to only submit once a year anyway (who are these people?) so the new system doesn't change behavior for the majority of PIs.

Reviewer specific

1) 65% of preproposal panelists thought the new system improved their experience, with only 20% feeling it detracted.

2) Only 20% of preproposal reviewers across IOS and DEB spent more time reviewing, compared to full proposals. This is relevant because there were many more preproposals to read than panelists would get in a full proposal round.

3) In the full proposal round, 74% of IOS panelists thought their experience was better this year than previous, with only 5% indicating a worse experience.

4) Whereas nearly 70% of panelists did not feel there was a change in the Intellectual Merit at the full proposal stage, nearly a third saw improvement.

5) Over half felt the full proposal Broader Impacts were improved, compared with previous years.

Investigator specific

1) The percentage of Beginning Investigators who were ranked in the High Priority category was on par with previous years.

This is notable, as some feared the Beginning and New Investigators were likely to take it in the teeth. These data suggest otherwise.

2) Both the percentage of submissions and high priority rankings from those working at RUIs (mostly undergrad institutions) was up in the new system.

Unfortunately, we do not have funding rate data yet because the budget isn't settled, but these indicators appear to be validating NSF's position here. Panelist satisfaction is up, ad hoc requests are down and the fear that some groups of investigators would be unfairly affected is not borne out by the data. I'll be listening in for other tid bits later today.

18 responses so far

When to apply for your first NSF grant

Dec 07 2012 Published by under [Education&Careers]

A lot of people starting up on the TT have asked me when they should apply for their first NSF grant. The answer to the question has nothing to do with time, but with data. One thing I really didn't appreciate early on was the inappropriately named "preliminary" data. I thought I could come in with a good idea, some supporting literature and a bit of data and it would totally be fine.

Not so much.

What I have learned along the long path strewn with the bodies of fallen proposals is that the bar for proof of concept is much higher than the n00b thinks. The unfortunate reality is that you've got to demonstrate your ability to generate data that support your hypotheses for at least two of your three aims. In many cases, you need to me most of the way to The Answer.

Established PIs can lean on a track record and toss in some suggestive data and off they go, but when starting out it is much less simple. If you were a forward thinking postdoc or had a mentor who knew how to prepare you to battle it out on the TT, chances are you brought some solid data to you lab to start your new life. In this case you may be able to send in a proposal right away.

If, like myself, you've decided to take your new lab in a new direction, you have a different row to hoe. If there's one thing I would love to tell N00b me, it is to get away from the computer and into the lab. I submitted grants in my first year that had exactly zero chance of getting funded because they lacked enough supporting data. Instead of slaving away on my first proposal, I could have been churning out the data to support it. While not crushing, this was a rookie mistake.

So rather than deciding on a particular time to write a proposal, decide when you think you have enough data to assuage reviewer concerns about the feasibility of your project.

The twitter conversation this morning that was the seed for this post is on storify here.

5 responses so far

Amateur

Dec 06 2012 Published by under [Education&Careers]

Thought my readers might find this perspective interesting:

The NSF system is designed for amateur, not professional, scientists. I realize many of you would prefer this but I find it asinine given that state and federal taxpayers are the same damn people.

Go comment.

7 responses so far

Thoughts on the transparency of panel service

Dec 03 2012 Published by under [Education&Careers]

As I mentioned on Saturday, I strongly encourage junior people to try and get on an NSF panel. A couple of people commented that they have tried to get on a panel and not been successful. Although this has not been my experience, I can see how this could easily be the case. As I already stated, POs need to balance a number of things when putting together a panel. If your research topic is smack in the wheelhouse for a particular panel, there may be many willing participants who fill your "panel niche" and it may be tough for you to get into the rotation.

What surprised me a little was Zen's comments:

As solid as this advice is, it bugs me. It makes it look like getting an NSF grant is determined by unwritten standards maintained by an established network of well-connected people, with fairly high barriers to entry.

The NSF shouldn't a be a country club....

....That there is a substantial number of replies indicating people want to serve, but haven't been invited, and that there's no clear understanding of why that is, does not help promote the appearance that panel selection is a transparent process.

This line of thinking doesn't make sense to me. How, exactly, would you make panel invites a transparent process? Considering that NSF heavily guards it's panel membership, I don't see how this goal is attainable. I'm not sure it should be.

The reality is that some people make better panelists than others, based on their experiences. Maybe the PO has seen your proposals and likes what they see. Maybe you tick the right boxes in terms of the knowledge that a panel needs to fill a hole. If it were just first come first serve then there would likely be large gaps in the field coverage of the panel. When you account for the need to include gender, ethnicity and employment stage/place balance, yes, there will be some people who have a hard time getting on a panel.

There's nothing Country Club or In Crowd about it. I did not know any of the POs on my first panel and my training was in another country, further reducing my connectivity. I contacted my PO after an unsuccessful application and asked to serve on the next panel while revising my proposal. For me, it worked. For others it may not be that simple, depending on their expertise and the needs of the panel.

I don't see how the selection process could or should be a "transparent" process, but maybe there are some arguments out there.

14 responses so far

Older posts »