Archive for: October, 2010

How to hold an NSF panel hostage

Ever get a panel summary back on one of your grants that has spelling mistakes or looks like it was rushed? WTF, right? I mean, can't these people take the time to tell you how to improve our proposal that they didn't like enough to fund?

Here's the thing, it may very well have been written under pressure while two people waited with bags packed and flights to catch looking over the writer's shoulder.

Let me back up for a second and explain part of the process I didn't understand before this week. I thought that the POs were the ones writing the panel summaries. I don't know why, but that was my impression. I figured they took detailed notes and wrote it all up before reading it back to an attentive panel while the solemnly nodded their heads. Not so much.

The POs do write detailed notes that they use later, but the panel summaries are written by one of the three panelists assigned to your proposal. Those three people are the only ones in the room who have read the proposal and the panel summary is based on the discussion between those people, with the ad hoc reviews taken under consideration. The scribe (in our case, the primary reviewer) then summarizes all the information and includes suggestions to improve the proposal. Once they are done, the other two panelists assigned to the proposal have to read and approve it before it goes to the POs for final approval.

That all sounds well and good, but when do those summaries get written? Well, if you are lucky they get written during one of the first two days while grants that the scribe is not assigned to are being discussed. Ever tried to write a summary of several people's thoughts while an unrelated (and possibly heated) conversation is going on around you? Not so easy.

If you are not lucky, the scribe for your proposal has waited until the last day to even start your summary. Although panelists can change their flights to earlier ones if the panel finishes early (and they often get done around lunch on the third day instead of going all the way until 5:00), no one leaves the room until they sign off on every proposal they are assigned to. That means that even if the proposals you are scribe for are all done, you still have to wait for all of the proposals that you read to be written up by the other scribes. One person, therefore, can keep 6-8 others in the room twiddling their thumbs just because they waited to start their summaries until the last minute. Sometimes, we're talking a couple hours.

You might imagine that those who are waiting against their will with flights to catch could be A) rather "encouraging" of the scribes to get their summaries done, and B) unlikely to do much editing of the summary once it is finally submitted for their approval. This can mean that your panel summary is less useful than it could be if someone who didn't wait until the last minute had prepared it.

Yet one more way a single person can have a major impact on the success of your proposal.

5 responses so far

Panelicious thoughts

It's been a long 48 hours and a lot of science has been discussed. A few observations so far

- You can not get funded if you don't excite someone on the panel enough to push your proposal. The ad hoc reviews are read, but it really comes down to the panel members.

- You NEED to keep improving your proposal. Even if you got an "outstading" rating and didn't get funded, every panel is different. It is scary as hell to tweak something that scored high in a previous round, but I have watch proposals go backwards this week because they thought they could just send it in based on the previous ranking and everything would be fine.

- What ends up in the top 20% is extremely biased by who is on the panel. What gets funded is up to the POs.

- One person can kill or champion your proposal.

- Scientists love to hear themselves talk.

- Being a PO is a tough job.

- Write for a broad audience, because most of the people reading your proposal work on something very different than you.

- Add a financial commitment to your broader impacts section. Trust me.

- Despite the work, being on a panel is a really interesting perspective on how to get things funded.

-I can't wait to get home.

12 responses so far

Grant questions

1) should unfunded PIs be included on panels or study sections?

2) Should postdocs (if funded by Federal funds) be included on panels or study sections?

Discuss with justification of your position.

24 responses so far

Debut

It is always a little strange when you finally get to a day you have been working towards for a while. In this case, it's not even my work that is being evaluated, but I would be lying if I said there wasn't some apprehensive anticipation about starting on my first NSF panel tomorrow. Yes, yes, I know I'll do fine and I know that everyone has been the new person on a panel at one point or another, but that hasn't kept me from re-reading all my reviews, plus those of everyone else on each proposal I am associated with. I've skimmed back over the proposals, especially the ones where my review was at odds with the others (which happened a few times).

It's not so much the having to defend my opinion that concerns me, I do that all the time. It is more 1) not knowing how this all goes down, and 2) being sure I don't miss something that results in someone not getting funded.

As for the first point, I don't want to be scrambling tomorrow after one or two proposals are presented because I realize that everyone else introduces some part of the proposal that I forgot to look over again. Oh, you wanted to talk about that? Um, give me a few minutes.... While this seems unlikely, I'll feel a little better after we get through a couple.

On the second point, I know how tough it is getting funding right now and how important these decisions are. Granted, I won't be making any decision in isolation, but I do feel obligated to fight for the proposals I think deserve it and against those that don't. On one proposal, in particular, I am in the minority opinion (somewhat drastically) but I think the proposal is full of problems. That said, I am open to being convinced otherwise and not being the only fly in the ointment.

More than anything, I just want to get this process started so that I am not trying to anticipate what is going to happen, whether I'll be taken seriously and what my overall role will be.

8 responses so far

And it was said, "Let thee delurk!"

Oct 07 2010 Published by under [Et Al]

Blogging is a funny form of communication. On the one hand it is a social activity, often generating a discussion of sorts among the writer and reader, as well as among readers and even other writers. On the other hand, the proportion of readers that actually comment is quite small, leading to a skewed interpretation of who is actually reading. I know a lot of people see blogs as a sort of magazine rack (I hope I'm not US Weekly) and prefer to just read rather than jump into the fray.

I have no problem with this, but on occasion I find it valuable to see if I can shake the tree a bit and get some of you who only read to voice up. While I enjoy interacting with those who comment regularly, I also like to encourage the silent readership to give me some feedback every now and then. In fact, I did this only a few months ago, but a lot has changed since then in terms of URLs and readership. From that perspective, I don't feel too narcissistic calling out the readership again so soon.

So tell me, dear reader:

1) Have you been reading for a while or found this blog in the past couple of months?

2) If you come by to read regularly, why?

3) Are you in academia/science, and if so, or not, what do you do?

4) What keeps you from commenting more often, or at all?

So let's have it, all you anonymous readers out there. What say you? Yes, YOU. Click in from your Google Readers and participate. Your feedback will be sincerely appreciated.

130 responses so far

Donors Choose time!

The official start date of our Science Blog Donor's Choose competition is Oct 10, but I'm gonna be out of town and a few others here have already turned their reads loose on their giving pages.

You can either click THIS LINK for the donor page I have set up or use the sidebar. The goal here is to pit blog collectives and independent blogs against one another in a steel cage death match (for the kids, of course) for your donations. So, donations made through the pages of various blogs will be our bragging rights as we gather money for classrooms across the country that need some extra help to teach STEM concepts to their kids.

I mostly chose projects that had a direct impact of teaching and learning science, but have a few others on there that I found interesting. I will also be adding new projects as the ones on the current page get closed out, so don't be afraid to check the page more than once!

2 responses so far

An educational perspective

At the moment, I am buried in these damn proposals I need to complete much sooner than I can fit nicely into the waking hours I have between now and the deadline. While this is not an exceptional situation, I have to admit that I am learning quite a bit about proposals, in general, even before my up-coming meeting in DC.

I have many, many reviews of grants I have submitted. I have also submitted a good number of ad hoc reviews myself, but as I mentioned previously, the process as a panelist is very different. Namely, I can read through a proposal and make my judgment on it and after submitting my review, immediately read over what others have submitted. As an added twist, panelists have access to the identities of the reviewers.

Reviews of my proposals are, of courses, nameless critiques of work I am close to. When I write an ad hoc review, it is done so in isolation without ever knowning the eventual outcome unless I dig through the awards page after the fact. Whereas the panel I am on is not my main field of expertise (I have proposals pending in those panels), it is close to a portion of what I do. Because of that and some overlap in other areas, I know almost none of the other panelists, but almost all of the ad hoc reviewers for the proposals I am dealing with. It is utterly fascinating to have all this information laid out in front of you, because like any field of science, the interactions between labs and people is more than citations.

Not only do I have the opportunity to see how other people I know evaluate the same proposal I am judging, but I can apply my insight into personal biases, conflicts and collaborations in a very interesting way. I can see how certain reviewers react to parts of proposals and how they interpret the information provided through the lens of their own work. This perspective is much more informative than I imagined it would be.

What new insights will the actual meeting bring?

4 responses so far

It's time for the bacterial evolution crowd to get their own toys

ResearchBlogging.org
Molecular phylogenetics has revolutionized our understanding of biodiversity and evolution like no tool before it. The advent of using the divergence among the gene and protein sequences of different organisms as both a proxy for the biological species concept (at the species level) and a way to compare organisms with no obvious shared morphological or cellular features has resulted in the most sweeping change of how we classify the diversity of life since Linnaeus. Like any paradigm shift, however, the new data available opened up a Pandora's Box of new issues that needed to be grappled with.

One of the major discoveries that came to light was that prokaryotic organisms (often called "bacteria", but actually made up of two very distinct lineages, the Eubacteria and Archaea) are not quite what we thought they were. You see, Darwinism laid the foundation for how evolutionary biologists interpret the world, and while Darwin got a lot of things right, there were many things he had no idea even existed, that have shaped how we understand evolution today (as there will be many things we don't know about today that will shape how scientists in 150 years understand evolution). While Darwin's concepts for descent with modification fit the multicellular world pretty damn well, things get a little muddled in the unicellular world, especially when we leave the eukaryotic cells many of us know, and move to the bacteria.

Bacteria don't play by the rules that work so well for the multicellular among us, and the differences are at the heart of a scientific debate that has been playing out for two decades. In the early stages of molecular phylogenetics, it was thought that the history of life's evolution could be revealed if we compared the sequence of a ubiquitous gene across all taxa. The small subunit (SSU) of the ribosomal RNA was chosen as the appropriate gene because no life form lacks ribosomes, and because of it's important role, the DNA sequence that codes for the functional RNA is highly conserved. As a result, the SSU is the most diversely sequenced gene around and is used today as the gold standard for comparison of bacterial and some eukaryotic species.

But there was a surprise in store when people started to sequence other genes from diverse bacterial lineages. The phylogenetic trees resulting from these new data did not match those of the SSU. In some cases, the differences were dramatic. It had been clear for some time that bacteria could exchange DNA, but the extent and potential evolutionary distance of the exchange took the community by surprise. Indeed, there are some bacterial genomes that appear to be a mixed bag of genes from diverse sources, such that their closest sister taxon can not be identified with certainty (e.g. Zhaxybayeva et al 2009). The evolutionary picture within and among bacterial groups is so confounded by lateral gene transfer (LGT), that the term "Tree of Life" has been abandoned by many, in preference of some variation on "Web of Life".

On top of the issues with LGT, there is the problem that bacteria just don't speciate the way eukaryotes do. The most recent issue of Biology & Philosophy (2010, 25(4)) is dedicated to the discussion of bacterial evolution and how it differs from that of eukaryotes, but the paper by Lawrence and Retchless (2010) really drives to the heart of the problem: we can not use models fashioned after evolutionary patterns in eukaryotes to understand prokaryotic evolution because the speciate in fundamentally different ways.

This message has been repeated far and wide and several researchers are actively proposing novel models for use in prokaryotic systems (e.g. Bapteste et al. 2009). Despite this, paper after paper are churned out using traditional phylogenetic methods to try and classify bacteria using the same assumptions applied to eukaryotic systems. What the hell is going on?

To tell you the truth, I don't know. I think part of the issue is availability and acceptability. Tried and true phylogenetic methods are well known and reasonably well understood by a large community of people. If one is writing a paper or grant proposal, introducing controversial or novel methodology is one way to make the process exceedingly more difficult on yourself (a whole new thing for reviewer 3 to reject outright without understanding it!). Playing it safe means that you can cite the large body of literature that is also applying the same methods, in some sort of schooling fish mentality. Another factor might be the radicalization of the LGT movement by researchers not willing to abandon the Tree of Life idea, for personal or political reasons. Rarely have I seen such vitriol unleashed at conferences as when the topic of rampant prokaryotic LGT comes up.

But the data are the data. It is abundantly clear that bacteria violate the assumptions inherent in the methodology currently used to model evolutionary history (even worse than eukaryotes do, but that's a different story). Until the bacterial evolution community comes up with and embraces new methods to model prokaryotic evolution, leaps in our understanding of that process will be limited.

References
Zhaxybayeva O, Swithers KS, Lapierre P, Fournier GP, Bickhart DM, DeBoy RT, Nelson KE, Nesbø CL, Doolittle WF, Gogarten JP, & Noll KM (2009). On the chimeric nature, thermophilic origin, and phylogenetic placement of the Thermotogales. Proceedings of the National Academy of Sciences of the United States of America, 106 (14), 5865-70 PMID: 19307556

Lawrence, J., & Retchless, A. (2010). The myth of bacterial species and speciation Biology & Philosophy, 25 (4), 569-588 DOI: 10.1007/s10539-010-9215-5

Bapteste, E., O'Malley, M., Beiko, R., Ereshefsky, M., Gogarten, J., Franklin-Hall, L., Lapointe, F., Dupré, J., Dagan, T., Boucher, Y., & Martin, W. (2009). Prokaryotic evolution and the tree of life are two different things Biology Direct, 4 (1) DOI: 10.1186/1745-6150-4-34

6 responses so far

What I am listening to while I read your proposal

Oct 02 2010 Published by under Etc

"The cost is more than what you get paid, but do it anyway...."

Word.

6 responses so far

Job vacancy: Journal Club Killa

Oct 01 2010 Published by under [Education&Careers], Etc

Having filled our previously advertised job in Seminar Napping, Employment University has a new position available immediately, entitled, Journal Club Killa.

Requirements:

Applicants must religiously attend journal clubs on topics they know little about, but purport to do research in. They must be able to ask a bevy of questions that derail the discussion into obscure topics of self-interest and be persistent enough to smother constructive conversation. The successful applicant will also never print out the paper being discussed and repeatedly spout random inquiries that make plain the fact that they never read the paper to begin with.

Preferred Skillz:

Candidates who prefer to talk at length about the way research was done 20 years ago are strongly encouraged to apply, particularly if they try to pass it off as though they are doing "cutting edge" work. Using the same arguments week after week, regardless of the paper of choice will also be reviewed favorably.

Compensation:

The job pays only in self-satisfaction and ill-tempered glares from presenting students, but if you are interested in this position, this seems to be all the compensation you ever seek.

Applicants should send three letters of reference attesting to their ability to read from the paper of the student next to them and inability to use anything but their outside voice when asking said student questions while others are talking. If possible, letter writers will be asked to comment on the candidate's refusal to back down from any position on how new data are wrong, despite overwhelming contrary evidence.

Please forward application packets to:PLS, Department of Science, Employment University

6 responses so far

« Newer posts Older posts »