Archive for: August, 2010

How much data should PIs check?

It sounds like a stupid question until you are balancing 4 different projects with 6 people generating data, but how much responsibility do PIs have for the data produced by their trainees and how deep does one have to go to ensure due diligence? Look at the raw data? The analyzed data? The analysis methods? Is there a difference between new and seasoned trainees?

In more than a few places (an example here), DrugMonkey and others advocate for new faculty to get away from the bench and focus on running the lab rather than generating the data. I don't have a problem with the advice, but inherent in that is the requirement that the trainees generating (and almost certainly analyzing) the data be trusted to do so if the most rigorous ways they know how to.

Obviously, the PI has the responsibility to train the people in their lab to perform these analyses, but how much double checking should be done? A case in which the result is clearly at odds with expectation is an obvious example for follow-up, but rarely are cases of data mis-analysis (or worse, data fraud) that actually reach the PI, cut and dry.

For the most part, it is the inadvertent mistake that can skew results without raising suspicions. Removal of "outliers" for "better fit" (which, BTW, can be very sketchy), sample contamination between two close-related species, slight mis-alignment in a phylogenetic alignment, etc., etc. The careful and experienced trainee may catch these, but any lab is often made up of trainees with a range of experience and aptitude for recognizing subtle problems. It is not uncommon for even triple-checked data to be later found to contain a significant error, which gets us back to the original question: How deep into the data should a PI go to make sure the end product represents the proper quality of data?

I don't know the answer and I suspect that it becomes a case by case issue, but for the PIs out there, how do you handle this? For the trainees, how much does your PI check through your work?

15 responses so far

The delicate daycare dance

Aug 30 2010 Published by under [Education&Careers], Etc, LifeTrajectories

From the perspective of planning one's week, I don't think there is any toddler malady worse than a head cold. It's right in the middle between "Oh, she's staying home today" and "Meh, daycare will deal." because it can clear right up or get worse by virtue of the fact that the kid can't sleep while coughing.

So the debate begins.

She's not that bad right? She hasn't coughed in a bit. Do we just send her hoping everything is fine? Maybe it will be and we can both get a full day at work, but if she gets 'kicked out' at any time during the day one of us has to leave work immediately and on short notice. To make matters worse, they ask that she not return for 24 hours.

You lose, back two spaces.

Commence schedule comparison. What do you have this morning? What can get done from home? How about each taking half a day? What about taking her to work? Where will she have the easiest day so she can get better quickly and our schedules can get off the tilt-a-whirl?

Where is the sweet spot that allows us to take the least amount of work off while getting her healthy the fastest?

We do this probably a dozen times a year and sometimes we get it right and others we don't, but it never fails to throw a monkey wrench in the best laid plans.

11 responses so far

From the department of WTF did I agree to?

Aug 26 2010 Published by under Etc

There is a stack of paper on my desk 4 5/6" tall, representing over two packages of printer paper (>1000 sheets). It is a beast. It is the grants I am either primary or secondary reviewer on for an NSF panel this fall.

Dude. Fuck! Sigh.

16 responses so far

Response to Feschotte

Before I went away last week, I wrote a post that was critical of a recent paper in Trends in Ecology and Evolution for essentially equating the term 'Eukaryotes' with animals. The corresponding author of that paper, Cedric Feschotte, came by to comment on the post, but because I was away at the time I am only getting to the response now.

I'll start by saying I have nothing against Cedric or his lab and only have a vague familiarity with their work, so I did not pick this paper for any other reason than it was the most recent paper that I have read, which made this mistake. The title of the paper is “Promiscuous DNA: horizontal transfer of transposable elements and why it matters for eukaryotic evolution”, and my issue with it is simply that the entire focus is on animals - a very small portion of eukaryotes. I take issue with this particular offense because I think it is disingenuous to claim to have discovered something in all of complex life, when in fact the findings are limited to a small subset of that group.

I am actually happy that Cedric came by to share his opinion and hopefully he will continue to. In the interest of discussion I'll include the points of comment by Cedric in italics and my response below each.

As the corresponding author on the TREE paper you are referring to, I am feeling obligated to rectify several inaccuracies in your post.

First, I have to say I am quite offended, not to say shocked, by your post. Because my group is actually one of the very few, along with my wife’s Ellen Pritham, to work on transposable elements in several ‘protist’ genomes, including apicomplexans, trichomonads, Entamoeba rhizarians…etc. We are collaborating with a number of eminent protist biologists and hence we are well aware that the diversity of eukaryotes is far from limited to metazoans.

Being aware of the difference is not the same as structure your writing to reflect it. I'm reading one paper without background knowledge of what Cedric's group has done in the past, nor who their collaborators are. IMO, this knowledge should not be required when evaluating an article. If you're aware of the diversity, then make that clear in a paper on animals by not claiming it is research that 'matters for eukaryotic evolution'.

Unlike most others in the TE field, we always try to be very careful in our wording to avoid generalizing some of our findings in metazoans to other eukaryotic groups. Granted, it is not often easy to do, it is a fine line between oversimplification and overspecialization – especially in journals that have tight constraints in title, abstract, references and overall text length. For simplicity and brevity, we have to use general terms such as ‘eukaryotes’ in titles and abstract, to make it clear that prokaryotes will not be covered and also, in this particular piece, to emphasize the differences with what is known on HT of mobile elements in prokaryotes. I guess we could have used ‘multicellular eukaryotes’, although I suspect the editor would have demanded to delete ‘multicellular’!!

This is strongly reminiscent of a conversation going on over at DrugMonkey's blog about the veracity of statements by scientists. It is easy to blame the editor and say that that they wouldn't allow a different title, but the people at TREE are good and certainly value accuracy over hype. And perhaps that is the root of the problem when we are talking about cases where the authors are fully aware that they are grossly over-stating their findings in a title and abstract. Are we getting too far into the journalistic mindset when it comes to titles? Is it all about grabbing a reader with a title, even at the cost of accuracy? I certainly hope not, because I think that undermines the central believe that scientists should strive for accuracy to the best of their ability at that time.

The telling statement is that it is Cedric's assumption that the more accurate title would have been turned down, but it was never considered in the first place.

More importantly, however, is that even the revised title proposed by Cedric would have been incorrect. In what way do animals represent 'multicellular eukaryotes' any better than they do eukaryotes? Off the top of my head I can think of 4 other independent lineages that have evolved multicellularity. None of these are discussed in any detail in the paper in question.

Secondly, and perhaps more importantly, before accusing us to cite papers (Keeling and Palmer) after only reading their abstract, you should be reading more carefully the ones you are writing about in your blog. In the Introduction of our paper, which you are paraphrasing partially and inaccurately in your post, we were careful to say that we are placing emphasis in this review on transfer of genetic material between the NUCLEUS of MULTICELLULAR eukaryotes. Indeed we state in the very first sentence of the paper:

“Horizontal transfer has long been recognized as a crucial
mechanism driving bacterial evolution [1]. In contrast, the
evolutionary significance of horizontal transfer between
the nuclear genomes of multicellular eukaryotes has
remained more obscure [2].”

We are perfectly aware of the growing number of HGT events reported in unicellular eukaryotes, a topic well covered in the Keeling and Palmer review. But note that almost all of these are transfers from bacteria or plastids to nucleus. There are still very few clear instances of HGT reported between the nuclear genomes of eukaryotes, even for unicellular organisms (but see the remarkable case of bdelloid rotifers). As for plants, until last month or so, there had been not a single report of nucleus-to-nucleus HGT between plants. All the cases of HGT reported by Palmer and others involved organelle genes.

In terms of TEs, the focus of our review, we still no very little on unicellular eukaryotes (aside from a handful of yeasts).

This is a non-sequitur. Just because the data on TEs aren't readily available in published genomes and you are unwilling to analyze newly released non-animal genomes for TEs in a review (which, BTW, I don't have a problem with), how does that excuse the title and abstract? I still don't get how it is okay to use a small subset of an enormous group and claim it as representative. If the data from the rest of the group is unavailable, then say so, but don't claim to speak for it by using the term eukaryote (or even multicellular eukaryotes) when referring to animals.

Most of the protist genomes sequenced so far are from species with very small genomes (e.g. apicomplexans) which seem to harbor very few, if any TEs.

This is blatantly false. Oomycetes (5 available)? Diatoms (3 available)? Ectocarpus (multicellular, but released after the paper in question was likely in press)? Many green algae? Emiliania? Dictyostelium? There are many more even before we get into the multiple fungal genomes and things like Monosiga, which would be a particularly good comparison against animals. Again, being aware of the diversity and actually using it are two very different issues.

Out of the >200 solid cases of HT of TEs reported to date, to the best of our knowledge, none involve a unicellular or protist species. It means we know virtually nothing about the phenomenon of HTT in these organisms. So I don’t see what else we could have possibly said about these organisms in our review. Clearly a lot of work remains to be done in this area. As I mentioned above, we are actively working on characterizing TEs in several, deeply diverged protist lineages.

In several places in our paper, we warn against the temptation to generalize the patterns inferred in this review based on a very limited sample of (mostly multicellular) eukaryotic species. For example, page 540, we wrote:

” Currently, the taxonomic sampling bias of whole-genome sequencing projects
among eukaryotes leans strongly towards animals
and fungi, making it difficult to infer patterns among
under-represented groups, for example unicellular eukaryotes
and plants. Thus, it is important to be cautious when
making generalizations about patterns of HTT based on
1.5 million extant eukaryotes become
available.”

I think we couldn’t be more explicit and more cautious. Don’t you think?

Agreed. I think this is the most explicit and cautious statement in the whole paper. Too bad it is buried on page four of an eight page paper with the title "Promiscuous DNA: horizontal transfer of transposable elements and why it matters for eukaryotic evolution". It hardly counters such statements, such as "We argue that the introduction of transposable elements by horizontal transfer in eukaryotic genomes has been a major force propelling genomic variation and biological innovation.", which can be found in the abstract.

For all these reasons, I found your post completely unfair and misleading. I am very sensitive to the issue of ignorance of the general public and lack of attention given to the wonderful diversity of unicellular eukaryotes, and I found myself regularly advocating and educating my students and colleagues about it.

In my opinion, it was poor judgment from your part to use our paper to illustrate a real and serious issue. It is not serving the cause eloquently. I hope you’ll realize this when you read it again, this time more carefully. And I hope people reading your blog will do the same.

I appreciate the sentiment here and I think it is great that Cedric is both aware of non-animal eukaryotic diversity and is interested in preachin' the word to others, but this was hardly a poor paper to use to illustrate my point. The intent and background of a author is not part of a paper, just the words on the page. If you want to talk about a large group of organisms, then do it, but talking the talk and then pulling a bait-`n-switch on the reader is not the same thing. This is ESPECIALLY true when the objective is hyping the title at the expense of accuracy. It is important to 'sell' one's papers in this day and age, but I think it is more constructive to sell them on their content and not default to the New York Post Handbook on Headlines.

3 responses so far

Is panic on the TT a good thing?

Aug 25 2010 Published by under [Education&Careers], Etc

I am catching up a little here in bloggy land and came across a post by Gerty-Z about anxiety in her first couple 'o months running her own show. Odyssey followed it up with a good response from the perspective of someone with both familiarity with the situation and enough distance to have perspective.

What caught my attention, however, was the comment by GMP (can't seem to link to comments over there at LabSpaces) that says:

What I am trying to say is -- panic away! Panic is good - it means you are on the right track; embrace it. You know what they say: if you're not scared, it means you are not paying attention!

On the surface, I can see where this makes some sense in a tongue-in-cheek kinda way, but I don't think I buy into the idea. Panic, to me, has an element of out-of-control to it. It is fine to do once in a while when things get overwhelming, but if you're in a state on panic most of the time, or even a good amount of the time, then you're likely fucked.

Have there been times in the last two years when I have internally freaked out. Hell yeah. Have there been times when I thought "what the fuck am I doing here?" Yes. We all go through this kind of stuff because, as I keep saying, this job is totally different than the training for this job. Even a good postdoc mentor can't adequately prep you for the reality of everything you have to juggle as the leader of a lab with a novel research focus / teacher of undergrads / new person who is eligible and naive enough for the departmental service no one else wants to do / lab equipment orderer / manager of people / accountant / money getter / navigator of an unfamiliar political landscape / etc x 3.8 fucking bazillion.

The reality of all this will cause anxiety. Managing the lab finances and personnel cost, alone, is enough to keep one up at night. Trust me. You're now immediately responsible for others. This ain't a solo ship anymore.

BUT, panic isn't going to help anything. It is allowed on occasion and in the presence of senior people to give you advice, but as CPP has eloquently put in the context of frustration in the lab, but I think the same goes for panic:

The moment that your trainees begin to think that you are not calm and confident in your expertise and in your ability to make your lab a success, they will lose trust in you as a mentor. And once they lose trust in you, you are completely fucked.

Trainees are like dogs: they can smell fear. If you are afraid of failure, and your trainees smell your fear, you are fucked.

So, while it is normal the freak out once in a while and healthy to be thinking a few moves ahead, with about 4 back-up plans, when the panic comes it is important to take a step back and try and come up with a plan to attack the most important things first. When you have the keys to your own lab, you're a leader by default and no one is going to follow someone in panic.

With only 2 months under her belt, I'm sure that Gerty-Z is overwhelmed and feeling like she can't keep her head above water. I know I did. But remember that everyone hits that stage (often more than once), the key is being able to reign yourself in and use the motivation to something productive. If writing isn't working, try something else, but don't allow yourself to just flail about. Things like mentorships, collaborations, students, etc., will fall into place if you just keep yourself moving forward.

At least, that's how I've survived two years so far.

12 responses so far

Tea is for the weak

Aug 24 2010 Published by under Etc

When it comes to hot caffeinated beverages, there are two types of people, those who drink coffee and those who prefer tea. I am a coffee drinker. While at work I typically drink roughly a liter of coffee a day. I enjoy coffee a lot, the darker the better. Oddly, I can also drop coffee without feeling the typical side-effects that most people report - headache, grumpiness, etc. This is lucky for me this week.

Where I am staying this week is a place where tea rules the day. Sad really, to see so many people so misguided. But after 5 days coffee free, I decided yesterday that I would like to find some coffee and several of my colleagues traveling with me agreed. So, I went to the hotel lobby and inquired about the nearest coffee shop.

"I believe they sell coffee at the gas station down the street" was the reply.

Struggling to contain disbelief about the recommendation, I tried to re-define what I was looking for, but only got a blank stare in return. So, to The Internet! A brief search yielded two places nearby that purported to hawk the caffeine delivery system I was searching for. But Google is apparently behind on updates for local businesses, because driving to these locations resulted in locating the local cement quarry and a neighborhood devoid of any businesses. Frustrated, I tried a bakery. Closed.

With the errand now taking far too long, what's a guy to do? The gas station cashier was very nice in helping me wrap my four cups to avoid a spill on the ride home. It was better than water dressed in brown. Barely.

That said, I don't know that I can file this under 'roughing it in the field'. More like, 'yes, I can complain even in a nice location'.

20 responses so far

An old friend

Aug 23 2010 Published by under [Biology&Environment], Etc

The trip I am on right now is a work trip to a place I have been many times. I know this area well and often visit the same locations while here that I have been to numerous times previous. One spot, in particular, is a special place for me. It is a place that I have often found myself distracted from the work I am doing by its incredible beauty. A place I have brought my family when we have visited together. A place I have always wanted to share with others.

In 2006, however, things changed. A combination of a rare natural event and human encroachment resulted in a massive and unfortunate incident that wiped this place clean of what had made it so impressive before. I was there not a few weeks later and the devastation was total. It was unrecognizable.

Fast forward a few years and there have been small signs of recovery, but the place remains a shell of what it was pre-2006. It's like visiting a sick friend who you keep hoping will get better, but the prognosis doesn't look good. Maybe it will just take time, but the likelihood of a complete recovery seems low.

I was at this place yesterday and it was in the worse shape I had seen it since arriving after the mess in 2006. Maybe it is the season or the weather or the year, but seeing this once-vibrant place still in such tough shape, and looking worse than last year, made me feel a little sick. Hopefully this is just a bump in the road on the timescale that is relevant for this kind of place, but it doesn't make me feel much better.

Get better old friend.

Fuck, sigh.

4 responses so far

If you're reading this, I'm already gone

Aug 17 2010 Published by under Uncategorized

Sorry folks, as much as I would love to stick around for the data release discussion fun, some of us have to do work. I'm off on a trip for a week with sporadic access to internet, but I'll try and check in. Don't trash the place while I'm gone (Yes, I'm looking at you Odyssey).

14 responses so far

Data release (blogwar or pillow fight?)

Since Mike the Mad Biologist appears to have called me out on my post about data release just before he entered the witness relocation program, I thought I might articulate my position a little more clearly. Maybe he'll join in the conversation as well, we'll see.

IMO, the sequencing centers and those making decisions on how data should be released are A) considering only one model of scientist (the NIH R01 investigator) and B) making these decisions based on 5 years ago.

As for A, I doubt that is going to change based on the money behind the R01 game. That said, I think it is useful to be aware that not everyone is looking to resequence the same shit over and over looking for SNPs that might mean something. There is quite a bit of work being put into sequencing novel things that are not at all related to other sequenced organisms. It is these projects that face the biggest challenges when it comes to instant data release, as I talked about on Mike's blog. They require more work to assemble, more work to annotate and are arguably more interesting to potential data poachers. If you have, say, a critical cell adhesion protein you have identified in sloths and sea squirts that you think has to do with the development of multicellularity, how fast are you going to go look in the kelp genome, knowing it is an independent evolutionary line of multicellularity? If someone sequences yet another species of sloth, you wouldn't be as interested because you already have the gene spanning sea squirts to sloths, right?

With regards to B, I don't have to tell you that genomics is rapidly changing to the point where sequencing a genome in one's own lab is not only feasible, but in some cases easier than farming it out. If you talk to most people involved in genome sequencing at a major center, you might come away with a different definition of 'free data' than you had previously.

Getting sequencing done for 'free' is a big deal, but not as big as it used to be. At the same time, there are more capable bioinformatics people to work on this stuff than a few years ago, the software is better and the data are better. On the flip side, there are more people combing through geneomic datasets to advance their own work than ever before. There are labs that literally wait at the end of genome center pipelines waiting for data in the same way that fish swarm sewage outfalls. What then, is the motivation for people to have sequencing projects done by publicly funded centers where the data are dumped to the public right away? What will it be in two years?

This brings us back to the comment by Bob Carpenter on my previous post

I’m confused. If the data’s publicly available, what’s unscrupulous about using it? Is it just people violating licensing terms like embargoes?

The first question gets to the root of my point. People feel like anything that is public is free to use, and maybe they should. But how would you feel as the researcher who assembled a group of researchers from the community, put a proposal together, drummed up support from the community outside of your research team, produced and purified the sample to be sequenced (which is not exactly just using a Sigma kit in a LOT of cases), dealt with the administration issues that crop up along the way, pushed the project through (another aspect woefully under appreciated) the center, got your research community together once they data were in hand to make sense of it all and herded the cats to get the paper together? Would you feel some ownership, even if it was public dollars that funded the project?

Now what if you submitted the manuscript and then opened your copy of Science and saw the major finding that you centered the genome paper around has been plucked out by another group and publish in isolation? Would you say. "well, the data’s publicly available, what’s unscrupulous about using it?"

It is really easy for people to say, "well fuck those slow bastards, they should have published faster!" but let's couch this in the reality of the changing technology. If your choice is to have the sequencing done for free, but risk losing it right off the machine, OR to do it with your own funds (>$40,000) and have exclusive right to it until the paper is published, what are you going to choose? You can draw the line regarding big and small centers or projects all you want, but it is becoming increasingly fuzzy.

This is all to get back to my point that if major sequencing centers want to stay ahead of the curve, they have to have policies that are going to encourage, not discourage, investigators to use them. If you think the technology won't be available in 5 years to sequence a genome for a couple grand in individual labs, then you have your head in the sand. Without constant re-evaluation of their mission and how they need to serve the community, that kind of competition could be the meteor that forces the shift from big centers who need tons of resources to survive, to small furry independents scattered all over the world.

25 responses so far

Eukaryotes: more than just animals

Aug 16 2010 Published by under [Biology&Environment]

If there is one thing that bothers the ever-loving-shit out of YHN, it is when people make broad claims about "the evolution of eukaryotes", when they really mean "the evolution of animals", one of the least interesting branches of the Tree of Life from an evolutionary standpoint. Every time I hear someone say "And we looked at this phenomenon across all eukaryotes, from humans all the way to flies!" I want to stand up in their presentation and whip my fucking laptop at them. Maybe when I need to upgrade...

In presentations it just show the stupidity of the lab and the animocentrism that most people use to view the world. In print, however, it had to get through reviewers and the editor. So, when I got the feed from the most recent issue of Trends in Ecology and Evolution and clicked on the link for the paper by Schaack et al., I expected some interesting stuff. The title of the paper is, afterall, "Promiscuous DNA: horizontal transfer of transposable elements and why it matters for eukaryotic evolution".  Sounds pretty cool, right? I like interesting eukaryote-wide phenomena that may have an impact of evolution of organisms. Isn't that what everyone reads to their kids at night?

But by the time I got through the first section, I knew this horse had blinders on and has galloping full speed to metazoatown. Ironically, the second paper they cite is a solid review by Keeling & Palmer on eukaryotic gene transfer which brings up several cases of movement of transposable elements among non-animal organisms (in fact they dedicate Box 4 to this very issue), but Sclaack et al only use this reference to state that gene transfer between eukeryotes "has remained more obscure" than that between prokaryotes. Um, true, but there is a shit-ton of data on the topic, much of it covered well in the Keeling and Palmer paper. It is usually a good idea to read more than the abstract when citing a paper.

In fact we know a LOT about the transfer of both genes and, to a lesser extent, non-coding DNA between eukaryotes. It happens often and if you search for the terms HGT or LGT and eukaryotes you get over 450 hits. Considering our understanding of this phenomenon has really emerged only as genome sequencing has expanded relatively recently, that's a decent number of papers.

But wait, PLS, you might say. The paper is about transposable elements, not all gene transfer! True, but not only has there been a significant amount of non-animal work done in this realm as well, the sheer number of genomes that have been sequenced from non-animal organisms in the last 5 years has yielded more than enough data to examine this question across eukaryotes.  And no, throwing in one or two references from the land plant lit does not get one of the hook.

So why does such a small portion of an entire group continually serve in the eyes of many as a proxy for the whole? It is, IMO, just pure blind ignorance and unwillingness to think beyond the obvious. If I wrote an article claiming to have discovered a critical force for the evolution of vegetables and then only discussed a couple of examples from squash, do you think that would get reviewed well? How fast would I get reviews back blasting me for over-stating my evidence? Would it even get past the editor's desk?

If you haven't opened a biology textbook since the 1970's, consider this a head's up that the whole 3 kingdom 'animals, plants, fungi' (in that order of importance) classification is long dead and things are much more complicated these days. In fact, we know a tremendous amount about the world around us and how insignificant a contribution animals make to the diversity of organisms on the planet. Perhaps it is time to stop learning about biodiversity from the Discovery Channel and introduce yourself to the other 95% of eukaryotes.

References:

Schaack, S., Gilbert, C., & Feschotte, C. (2010) Promiscuous DNA: horizontal transfer of transposable elements and why it matters for eukaryotic evolution. Trends Ecol. Evol. (25) 537-546.

Keeling, P., Palmer, J. (2008) Horizontal gene transfer in eukaryotic evolution. Nat. Rev. Genet. (9) 605-618.

19 responses so far

Older posts »