Scientific dissention: shouldn't we all be nice?

In case you've been living under a scientific rock, you might have heard about the Science article (PMID:21127214) reporting the discovery of a prokaryote that can incorporate arsenic into its DNA in place of phosphorus. Obviously this hit the press in a big way and was picked up by a shit ton of bloggers, who spread the word. Given that the science was done in a NASA lab, there was quite a bit discussed about the implications of this finding for life on other planets.

Then people started looking hard at the science. Rut-roh, Elroy.

One of the first and most vocal was blogger and UBC microbiologist Rosie Redfield, who jumped all over the paper. As Carl Zimmer reported at The Slate, many others agree with Rosie's assessment.

While I find the subject of the paper interesting and the debate more so, what really caught my attention was the scientific communities reaction to Rosie's critique.

Carl Zimmer contacted two of the authors of the paper about the apparent flaws in their study, and they circled the wagons:

"We cannot indiscriminately wade into a media forum for debate at this time," declared senior author Ronald Oremland of the U.S. Geological Survey. "If we are wrong, then other scientists should be motivated to reproduce our findings. If we are right (and I am strongly convinced that we are) our competitors will agree and help to advance our understanding of this phenomenon. I am eager for them to do so."

"Any discourse will have to be peer-reviewed in the same manner as our paper was, and go through a vetting process so that all discussion is properly moderated," wrote Felisa Wolfe-Simon of the NASA Astrobiology Institute. "The items you are presenting do not represent the proper way to engage in a scientific discourse and we will not respond in this manner."

Wow. Really? Did I just slip into the 19th century by accident? Pass the watercress sandwich tray, please.

And what about the comments on Rosie's blog? While many were supportive, comment after comment after comment after comment after comment after comment after comment, hopped aboard the old "this isn't how scientific debate is done, this is unprofessional!" train. Choo-choo.

Dude. Fuck. Sigh.

This idea that science is somehow pure and fact-based alone, while being performed by scholars in tweed having "civil" conversations over tea makes me want to poke my eyes out with a rusty nail. Did Rosie's post relay her mistrust of the data and the motivation behind some of its gathering? Absolutely. But do her arguments against the data (and those brought up in the comment section) raise all sorts of red flags that we have seen from other cases the ended with a retraction of a high profile paper? Hell yes.

I don't know whether some of the fairly massive missteps in the data acquisition were a case of people being blinded by their own "discovery" or pushing out the answer they wanted, but there is no question that these data are premature, at best. However, both the paper itself and the way it has been presented at conferences, indicate very little of this preliminary vibe. Guess what? If you rush dodgey data out there and claim you have found a novel life form, you're going to hear it from the masses.

To my mind, this is a "good thing" and evidence that the speed of information transfer is working to make science better, not less professional.

27 responses so far

  • Jo says:

    What have watercress, tweed and tea got to do with dodgy science? You seem to be implying that the British establishment is associated with 'bad' and all that is good comes from hipster bloggers.

  • Coturnix says:

    This is nothing new, just that the case is more prominent. The world is changing, but not everyone is moving at the same speed. The slow ones get hurt in the process, whine for a while, learn and speed up....

  • proflikesubstance says:

    Jo, you seem to be implying the British establishment is entirely made up of ritualized "high society".

  • Jo says:

    >>Jo, you seem to be implying the British establishment is entirely made up of ritualized “high society”.

    Not me. It's a stereotype we seem to have to live with.

  • Bashir says:

    I'm confused. Is the issue here that Rosie and others weren't nice in their critiques or simply that such cirques took place on the *gasp* internet.

    Is the content of what Rosie wrote any different than the back and forth that goes on in departmental brown bags or conferences? Plenty of that scientific discourse is already unpeerreviewed.

  • gc says:

    Actually, the 'unsupportive' comments you point to do make a valid point. It is not clear from the arguments that this is a case of willful deception. Fraud ('flim-flam') is different from not being sufficiently careful or being incompetent.
    While it'd be great if the authors could answer the criticisms online, I think I can understand why they may refuse to engage in blog discussions which are not well-moderated and where their integrity is questioned without evidence .

  • proflikesubstance says:

    GC, this is what Rosie wrote:

    Lots of flim-flam, but very little reliable information....There's a difference between controls done to genuinely test your hypothesis and those done when you just want to show that your hypothesis is true. The authors have done some of the latter, but not the former....I don't know whether the authors are just bad scientists or whether they're unscrupulously pushing NASA's 'There's life in outer space!' agenda.

    Flim-flam is not synonymous with fraud, last time I checked. She does, however, accuse them of over interpretation and possible misrepresentation. I don't disagree with these statements and the reaction from the comments I pointed to is to take her to task for calling this out. Shall we shoot the messenger?

    This is a big story, so most people would be sure to cover their ass and make sure they are right. Leaving out the proper controls and appearing to selectively not do certain experiments does not make for a reliable piece of science.

  • tideliar says:

    Larry Moran (http://sandwalk.blogspot.com/) is wading into and fighting the good fight, which is great to see. WIsh I had tie to comment too :/

  • Hermitage says:

    So they were happy to throw their hat into the media ring to hawk their awesomeness and oversell the Incredible Potential of their findings, but then want to hide behind peer review when they get called out on their work? Absolutely comical. An unwillingness to address holes in their protocols points to the fact they rushed the cake to the viewing case without cooking it all the way in the middle.

  • gc says:

    PLS, I agree that it is great that she raised those points...but it just gets a bit too muddled when you see some personal jabs.

    Also, since those microbes are available now, all their claims can be tested.

  • proflikesubstance says:

    GC - As Hermitage points out, the game works both ways. When a paper comes out (and is hyped in a huge way by NASA, complete with full press conference) that is clearly missing very important controls and experiments, why shouldn't we question the motives of the authors? In fact, I think it is critical to do so.

    There is no way they expected this work to slip under the radar, in fact they made sure it didn't. Given the way it was sold and the pretty massive holes, I'm curious what your alternative is? Before you answer, do check out the post by Larry Moran, that Tideliar links to above and see if you can reconcile the video with the results.

  • Anon says:

    I cannot judge the quality of the data as my field it too far removed from the field relevant here, but I agree with the authors response you cited above. In my field at least, if you have an issue with a paper, you write a comment article; the authors are allowed to respond, and then both the comment and the rebuttal are peer reviewed before publication. I don't think the authors are, as you say, going back to the 19th century. They are under no obligation to entertain everyone and anyone on the internet who happens to have an opinion about their work.

  • Bob O'H says:

    Rosie Redfield wrote this in her post too:

    (Might they have not presented assays using properly purified (washed) DNA because these turned out to not have any arsenic? Am I just paranoid?)

    which is suggesting fraud.

    I'm not sure the critics of her post should be dismissed out of hand. The context is a post about the science of the GFAJ paper, and one which uses formal scientific language. If the science is what matters then the motives behind doing the science shouldn't be important, and as professional scientists we should be able to separate the two out when we are acting as scientists. The post was clearly intended to be read as a scientific post, so why shouldn't it be criticised by that standard?

  • tideliar says:

    Bob - Because it isn't peer reviewed?

  • antipodean says:

    Maybe Rosie should hold a press conference then? They'd have to respond to that.

  • antipodean says:

    I think it has been peer reviewed. It's just not in a 'peer-reviewed journal"

  • Hermitage says:

    Unless they plan on never going to a conference, demanding all rebuttals go through a peer-review process before they will answer it sounds, at best, extraordinarily strange.

  • gerty-z says:

    I guess it is fine if the authors don't want to respond to every comment on the internet. But, it seems very disingenuous to claim that scientific discourse is always through the process of peer review. It is just lame to bother saying anything if you are just going to say "i'm not gonna talk about it here". I think that it would be great to hear their response to some of these criticisms. Many of the points raised seem fairly straightforward, and they can either defend their work or not. Perhaps dueling press conferences??

  • Bob O'H says:

    I don't see that that should be the criterion that should be applied. Think about it as a question of hats. When we write a scientific paper or presenting work at a conference, we're wearing a Scientist hat. The line that NASA is pushing is that they will only respond to Scientists (as recognised by their hats), and only in peer reviewed literature. i.e. their philosophy is that Scientists only communicate in the peer reviewed literature (and possibly conferences).

    Rosie Redfield wrote her blog post with her Scientist hat on: that's clear from the way language is used. But Scientists should only be concerned with the science, not about the people doing the science (a convention that I agree with, BTW, as it prevents science - or perhaps Science - from descending into mud slinging).

    I guess the conflict is because when one is writing on a blog, one has a Blogger hat on. This can mean different things to different people, but it generally implies a more informal approach to writing, with all sorts of asides, jokes, insults, and the use of the sort of foul-mouthed language that, in public, would quickly attract a stern look from your betters (I do hope you're not the sort of chap who indulges in this sort of profanity, Tideliar). the question is whether someone writing with a Blogger hat on can also write with their Scientist hat on, and if so should how far should they go to moderate their Scientist behaviour. I don't know what the answer is, but it should depend on who you're trying to communicate to, and if you want to write for people who expect Scientists to write about Science, not people, then it's probably better to stick to that.

    I guess things will change over time, as we (as a society) work out how blogs relate to the formal Scientific literature.

  • proflikesubstance says:

    Bob, I don't think your false dichotomy makes any sense. You'll note that Rosie has written a Technical Comment to Science outlining her concerns. I'm sure the fact that she posted it on her blog will be of great concern to you, but here's where the false dichotomy lies. We can be both bloggers AND scientists, wearing both hats at once.

    I agree with you that scientific discussion needs to occur in the realm that is most familiar to the most scientists, but this example is an excellent demonstration of how powerful the bloggosphere can be in participating in that discussion. Rosie posted her initial response and many people joined in. Through the discussion several novel points were brought up and clarified, further honing the argument. The end result was a more clearly defined response to the journal.

    Just because the "right words" (as defined by who exactly?) were not used in the initial post doesn't mean shit. I'm sure the right words would also not be used if this discussion happened at a conference or in the pub afterwards. The point is that the discussion happened on a broad level and contributed to the "formal" response.

    I do not think the authors are under any obligation to respond to every internet criticism, but after running their media parade through town, they closed and locked the doors the second anyone asked a question. If they were in another public sphere (sports, politics, etc.) the media would be ALL. OVER. THEM. Instead, they trot out the tired excuse of "we only speak in the literature" as the confetti gets swept up around them.

    The media knife cuts both ways and I don't see this ending well for them, or NASA.

  • Natalie says:

    I find it amusing over on the original post Sanford May @3:01 AM suggests Rosie should temper her criticisms and use a different venue because she has clout in the field, yet over here Anon @3:52 believes the authors of the paper needn't engage "everyone and anyone on the internet who happens to have an opinion about their work."

    This issue relates to science outreach/engaging the public/making science accessible IMHO. To insist that reproach happen only through traditional journal-based editorials alienates those who think we just sit in our ivory towers all day. Heaven forbid we go slumming it in the blog-o-sphere like the unwashed masses.*eyeroll*

    The internet makes the whole fucking world a democracy of dissent. Buck up.

  • Anon from 3:52 above says:

    To insist that reproach happen only through traditional journal-based editorials alienates those who think we just sit in our ivory towers all day. Heaven forbid we go slumming it in the blog-o-sphere like the unwashed masses.*eyeroll*

    One of the most dramatic counterarguments to this would be a scientist who specializes in evolution having to defend it against every moronic crackpot who happens to have an "opinion" that "it's just a theory". It's a democracy, after all? Their "opinion" counts as well as that of a scientist, right? Fuck no.

    Science is not a democracy. There are things that are wrong and those that are right. If you want to prove something is wrong, prove it as a scientist -- there are conferences and peer reviewed papers where you can make your case.

    Just because people have an opinion about what you do/say/are doesn't mean squat. Does not mean they have a point, does not mean you have to care or to engage. Sure, the blogosphere is there with dissenting voices. So what? I am sorry, but nobody owes the blogosphere anything.

    Making science accessible should best be done by people of techical background who also have training in education and/or scientific popular writing, and ideally have no bone to pick (this appears not to be the case with Rosie here). Beating over primary findings on a blog does not help the public -- the public cannot appreciate the details of the work, is not qualified to decide if the methods were correct or not; but the public will remember that someone implied NASA people were frauds. Even if they are not, the damage is done -- to those people as well as to other NASA scientists and scientists at large.

  • Natalie says:

    Dramatic counter arguments aside, what I'm pointing out here is that Rosie is definitely NOT a moronic crackpot. She blogs under her own name, which I think is paramount when deciding who should be engaged online (to save energies at least). I'm not suggesting the authors wade through the comment section of her post to reply, but a nicely crafted statement rebutting some of her arguments sent directly to her (to be posted online) would be fan-fucking-tastic. And immediate.

    (I guess I'm dating myself here as an internet kid. I don't WANT to wait months to hear their side of this story)

    I see science blogging as an extension of our poster sessions, of our conference talks, of our pub discussions. We clearly disagree about where and how science outreach happens. Requiring specialized outreach scientists to reach the public absolves us (me, you, PLS, Rosie) of our personal responsibility to do our part explaining the scientific process and findings to the public. This includes why I feel its OK to question results at all times using all platforms (ie "this is how it is suppose to work, don't worry").

  • chall says:

    I agree with the comments RR made on lots of the science, and some other stuff that I thought was odd that it wasn't mentioned in the article (maybe due to word contrain and space most likely?) but some of the statements might have been a bit too "attacking". That said, I'm overly cautious when I deliver critique and try and be very clear on the "problem" rather than "why it might not have happened" (as with the log scale discussion on the growth). My main problem with the growth curve was that I didn't think it seemed significant enough and that the time frame they looked at seemed a bit short.... therefore, I had more "threorethical hypothesis problems" with the research.

    As for the "we don't want to address any of the comments"... well, I can certainly see that most communications should imho be published as a "letter to editor" or "technical comment" or "correspondance" since that means you are invoking your reputaion as a scientist and are happy for people to see your problems with the research.

    However, as in this case, when you go all out on a press conference, and state a LOT of new and famous stuff (especially since most of us know that once you go into the media "finding eternal life" it doesn't really go away even if the article you wrote actually just found "how to become a few years older" - you're still getting the "eternal life fame factor to your name") I think it is reasonable to accept that people will look at the press release, then actually read you paper and then address something they might not agree with. Hyperbole and all. And if you are aware that the press release is a bit too hyped, maybe address the concerns and explain why you did as you did. After all, many of our articles don't permit the space to explain exactly WHY we did the things we did, only THAT we did it and sort-of-how (don't get me started on M&M sections).

  • [...] Arsenic? and NASA arsenic story – let’s lay off the personal attacks on all sides and Scientific dissention: shouldn’t we all be nice? and Arsenic about face and My Letter to Science and DNA, Phosphorus, and Arsenic and NASA can’t [...]

  • [...] “Prof-like Substance”. Dec 8 2010. Scientific dissention: shouldn’t we all be nice? The Spandrel Shop. http://scientopia.org/blogs/proflikesubstance/2010/12/08/scientific-dissention-shouldnt-we-all-be-ni... [...]

Leave a Reply


1 + eight =