Tag: Carl Zimmer

"Scientific Pride and Prejudice" in the @nytimes makes claims about sciences not using evidence correctly; alas no evidence presented

Well, I guess I can say I was not pleased to see this tweet from Carl Zimmer.

//platform.twitter.com/widgets.js It is not that I have a problem with what Carl wrote. It is just that, then I went and read the article he referred to: Scientific Pride and Prejudice in the New York Times By Michael Suk-Young Chwe. And it just did not make me happy. I reread it. Again and again. And I was still unhappy.

What bugs me about this article? Well, alas, a lot. The general gist of the article is that “natural” scientists are not aware enough of how their own preconceptions might bias their work. And furthermore that literary criticism is the place to look for such self-awareness. Well, interesting idea I guess but alas, the irony is, this essay presents no evidence that literary criticism does better with evidence than natural science. Below are some of the lines / comments in the article that I am skeptical of:

  • “Scientists now worry that many published scientific results are simply not true.”
  • “Scientists, eager to make striking new claims, focus only on evidence that supports their preconceptions. Psychologists call this “confirmation bias. We seek out information that confirms what we already believe. ”
    • This statement is misleading. Confirmation bias according to all definitions I could find is something more subtle. For example Oxford Dictionaries defines it as “the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories.” That is, it is a tendency – a leaning – a bias of sorts.
    • I would very much like to see evidence behind the much more extreme claim of this author that scientists focus “only on evidence that supports their preconceptions”. 
    • In my readings of actual research on confirmation bias I can find no evidence to this claim. For example see the following paper Confirmation bias: a ubiquitous phenomenon in many guises. which states:
    • As the term is used in this article and, I believe, generally by psychologists, confirmation bias connotes a less explicit, less consciously one-sided case-building process. It refers usually to unwitting selectivity in the acquisition and use of evidence. The line between deliberate selectivity in the use of evidence and unwitting molding of facts to fit hypotheses or beliefs is a difficult one to draw in practice, but the distinction is meaningful conceptually, and confirmation bias has more to do with the latter than with the former.

    • “Despite the popular belief that anything goes in literary criticism, the field has real standards of scholarly validity”
      • This is a red herring to me. I can find no evidence that their there is a popular belief that “anything goes” in literary criticism. So the author here sets a very low bar and then basically any presentation of standards is supposed to impress us.
    • “Rather, “the important thing is to be aware of one’s own bias.”
      • The author then goes on to discuss how those in the humanities are aware of the issues of confirmation bias and rather than trying to get rid of it, they just deal with it, as implied in the quote.
      • The author then writes “To deal with the problem of selective use of data, the scientific community must become self-aware and realize that it has a problem. In literary criticism, the question of how one’s arguments are influenced by one’s prejudgments has been a central methodological issue for decades.“
      • Again, this implies that scientists have not been thinking about this at all which is just wrong.
    • And then the author uses the Arsenic-life story as an example of how scientists suffer from “confirmation bias.”  If you do not know about the arsenic life story see here.  What is the evidence that this was “confirmation bias“?.  I think more likely this was a case of purposeful misleading, overhyping, and bad science.  
    • Then the author gives as an example of how science actually is prone to confirmation bias by presenting a discussion of Robert Millikan’s notebooks in relation to a classic “oil drop” experiment.  Apparently, these notebooks show that the experiments got better and better over time and closer to the truth.  And in the notebooks Millikan annotated them with things like “Best yet – Beauty – Publish”.  And then the author concludes this means “In other words, Millikan excluded the data that seemed erroneous and included data that he liked, embracing his own confirmation bias.”  I don’t see evidence that this is confirmation bias.  I think better examples of confirmation bias would be cases where we have now concluded the research conclusions were wrong.  But instead, Millikan was and still is as far as I know, considered to have been correct.  He won the Nobel Prize in 1923 for his work.  Yes, there has been some criticism of his work but as far as I can tell, there is no evidence that he had confirmation bias. 
    • I am going to skip commenting on the game theory claims in this article.
    • Then the author writes “Perhaps because of its self-awareness about what Austen would call the “whims and caprices” of human reasoning, the field of psychology has been most aggressive in dealing with doubts about the validity of its research.”  Again – what is the evidence for this? Is there any evidence that the field of psychology is somehow different?
    I could go on and on.  But I won’t.  I will just say one thing.  I find it disappointing and incredibly ironic that an article that makes claims about how some fields deal better with evidence and conformation bias than other fields does not present any actual evidence to back up its claims.  And many of the claims pretty clearly run counter to available evidence.

    UPDATE 9:20 AM 2/2/2014: Storify of discussions on Twitter

    //storify.com/phylogenomics/discussion-of-scientific-pride-and-prejudice-and-m/embed//storify.com/phylogenomics/discussion-of-scientific-pride-and-prejudice-and-m.js[View the story “Discussion of “Scientific Pride and Prejudice” and my response to it” on Storify]
    ——–
    This is from the “Tree of Life Blog”
    of Jonathan Eisen, an evolutionary biologist and Open Access advocate
    at the University of California, Davis. For short updates, follow me on Twitter.

    ——–

    Video of Carl Zimmer’s talk from the DOE-Joint Genome Institute User meeting

    I really love this world of sharing and openness in science.  Here is a video of Carl Zimmer’s Keynote talk from the DOE Joint Genome Institute User meeting.

    I note – before his talk I took Carl out to lunch.  Since of course I cannot do anything in a standard way, I bought some sandwiches and some drinks and snacks and took him to the top of Mt. Diablo which is nearby Walnut Creek.  Alas, it took a LONG time to drive up to the top and it was very very windy.  And as I started to get a bit queasy from all the turns I think Carl probably was wondering what I was doing.  But the view from the top was nice (even though the firggin’ visitor center and their nice viewing area was closed).  We did see some snow which was also nice.

    And then we headed back down, down, down, down, down.  So- I am telling you this because we got back about 1.5 hours before his talk so he did not have much time to recover from the long and windy road.  And yet, I loved his talk.

    For his talk I took “visual” notes using my iPad.  I made these using Notability and then it exports as a PDF which was awkward and then I took that an converted to JPGs since Blogger won’t deal with PDFs.

    Hope you enjoy the notes.

    ——–
    This is from the “Tree of Life Blog”
    of Jonathan Eisen, an evolutionary biologist and Open Access advocate
    at the University of California, Davis. For short updates, follow me on Twitter.

    ——–

    Carl Zimmer on "Who Owns Your Microbes"?

    There was an interesting piece by Carl Zimmer in the New York Times a few days ago: Our Microbiomes, Ourselves – NYTimes.com
    In the piece Zimmer discusses the issue of who owns your microbiome. This can be considered an extension of the concept of “Who owns your cells?” such as has been discussed in the context of Rebecca Skloot’s HELA book.
    My favorite line(s):

    Monitoring the bacteria flushed into the sewer system of a town, for instance, might reveal a lot about the entire town’s health. But a regulation requiring permission from every resident of the town would stop the study dead in its tracks

    Personally I think none of us own our microbes – since we get them from the world around us and likely share them with millions of others. It would be akin to saying we own genes found in all humans. But there very well may be some person specific alleles in microbes that could in a way be akin to person specific cell lines. Not sure.

    Anyway – I think I am going to name all my microbes as a first step in protecting my rights to them …

    ——–
    This is from the “Tree of Life Blog”
    of Jonathan Eisen, an evolutionary biologist and Open Access advocate
    at the University of California, Davis. For short updates, follow me on Twitter.

    ——–

    Bacteria & archaea don’t get no respect from interesting but flawed #PLoSBio paper on # of species on the planet

    ResearchBlogging.org

    Uggh. Double uggh. No no. My first blog quadruple uggh. There is an interesting new paper in PLoS Biology published today. Entitled “How many Species Are There on Earth and in the Ocean?” PLoS Biol 9(8): e1001127 – it is by Camilo Mora, Derek Tittensor, Sina Adl, Alastair Simpson and Boris Worm. It is accompanied by a commentary by none other than Robert May, one of the greatest Ecologists of all time: PLoS Biology: Why Worry about How Many Species and Their Loss?

    I note – I found out about this paper from Carl Zimmer who asked me if I had any comments.  Boy did I.  And Zimmer has a New York Times article today discussing the paper: How Many Species on Earth? It’s Tricky.  Here are my thoughts that I wrote down without seeing Carl’s article, which I will look at in a minute.

    The new paper takes a novel approach to estimating the number of species. I would summarize it but May does a pretty good job:
    “Mora et al. [4] offer an interesting new approach to estimating the total number of distinct eukaryotic species alive on earth today. They begin with an excellent survey of the wide variety of previous estimates, which give a range of different numbers in the broad interval 3 to 100 million species”
    ….
    “Mora et al.’s imaginative new approach begins by looking at the hierarchy of taxonomic categories, from the details of species and genera, through orders and classes, to phyla and kingdoms. They documented the fact that for eukaryotes, the higher taxonomic categories are “much more completely described than lower levels”, which in retrospect is perhaps not surprising. They also show that, within well-known taxonomic groups, the relative numbers of species assigned to phylum, class, order, family, genus, and species follow consistent patterns. If one assumes these predictable patterns also hold for less well-studied groups, the more secure information about phyla and class can be used to estimate the total number of distinct species within a given group.”
    The approach is novel and shows what appears to be some promise and robustness for certain multicellular eukaryotes. For example, analysis of animals shows a reasonable leveling off for many taxonomic levels:

    Figure 1. Predicting the global number of species in Animalia from their higher taxonomy. (A–F) The temporal accumulation of taxa (black lines) and the frequency of the multimodel fits to all starting years selected (graded colors). The horizontal dashed lines indicate the consensus asymptotic number of taxa, and the horizontal grey area its consensus standard error. (G) Relationship between the consensus asymptotic number of higher taxa and the numerical hierarchy of each taxonomic rank. Black circles represent the consensus asymptotes, green circles the catalogued number of taxa, and the box at the species level indicates the 95% confidence interval around the predicted number of species (see Materials and Methods).
    From Mora C, Tittensor DP, Adl S, Simpson AGB, Worm B (2011) How Many Species Are There on Earth and in the Ocean? PLoS Biol 9(8): e1001127. doi:10.1371/journal.pbio.1001127
    They also do a decent job of testing their use of higher taxon discovery to estimate number of species.  Figure 2 shows this pretty well.

    Figure 2. Validating the higher taxon approach. We compared the number of species estimated from the higher taxon approach implemented here to the known number of species in relatively well-studied taxonomic groups as derived from published sources [37]. We also used estimations from multimodel averaging from species accumulation curves for taxa with near-complete inventories. Vertical lines indicate the range of variation in the number of species from different sources. The dotted line indicates the 1∶1 ratio. Note that published species numbers (y-axis values) are mostly derived from expert approximations for well-known groups; hence there is a possibility that those estimates are subject to biases arising from synonyms.

    So all seems hunky dory and pretty interesting.  That is, until we get to the bacteria and archaea.  For example, check out Table 2:

    Table 2. Currently catalogued and predicted total number of species on Earth and in the ocean.

    Their approach leads to an estimate of 455 ± 160 Archaea on Earth and 1 in the ocean.  Yes, one in the ocean.  Amazing.  Completely silly too.  Bacteria are a little better.  An estimate of 9,680 ± 3,470 on Earth and 1,,320 ±436 in the oceans.  Still completely silly.

    Now the authors do admit to some challenges with bacteria and archaea. For example:

    We also applied the approach to prokaryotes; unfortunately, the steady pace of description of taxa at all taxonomic ranks precluded the calculation of asymptotes for higher taxa (Figure S1). Thus, we used raw numbers of higher taxa (rather than asymptotic estimates) for prokaryotes, and as such our estimates represent only lower bounds on the diversity in this group. Our approach predicted a lower bound of ~10,100 species of prokaryotes, of which ~1,320 are marine. It is important to note that for prokaryotes, the species concept tolerates a much higher degree of genetic dissimilarity than in most eukaryotes [26],[27]; additionally, due to horizontal gene transfers among phylogenetic clades, species take longer to isolate in prokaryotes than in eukaryotes, and thus the former species are much older than the latter [26],[27]; as a result the number of described species of prokaryotes is small (only ~10,000 species are currently accepted).

    But this is not remotely good enough from my point of view. Their estimates of ~ 10,000 or so bacteria and archaea on the planet are so completely out of touch in my opinion that this calls into question the validity of their method for bacteria and archaea at all. 
    Now you may ask – why do I think this is out of touch. Well because reasonable estimates are more on the order or millions or hundreds of millions, not tens of thousands. To help people feel their way through the literature on this I have created a Mendeley group where I am posting some references worth checking out.

    I think it is definitely worth looking at those papers.  But just for the record, some quotes might be useful.  For example, Dan Dykhuizen writes

    we estimate that there are about 20,000 common species and 500,000 rare species in a small quantity of soil or about a half million species.

    And Curtis et al write:

    We are also able to speculate about diversity at a larger scale, thus the entire bacterial diversity of the sea may be unlikely to exceed 2 × 10^6, while a ton of soil could contain 4 × 10^6 different taxa.

    Are their estimates perfect?  No surely not.  But I think without a doubt the number of bacterial and archaeal species on the planet is in the range of millions upon millions upon millions.  10,000 is clearly not even close.  Sure, we do not all agree on what a bacterial or archaeal species is.  But with just about ANY definition I have heard, I think we would still count millions.

    Given how horribly horribly off their estimates are for bacteria and archaea, I think it would have been better to be more explicit in admitting that their method probably simply does not work for such taxa right now.  Instead, they took the approach of saying this is a “lower bound”.  Sure.  That is one way of dealing with this.  But that is like saying “Dinosaurs lived at least 500 years ago” or “There are at least 10 people living in New York City” or “Hiking the Appalachian Trail will take at least two days.”  Lower bounds are only useful when they provide some new insight.  This lower bound did not provide any.
    Mind you, I like the paper.  The parts on eukaryotes seem quite novel and useful.  But the parts of bacteria and archaea are painful.  Really really painful.
    Mora, C., Tittensor, D., Adl, S., Simpson, A., & Worm, B. (2011). How Many Species Are There on Earth and in the Ocean? PLoS Biology, 9 (8) DOI: 10.1371/journal.pbio.1001127
    ——–
    This is from the “Tree of Life Blog”
    of Jonathan Eisen, an evolutionary biologist and Open Access advocate
    at the University of California, Davis. For short updates, follow me on Twitter.

    ——–

    More (you know you wanted it) on fecal transplants and the microbiome

    ResearchBlogging.org

    Image from
    I Heart Guts blog

    There is an interesting mini review in the Journal of Clinical Gastroenterology’s September issue that may be of interest to some out there. It is entitled “Fecal Bacteriotherapy, Fecal Transplant, and the Microbiome” by Martin Floch and well, the title is indicative of the article.

    Yes, the fecal transplant meme is here to stay. Sure, the cognoscenti already knew about fecal transplants. Perhaps they had read Tara Smith’s discussion of it in her Aetiology blog in 2007. Perhaps they had pondered it when they read the article from my lab on intestinal transplants. Perhaps they had seenthis discussion on MSNBC, or various other stories out there such asthis or this post from Angry by Choice. Or, maybe you just learned about it from Bora’s Carnival of Poop.

    But the meme on fecal transplants really spread and many may have first heard about fecal transplants from Carl Zimmer’s New York Times article a month or so ago “How microbes defend and define us“

    In the article Zimmer discussed how Dr. Alexander Khoruts used a fecal transplant to treat a woman with a persistent and severe Clostridium infection. And Zimmer discusses how, thought such transplants had been done before, this was the first time that the microbial community was carefully surveyed before and after. (Note, my favorite part of the article is this part, where my friend Janet Jansson describes her reaction:

    Two weeks after the transplant, the scientists analyzed the microbes again. Her husband’s microbes had taken over. “That community was able to function and cure her disease in a matter of days,” said Janet Jansson, a microbial ecologist at Lawrence Berkeley National Laboratory and a co-author of the paper. “I didn’t expect it to work. The project blew me away.”

    Anyway Zimmer’s article, as with many of his, garnered a lot of response and got many people discussing the poop on fecal transplants.

    Well, this issue of the Journal of Clinical Gastroenterology may now be the biggest pile of information about fecal transplants around. That is because, in addition to this little review mentioned above, there are in fact three articles in this issue relating to fecal transplant. Alas, most of you out there will probably only be able to read the review since the other articles are behind a pay wall.

    But the review is good. And I think this is not the last you will hear about this. (Though I note that, even though I think fecal transplants have some major potential, they seem to be being oversold a bit by many as some cure all — fodder for a future “Overselling the Microbiome Award” I am sure).

    I will end with this line from the review which raises some other issues about fecal transplants:

    Probably one of the major problems is to define how this therapy can become socially accepted. (Can you imagine the Food & Drug Administration discussion?)

    Floch, M. (2010). Fecal Bacteriotherapy, Fecal Transplant, and the Microbiome Journal of Clinical Gastroenterology, 44 (8), 529-530 DOI: 10.1097/MCG.0b013e3181e1d6e2

    Grehan, M., Borody, T., Leis, S., Campbell, J., Mitchell, H., & Wettstein, A. (2010). Durable Alteration of the Colonic Microbiota by the Administration of Donor Fecal Flora Journal of Clinical Gastroenterology, 44 (8), 551-561 DOI: 10.1097/MCG.0b013e3181e5d06b

    Khoruts, A., Dicksved, J., Jansson, J., & Sadowsky, M. (2009). Changes in the Composition of the Human Fecal Microbiome After Bacteriotherapy for Recurrent Clostridium Difficile-associated Diarrhea Journal of Clinical Gastroenterology DOI: 10.1097/MCG.0b013e3181c87e02

    Yoon, S., & Brandt, L. (2010). Treatment of Refractory/Recurrent C. difficile-associated Disease by Donated Stool Transplanted Via Colonoscopy Journal of Clinical Gastroenterology, 44 (8), 562-566 DOI: 10.1097/MCG.0b013e3181dac035

    ——–
    This is from the “Tree of Life Blog”
    of Jonathan Eisen, an evolutionary biologist and Open Access advocate
    at the University of California, Davis. For short updates, follow me on Twitter.

    ——–