Please make it stop – overselling the microbiome award for rugby, exercise, microbiome stories

Update added 11/2/14 – for all my posts on Overselling the Microbiome go here. 

Well, I think today’s lesson is, many people, including many scientists and science reporters, just do not get that there is a difference between correlation and causation.  I know – this is like beating a dead horse since many write about this issue.  But it just needs to be called out every time until it stops.  And today’s fun comes from stories and the original research articles about how exercise supposedly alters the gut microbiome.

I was pointed to this just a few minutes ago on Twitter:

//platform.twitter.com/widgets.js
In this Tweet Bernat Olle points to a “news” story in Medpage Today: Exercise Boosts Gut Microbiome Diversity by Kristina Fiore.   Well, so of course I started digging around.  And, not surprisingly, the study that this is based on shows absolutely no causal connection between exercise and the gut microbiome.  The study is in the journal “Gut”: Exercise and associated dietary extremes impact on gut microbial diversity.  And here is what they did:

  • They selected subjects – 40 “elite” rugby players.
  • They identified healthy male “controls” with similar age and size and from similar place. 
  • Then they collected faecal and blood samples from participants and did surveys about their nutrition and clinical data.
  • Among many measurements, they did 16S sequencing from the fecal samples
  • Then they did some bioinformatics and found differences between the rugby players and the controls in many features including microbiomes.
And amazingly, from this they report, in their abstract

The results provide evidence for a beneficial impact of exercise on gut microbiota diversity but also indicate that the relationship is complex and is related to accompanying dietary extremes.

The key part of this to me is 

The results provide evidence for a beneficial impact of exercise on gut microbiota diversity

For which they have no support.  They do not in any way show that exercise has ANY affect on the microbiota.  They show it is correlated to the microbiota.

And sadly there is a commentary on the article in the same issue of Gut that makes the same mistake.  Georgina Hold in The gut microbiota, dietary extremes and exercise writes:

The article is the first report that exercise increases gut microbiota richness/diversity and highlights that exercise is another important factor in the complex relationship among the host, host immunity and the microbiota.

No.  They did not show exercise increased gut microbiota diversity.  How can the difference between correlation and causation be missed in these articles?  Are these not even reviewed?  Sure – this is consistent with exercise affecting microbiomes but it is also consistent with rugby players having different diets and other behaviors.  There is a big difference between showing cause and effect and showing correlation.  For not distinguishing between correlation and causation regarding the rugby player microbiomes I am giving all involved here an “Overselling the Microbiome Award“.

Here is a microbiome theory I will leave you with.  I hypothesize that these papers, and all the other ones that oversell the microbiome, themselves cause major changes in the microbiome of many people.  Evidence for this?  Well, none yet.  But I have a correlation.  The correlation is, after reading these papers,  I feel sick to my stomach.  That must be proof right?


UPDATE 6/11/14

Author of the Medscape Medpage today article Kristina Fiore says she will update the article to more accurately reflect the science. See some of the thread below

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js


UPDATE 2: 6/11/14.

The press release from Gut associated with this paper contains many inaccurate statements.

Examples include:

  • Title: Exercise boosts diversity of gut bacteria
  • Text: Exercise boosts the diversity of the bacteria found in the gut, indicates the first study of its kind published online in the journal Gut.
Somewhat surprised that such mistakes would come from the journal itself.


UPDATE 3: 6/12/14.

Kritina Fiore has fixed the Medpage article.  Nice.


UPDATE 4: 6/12/14.

Science Magazine gets the causation vs. correlation issue wrong in their little news piece about this.  Yuck.

//platform.twitter.com/widgets.js


UPDATE 5: 6/12.

Alexandra Sifferlin has a good article about this at Time

//platform.twitter.com/widgets.js

//platform.twitter.com/widgets.js


UPDATE 6.

More accurate coverage by Claire O’Connell in the Irish Times Generally a good article here: Rugby players show good guts


UPDATE 7.

Popular Science messes it up too


UPDATE 8.

Keeping track of some of the Tweets about this on Storfy.


UPDATE 9 6/13/14.

NPR News Falls for the Hype

//platform.twitter.com/widgets.js


UPDATE 10: 6/13/14.

Just found another inaccurate claim in the original paper


UPDATE 11: 6/13/14

Oh FFS. Now I have found some articles reporting not only that exercise affects gut microbial diversity but that this is why exercise reduces obesity. See Exercise lowers obesity risk by stimulating diverse gut bacteria in the NVO News, for example.

 

Quote from the story:

A latest research suggests that exercise actually lowers obesity risk by stimulating diverse gut bacteria


UPDATE 12: 6/13/14

Fox News did better with the science (at least in their headline) than many other News Agencies (and much better than NPR).  They report “Exercise may lead to healthier gut bacteria“.

Just that word “May” makes me happy.  I know.  Low bar.  But I will take what I can get.

UPDATE 13: 6/13/14

Genome Web also is reporting on the story and on the “overselling” that was done.

UPDATE 13: 6/18/14

And now the New York Times joins the fray: Exercise and the ‘Good’ Bugs in Our Gut where Gretchen Reynolds writes:

The findings suggest that, in addition to its other health benefits, frequent exercise may influence our weight and overall health by altering the kinds of organisms that live inside of us.

No – the findings do not suggest that.  The findings are consistent with that theory but they are consistent with many many many other theories.  FFS this is maddening.  And the article ends with a quote from one of the authors:

But even in advance of those findings, he said, it seems likely that any amount of exercise should make your gut more welcoming to the bacteria that you want residing there.

I note – I found out about this article via Twitter

//platform.twitter.com/widgets.js

About Jonathan Eisen

I am an evolutionary biologist and a Professor at U. C. Davis. (see my lab site here). My research focuses on the origin of novelty (how new processes and functions originate). To study this I focus on sequencing and analyzing genomes of organisms, especially microbes and using phylogenomic analysis

6 thoughts on “Please make it stop – overselling the microbiome award for rugby, exercise, microbiome stories

  1. Jonathan, thank you for correcting my interpretation of the overinterpretation. I'm usually pretty vigilant about not mistaking correlation for causation, as well as normally looking for confounding variables. I didn't have a chance to read the original paper before I tweeted Kristina's article. Points well-taken.

    Like

  2. BMJ.

    There is your problem: those clowns will accept anything.

    Seriously, I have never – no, not ever – been pointed to a “journal” in the BMJ stable and found it to be worth anything – no, not anything.

    These is why reputable journals have peer review with teeth, because we have centuries of experience with overly ambitious people overly hyping their work.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s