Skip to main content

More Shoddy Reporting from the BMJ

 The BMJ really seems to like its investigative reporting, but the quality of this reporting, as of late, leaves me uneasy; it frequently touches on popular concerns regarding industry's role in the scientific process, and their undoubted desire to support research that will shine well on their products. They had their sugar piece that attempted to paint actually legit researchers like Susan Jebb in a bad light. They gave voice to Nina Teicholz's perspective on the dietary guidelines - Ms Teicholz paints herself as someone who is concerned about the quality of evidence and bias that goes into the Dietary Guidelines, but from reading her work, she doesn't seem to have a universally applied ruler with which she measures the quality of research (#IIFYB). She attempts to sell to us that recommendations to replace saturated fats with PUFAs have been bought by the vegetable oil industry. Despite what I find to be weak arguments, she has the amazing ability to distract people from the fact that her recommendations are supported by little to no evidence while getting everyone to notice the limitations of the evidence supporting current guidelines. Her piece has prompted multiple responses, including one from the DGAC.  The responses to this have been thorough enough that I don't feel the need to further comment.


However, I feel inclined to comment on the BMJ's most recent issue related to calcium/vitamin D and osteoporosis. It attempts to paint a complex web of concern surrounding recommendations from major organizations about the role of calcium/vitamin D supplements, and the role of industry in influencing the evidence. Scott Gavura covered the issue nicely here, concluding that the evidence for benefit from supplementation is there but modest, and the evidence for the authors' arguments regarding industry influence is weak.

I wanted to emphasize some of/add a few comments onto Dr Gavura's thoughts:

1. It remains unclear to me the extent of the authors' issues related to vitamin D and calcium - while they clearly don't like supplements, are they taking issue with the IOM's DRIs? Right off the bat, after discussing the failure of many calcium and/or vitamin D supplementation trials, they state:


The authors' conclusion that increased calcium and vitamin D shouldn't be recommended is based entirely on their previous discussion of largely null supplementation trials. "Increased calcium and vitaminD" could be very general, and just mean excessive intakes through supplements, but following this conclusion, they state "however, many guidelines published since then recommend.." suggesting to me that they're also using null supplementation trials to question the recommendations of major scientific bodies.  This concern of mine is supported by their table 1 - they list several groups' recommendations for general intakes of calcium and vitamin D, through all sources, but also include the USPSTF's lack of recommendations for calcium/vitamin D supplementation in specific populations - its highly concerning that the authors appear to have conflated the USPSTF's lack of recommendations for supplements with some sort of statement about the general recommendations for total calcium intake through foods/supplements combined.

Supplementation trials, like those included in the USPSTF's report, should not be used as evidence that the a recommendation/DRI from the IOM is wrong. Why is that approach invalid? Let's take a look at the largest of the trials included in the United States preventive Services Task Force (USPSTF) analysis, which the authors regularly cite to support their perspective, as an example. The women included were participants in the WHI and, at baseline, average calcium consumption was just shy of the 1200mg recommendation by the IOM.  By randomizing some of these women to an extra calcium supplement plus vitamin D, and then comparing against relatively well nourished women, we're really testing whether excess calcium supplementation on top of intakes at near recommendation levels is beneficial - despite how the BMJ authors appear to be wielding supplementation trial data/the USPSTF report, they are not an argument to suggest that the DRI is wrong, or using calcium supplements to hit the DRI wouldn't be beneficial. (Not surprisingly, when we look at their secondary analyses, those who were not taking calcium supplements at baseline saw significant benefits for reducing hip fracture rates (RR for nonusers of supplements: .70))

(Left Column: Intervention Group, Right Column: Controls)


If the trials cited looked at the randomizing women with low calcium intakes to calcium supplements that helped reach the DRI, or placebo, and saw no beneficial effects on fracture risk, that would enable the BMJ authors to question recommendations. This, however, is not what most of the trials cited by the BMJ authors or the USPSTF are doing. If the BMJ wants to question the calcium recommendations, they should focus critiques on the Ca balance studies the IOM uses to derive the DRI, not on null supplementation studies.

 Regarding vitamin D, the authors have more of a point that there is quite a range of recs and I would argue some of these recommendations are excessive, but that's largely due to the literature being an all out war zone regarding the appropriate blood levels of 25OHD and subsequent intake of vitaminD needed to maintain said level. Most of the recommendation levels are similar with only the two Endocrine societies recs popping out as outliers. 

  2. The authors mis-portray the perspective of the NOF - they conveniently place the NOF's concern about failure to meet the DRI's next to the USPSTF failure to find benefit of supplementing calcium/vitaminD . I've screenshott'ed below how they represent the NOF. With the way the USPSTF report is presented, the reader likely interprets this as the NOF is out there recommending large calcium supplements for the older populations to prevent fractures, contradicting the USPSTF report. But looking at the link that the BMJ authors cite as evidence that against the NOF (here), it's hard to find any statements that are concerning or contradicting the USPSTF report. The NOF fully states that the results of the USPSTF were not surprising, and as they have continually stated, all individuals should aim to get the calcium DRI (1000-1200mg) from foods first, and supplement only to make up shortfalls. This doesn't read to me like some bought out industry shill perspective - this is exactly what I would tell any future client I might have, or how I would convey the evidence to students that I TA for. This report only adds to the confusion perpetuated by similar reports that uses failed supplementation trials to cast doubt on the importance of achieving the DRIs for vitamin D and calcium for bone health.



3. I've talked about mainly calcium, because the vitamin D story is quite complicated. As Dr Gavura noted, the most recent Cochrane review supports the use of vitamin D + calcium to prevent hip or any type of fracture, but we should address some of the BMJ authors' points about the very high vitamin D levels recommended by some groups in their Table 1. I've spoken about some of the issues with vitamin D in the past, and have not generally been a passenger on the vitamin D supplements train, but to paint this as a controversy because of industry/academic influence is over-stretching it IMO. The authors of the BMJ piece are particularly critical of the Endocrine Society's recommendations, who vehemently disagreed with the IOM's report. But many people in the field did. Walter Willett, chair of the Harvard School of Public Health, came down very hard on the IOM's report - for any interested, Harvard held a roundtable discussion about the IOM report with representatives from the two different camps. The entire vitamin D chapter in the well respected nutrition textbook, "Present Knowledge in Nutrition", writes about vitamin D on the basis that it disagrees with the IOM report. This issue really reflects the field of nutrition's issues with observational vs RCT data, and chasing after outcomes where having causal data is nearly impossible (vitamin D has been linked to everything under the sun, including many types of cancers). I personally think it's important to note that the blood levels recommended by these groups advocating for higher levels (e.g. ES, 30ng/ml) don't approach the blood levels that are a cause for concern according to the IOM (greater than 50ng/mL), nor do the recommended doses reach the IOM's UL (e.g. ES, 2000IU, UL 4000IU), leading many to take a sort of 'what's the harm' approach when observational evidence indicates possible benefit. I don't agree with that data or making recommendations off of associations without a greater understanding of the underlying biology, but I also don't think it's egregious given the low cost of vitamin D supplements, potential for benefit, and lack of likelihood of harm. Passing off this issue as something of industry/academic bias is doing the scientific controversy as disservice IMO.

4. The last point I'll comment on is the paragraph below in the BMJ report. They, like many lately, are trying to argue that industry funding of research makes the research flawed:


The report offers no actual critique of the re-analysis of the WHI paper that I discussed above - the reanalysis, not surprisingly, finds significant benefits in those that received supplements but weren't baseline supplement users (AKA benefits for those whom the supplements weren't extra/leading to excessive calcium). They cite a 'contemporaneous analysis' that didn't show reductions in hip fracture risk with calcium supplementation, but casually fail to mention that this paper acknowledges the results of previous analyses showing calcium and vitamin D together reduce the risk of hip fracture, a conclusion that goes against the BMJ authors' main efforts. If the WHI had only supplemented calcium, it would've been appropriate to invoke this contemporaneous analysis of calcium-only supplementation trials, but the WHI didn't, so this is a misrepresentation again by the BMJ authors.

If you couldn't guess, I find this report quite sloppy, and a rather poor example of industry influencing the evidence. This adds to my personal concern that a hyperawareness of financial conflicts of interest can paint a picture that is far from accurate. Passing off legitimate scientific debate as though it's all about industry corrupting the science doesn't do anyone any good.

Comments

Popular posts from this blog

Beware the Meta-Analysis: Fat, Guidelines, and Biases

Headlines were abuzz this week, reporting that a new review of randomized controlled trials at the time of the low-fat guidelines didn't support their institution. Time , Business Insider , and The Verge all covered the topic with sensationalist headlines (e.g. 'We should never have told people to stop eating fat' #weneverdid). I won't spend every part of this blog picking apart the entire meta-analysis; you can read it over at the open access journal, BMJ Open Heart (1) -- (note, for myself, i'm adding an extra level of skepticism for anything that gets published in this journal). I'm also not going to defend low-fat diets either, but rather, use this meta-analysis to point out some critical shortcomings in nutritional sciences research, and note that we should be wary of meta-analyses when it comes to diet trials. First off, let's discuss randomized controlled trials (RCTs). They are considered the gold standard in biomedical research; in the hierarc

On PURE

The PURE macronutrients studies were published in the Lancet journals today and the headlines / commentaries are reminding us that everything we thought we think we were told we knew about nutrition is wrong/misguided, etc. Below is my non-epidemiologist's run down of what happened in PURE. A couple papers came out related to PURE, but the one causing the most buzz is the relationship of the macronutrients to mortality. With a median follow up of 7.4 years, 5796 people died and 4784 had a major cardiovascular event (stroke, MCI). The paper modeled the impacts of self reported dietary carbohydrate, total fat, protein, monounsaturated (MUFA), saturated (SFA), and polyunsaturated (PUFA) fatty acid intakes on cardiovascular (CVD), non-CVD and total mortality; all macros were represented as a percentage of total self reported energy intakes and reported/analyzed in quintiles (energy intakes between 500-5000kcals/day were considered plausible..). All dietary data was determined by a

Want To Buy: A Placebo

A well-designed/performed, double-blind, randomized, placebo-controlled trial provides a high level of certainty about the effectiveness of an intervention. In scientific training, the need to utilize a placebo relative to your variable of interest is one of the first things you learn when designing an experiment. As many in the basic sciences and evidence-based medicine fields have become more interested in nutrition and its impact on health/biology (their interest is well-justified), there has been insufficient appreciation for the difficulty in performing nutrition research. This day 1 principle of "placebo-controlled" poses a particular challenge for many nutrition experiments: there is no placebo.  Consider an example that actually plagued causal inference in nutrition history: It was known that feeding diets high in saturated fatty acids was associated with higher LDL. Does that mean that saturated fat raises LDL? How would you design a study to show