Skip to main content

Beware the Meta-Analysis: Fat, Guidelines, and Biases

Headlines were abuzz this week, reporting that a new review of randomized controlled trials at the time of the low-fat guidelines didn't support their institution. Time, Business Insider, and The Verge all covered the topic with sensationalist headlines (e.g. 'We should never have told people to stop eating fat' #weneverdid). I won't spend every part of this blog picking apart the entire meta-analysis; you can read it over at the open access journal, BMJ Open Heart (1) -- (note, for myself, i'm adding an extra level of skepticism for anything that gets published in this journal). I'm also not going to defend low-fat diets either, but rather, use this meta-analysis to point out some critical shortcomings in nutritional sciences research, and note that we should be wary of meta-analyses when it comes to diet trials.

First off, let's discuss randomized controlled trials (RCTs). They are considered the gold standard in biomedical research; in the hierarchy of evidence, they sit near the top, just below systematic reviews and meta-analyses (which either include or are based upon RCTS). We tend to glorify RCT's because they determine causality (other 'lower' types of evidence can show association/correlation, but not causation). If I want to know that statins prevent cardiovascular diseases, I need to perform large randomized controlled trials to causally demonstrate that they prevent heart disease.

But in nutrition, while RCTs are important, they aren't the end all be all for population recommendations. Classically, nutrition has studied individual nutrients, be they macro- (fat, carb, prot, alcohol) or micro - (vitamins and minerals). While I agree that this approach is important, we cannot pretend that foods are one chemical, or even a couple chemicals. Over 25000 chemicals have been associated with the food metabolome.  You can certainly demonstrate causality via supplementation trials, but if you do a randomized controlled trial varying 'fat' content in the diet, you would have to isolate the type of fat, the total amount and food source. Seeing as all foods contain mixtures of fatty acids, along with an abundance of other nutrients, a low fat diet trial doesn't isolate for one thing; we don't eat purified diets, and therefore, will not isolate for one nutrient. Apart from the glaring issue that low fat could mean a diet high in legumes or a diet high in sugar, RCT's are limited in their length of time; we will never have a randomized controlled trial that lasts 60-80 years to show that high calcium intakes prevent osteoporosis. Even if we did, we would have a hard time ensuring that people actually ate that diet (would someone adhere to 3 glasses of milk per day? how could we tell, accurately? is it just the calcium or the milk complex that helped bone?). Even if adherence was high, would that ensure that that one nutrient was/wasn't important for that outcome? (think: what were the background diets; they were likely not all the same) Nutrients are notorious for interacting with each other - if we did an 60 year long RCT of calcium supplementation, without considering vitamin D, we would likely make erroneous conclusions about the important role of calcium in bone health. RCTs are the gold standard for causality, but elevating RCTs that attempt to isolate for one nutrient to a level where we assume there is no other level of evidence that could compete is naive in nutrition and public health - but that is precisely the assumption that comes with this meta-analysis.

One has to wonder why the authors thought that policy makers at the time of the low/modified fat recommendations would have waited decades (or thought it possible) for randomized controlled trials to come out showing that low-fat diets worked for preventing CVD - it's an impossible standard to hold. As Marc LaLonde, Canadian Minster of National Health, commented in the original McGovern report that this meta-analysis critiques, Canada's health problems were significant that they had to act now (dare we admit health wasn't perfect before the low-fat recs), and one could debate the consumption of butter and eggs in the diet endlessly (also, check out the totality of their recommendations and lemme know how bad they sound) . The policy makers at the time did have Keys' 7 Country Diet Study to act on, and a number of studies showing that CVD biomarkers were affected by fat (particularly the type of fat) in the diet (more from Seth at TheScienceofNutrition on the 'biased' Keys study).  Much like the original 1980's dietary guidelines, the McGovern report acknowledged the many limitations of the low-fat guidelines (e.g. not all advisor's agreed - some even said diet didn't have evidence to support it mattering at all, there was huge genetic variability in response to dietary cholesterol, etc), and never advocated for consumption of just any kind of carbohydrates at higher levels of calories - but that's what Americans did. We're still waiting for RCT's that isolate for just one dietary component that is clearly linked to a cardiovascular outcome - they have not happened, and likely will never. Even our low-fat trials that we've used to toss aside the low-fat paradigm (again, something I don't advocate) didn't use nearly the levels of fat that original epidemiological associations suggested (e.g. Japan ~10% fat) - we discounted low fat off of self-reported intakes at ~25- 30% of kcals from fat, with pretty piss-poor fiber intakes #nottherecommendations (2). I long for the day that RCTs are a true gold standard in nutrition, but if you're going to talk about eating to prevent diseases in 10-60 years, RCTs are not gonna fly.

The other point I wanted to touch on is meta-analyses of diet trials. Frank Hu had a really good commentary on the illusion of big data in nutrition, and how meta-analyses can be severely misleading (3). I've talked about the pitfalls of meta-analyses before in my post on sodium. Since meta-analyses are the result of pooling data from RCTs, the limitations that apply to RCTs also apply to the meta-analysis. While grouping data together might sound wonderful and give us a lot of statistical power, this comes with the assumption that our trials did the same thing, providing the same treatment in a similar population with similar levels of adherence. If they did this, it would be easy to compile the data, but significant heterogeneity in trials can severely limit a meta-analysis in nutrition,... so let's take a look at this recent meta-analysis. It uses the following trials:

1. The Rose Corn Oil trials (4) - 70 patients, 3 groups (control, corn oil, olive oil), most of whom had a history infarction. They received 80g/d of the oil, leading to 500-600 kcals per day coming from oil and about a quarter of their total kcals coming from fat.
2. Low-fat diet in myocardial infarction: A controlled trial (5)- I can't access this full text - other texts reference it and say that the test group only had a small percent reduction in plasma cholesterol based on the treatment.
3. CONTROLLED TRIAL OF SOYA-BEAN OIL IN MYOCARDIAL INFARCTION: The objective of this trial was to see whether replacing saturated fats with unsaturated reduced the incidence of MCI's. The men in this trial already had an infarction. 199 men were in the test group, and 194 in the control. The test diet was supposed to take 85g of soybean oil per day, and had a higher fat intake than the control group. 46% of kcals came from fat for the test diet group.
4.A Controlled Clinical Trial of a Diet High in Unsaturated Fat in Preventing Complications of Atherosclerosis (7):  The experimental diet (n= 424) was 39% fat, and the control (n=422) was 40%. Adherence averaged 56% for the control subjects and 49% for the experimental group. The experimental diet was higher in linoleic acid than the control (10 vs 38 % of total fatty acids).  Men were not excluded if they had previous atherosclerotic complications. This was the only primary prevention trial in the meta-analysis, and did not find statistically significant results for the primary outcomes, but did for primary and secondary when grouped)
5.  Leren's Oslo Diet Heart Study publications (8,9 ): Already had one myocardial event in. n =412. Experimental group consumed 104grams (39% of calories). 62% of the experimental group was judged to be excellent adherers. The participants were encouraged to eat a lot of fish, and consumed a lot of omega 3's/vitamin D, while also being encouraged to eat more fruits/vegetables/nuts. The trial was 5 years long.
6. Sydney Diet Heart Study (10)- This study attempted to replace saturated fats for polyunsaturated fats largely using Miracle Margarine. (updated:) It has been critiqued because, at the time, the margarine would have contained trans fats too. It's not known, based on the dietary intervention, what role this would've played. Some of the controls in this diet also began substituting the n-6 oil for butter after their coronary event. The safflower oil contains an insignificant amount of n-3 relative to other trials in the intervention that used soybean oil. More information on the trans-fat controversy of the diet can be found here, here and here.

Looking at these trials, I'm curious why the authors didn't just skip the meta-analysis and say "there wasn't a significant body of primary prevention trials from this period demonstrating an effect of low or modified fat diets on CVD outcomes" . Since we make public health recommendations for the primary prevention of disease, why do a meta-analysis on secondary prevention trials to comment on public health recs? Furthermore, it's been noted for decades that secondary prevention trials are not a great way to test the lipid hypothesis (11):, "...because of multiple changes in lifestyle men who have had myocardial infarction are not a good choice for testing the lipid hypothesis". Despite this, the authors of the meta-analysis go on to claim in their discussion "This under-mines the role of serum cholesterol levels as an intermediary to the development of CHD" - a pretty strong claim coming from a meta-analysis that looked at people who had already developed CHD.......Not only were they not primary prevention trials, they all didn't employ the same treatments - again, heterogeneity in diet trials confounds (e.g. long chain omega 3's in Oslo, lack of omega 3's and possible trans fats in Sydney), and make meta-analyses of them rather inappropriate. I'm failing to see what the point of this meta-analysis is - one could have simply written a 2 sentence review paper and more concisely stated their point that causality between fat modification and CVD had not been demonstrated at that time.

You can certainly take issue with the recommendations, and the quality of the data used to set them (I certainly don't have tons of faith in food frequency questionnaire-based epidemiology). You can also take issue if you think that they led to policies that promoted high calorie, low nutrient intakes. I, personally, see the issue as interpretation of the guidelines (what they recommend and what people ate are highly discordant). If anything, the low-fat guidelines should serve as a cautionary tale for further reductionist public health recommendations. But what I'd advise against is glorifying the RCT as the sole source of evidence to be used for nutrition recommendations for chronic disease prevention. RCTs in diet trials have inherent limitations, described above, and simply relying on  them would leave us with virtually no public health recommendations beyond nutrient deficiency prevention. We don't live in a world where participants  in diet trials have 100% adherence for decades, and eat the exact same background diets, apart from the manipulated variable. If we want to make recommendations about preventing chronic disease, we have to rely on, to some degree, epidemiological data, and short term trials altering biomarkers - and that's what the original guidelines did. (Actually, the farther I've gotten in my career/education, the more I've realized the best thing for the future is not picking apart the limitations of our evidence based but improving them e.g. find better biomarkers, implement genetic/molecular variables into epidemiology - improving limited methodology is better than waiting for human behavior to completely change).

I'm also curious what the authors of this meta-analysis, notable for their disdain for sugar and carbohydrates, think about the original McGovern report's demonization of sugar and 'refined products'. There were no massive randomized controlled trials showing that sugar/refined carbohydrates reduced any disease rate at that time, yet the guidelines suggested reducing sugar intake by about 40%. Why was this not included in their meta-analysis? I guess if you're trying to sell a sexy story of  dubiously 'bad' science, being impartial isn't supported by observational data ;) #IfItFitsYourBias #IIFYB

Here are some expert thoughts from science media centre on this issue.

Note: based on the comments i've received, I'd like to note that I am fully aware that observational evidence is far from perfect. Because RCTs aren't perfect does not mean we overly extrapolate from correlations (anyone who follows me on twitter knows I am highly critical of those who draw near causation out of correlation e.g. saying things like not eating dairy prevents prostate cancer). If anything, the point of this post is to point out inherent limitations in nutritional sciences research as a whole, and demonstrate that research, and our interpretation of it, needs to be improved. Many in the field seem unwilling to accept that there are certain areas of research (e.g.  dairy and prostate cancer) that our lines are of evidence are inherently limited by and we can't say much about it, with any causal language - talk to an oncologist about 'anti cancer' diets and check their enthusiasm, and then watch every Netflix documentary about food; note the discordance.

1. http://openheart.bmj.com/content/2/1/e000196.short
2. http://jama.jamanetwork.com/article.aspx?articleid=202339
3. http://onlinelibrary.wiley.com/doi/10.1111/nure.12156/abstract
4. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2166702/
5. http://www.ncbi.nlm.nih.gov/pubmed/4158171
6. http://www.ncbi.nlm.nih.gov/pubmed/4175085
7. http://circ.ahajournals.org/content/40/1S2/II-1.abstract
8. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1750292
9. http://circ.ahajournals.org/content/42/5/935.abstract
10. http://www.ncbi.nlm.nih.gov/pubmed/727035?dopt=Abstract
11. http://link.springer.com/chapter/10.1007%2F978-1-4684-0967-3_18

Comments

  1. This comment has been removed by the author.

    ReplyDelete
  2. Nice post, Kevin. It's funny how RCT's are the only studies that count when it comes to saturated fat, but we never see anyone requiring RCT's on trans fats or sugar, etc. Harcombe et al. write in their paper that "RCTs provide the best evidence", with reference to a BMJ article that says: "Limited randomised controlled trials need other forms of evidence to be appraised and considered." and "… the weight given to each bit of evidence should be determined by a detailed appraisal of the characteristics of that evidence".

    As I wrote on my own blog about this (in Norgwegian here: https://sunnskepsis.wordpress.com/2015/02/11/fettrad-og-vitenskapelige-kortsyn/), the "RCT fallacy" is one of the "six persistent research misconceptions" Kenneth Rothman writes about in this highly recommended paper: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4061362).

    ReplyDelete
    Replies
    1. This RCTs required issue is the most intellectually bereft. It is applicable for phenomena for which a reasonably short term effect can be expected to be measured. Diet and chronic disease do not qualify unless we are talking about extremes that have either an acute toxic effect or establish a deficiency that would do the same.

      I somehow doubt Harcombe can provide a single RCT to support the claims that came out of her 15 years of research on what makes us fat: Candida, food intolerance and hypoglycemia. No? Didn't think so ;-)

      Delete
  3. "If we want to make recommendations about preventing chronic disease, we have to rely on, to some degree, epidemiological data, and short term trials altering biomarkers"

    While trials have limitations, I think it should not be forgotten how limited observational data is (http://www.ncbi.nlm.nih.gov/pubmed/22996110), especially with regard to diet where associations tend to be weak and inconsistent. Recommendations on diet should not be made on observational data alone, and RCTs should be required. Yes, trials have limitations, but this does not make observational data any more reliable. It should also be pointed out that sometimes the simplest explanation is the correct one, and the reason trials fail could simply be because the interventions are ineffective.

    Short term trials on biomarkers are even more problematic since surrogates are not actual clinical endpoints (and limited only to a few known biomarkers), and reliance on them can even be dangerous -
    http://www.ncbi.nlm.nih.gov/pubmed/23529157
    http://www.ncbi.nlm.nih.gov/pubmed/21844159


    "This study attempted to replace saturated fats for polyunsaturated fats largely using Miracle Margarine. It has been critiqued because, at the time, the margarine would have contained a whopping dose of trans fats too (not exactly a good intervention...). The recent Chowdhury saturated fat meta-analysis caught a lot of flack for including this study because, when you remove it, you see significant effects of Omega 6's on CHD - see the comments on that meta analysis here".

    The main intervention in Sydney was liquid safflower oil, not Miracle Margarine. It would also be more accurate to say that the intervention was a replacement of saturated fat + trans-fat with polyunsaturated fat + trans-fat. Whether the study was confounded by trans-fat is not known, but the authors do not think trans-fat played a role (http://www.bmj.com/content/346/bmj.e8707/rr/638479). Those making this claim are biased against saturated fat and hold a major double-standard, since mere possible confounding is an issue Syndey, yet not a word is said about known confounding (even by trans-fat!) in Oslo, STARS, and FMHS, which by the way are also trials in Chowdhury's analysis. Also, complaining about confounders in trials and then turning around and citing observational data makes no sense, since confounding is a far greater problem there.

    Dietary clinical trials have taught us a lot, but some people because of an anti-SAFA mindset, prefer to ignore the most consistent hypothesis from dietary trials that excessive omega-6 and too little omega-3 is a problem -- saturated fat AND carbohydrates are likely not the problems. Large randomized trials conducted for "decades" are not needed as many small (e.g. few hundred participants if secondary prevention) well-controlled trials on hard endpoints all pointing in the same direction (even if results are non-significant) can also be compelling.

    ReplyDelete
    Replies
    1. Yes, the record is full of data on how diets rich in nuts and olive oil, grains and legumes induce obesity and disease.

      /sarcasm

      Delete
    2. Perhaps because you don't want to.

      Your message contained all the "classic" errors you're bound to face when facing the SAFA advocates.

      1) Trying to downplay the well-known problems in the old trials by reminding that observational studies have problems. Well, duh. What we know about SAFA and CVD doesn't depend on either epidemiological studies with their inherent problems nor on trials with another set of inherent problems (nor that you can ignore epidemiological studies just like that, as you seem to do). May I remind you that the effect of SAFA on LDL-levels is well-known (as well as the mechanism, impairment of LDL receptor function), cf. metabolic ward studies? Or that there's similar evidence indicating that SAFA is impairs HDL functionality, too, when compared to unsaturated fats? And that when you study the trials/meta-analysis which actually address the issue of what to replace SAFA with & how much & the context ... Well, you get just what is to be predicted: a benefit when replacing SAFA with PUFA. And there are of course differences in WHERE you get SAFA, more of that in point 3), which further attenuates the findings - not due to SAFA itself .

      2) "Interventions are ineffective". Well, d'uh, once again: indeed they are. But NOT because of the intervention per se but because people just cannot be bothered to eat healthily. The heterogenous results in heterogenous populations pretty much show what we already knew on the basis of what I stated in 1): the modest results achieved in modifying the key risk factors (e.g. minor deductions, reported often in tot. chol. levels not even in LDL) mean disappointing results. Think of WHI, for instance. You cannot argue that reducing SAFA per se doesn't reduce CVD end points if this reduction isn't really even achieved. You can of course argue that such a focus in itself is of minor importance as it is hard to implement. But that's a whole different thing.

      3) False dichotomies a la "SAFA isn't a problem but excessive n-6 is". Err, in case you've missed it, there's evidence pointing that ALL of the aforementioned (too much n-6 from refined oils & too little n-3, too much SAFA & refined carbs) are well-known issues. There's, however, data to indicate that while replacement of SAFA with carbs as in "carbs in Western context" does very little either way in terms of CVD risk, e.g. intake of PUFA - both n-6 and n-3 - in moderation as well as carbs from whole grain and fruits & veggies - decreases the risk.

      No one is arguing that SAFA is poison as such. Depending on the amount & the overall risk, however, it can be detrimental. In addition, foods richer in SAFA can be either neutral or even beneficial due to the fact that it's not all about the fat context as whole foods have effects on risk factors that cannot be reduced to a single component. E.g. eating fermented cheese (quite heavy on SAFA) doesn't seem to increase one's LDL due to various mechanisms.

      Delete
    3. Zahc, after seeing seeing your comment on the Skeptical Nutritionist in October of last year, I read about the Sydney study and the responses. After that I didn't bother seeking out the others studies you mentioned there. What would have been nice is if the authors of this BMJ Open Heart had just gone over the all of these RCTs on saturated fat, the methodology, the pros and cons. That would be a much better use of time. I think your point about the double-standard is probably a good one. As the other commenter pointed out on Skeptical Nutritionist they are small studies and so my impression is they are too prone to non-sampling error. But Ramsden's speculation that the control group may have somehow got trans fat or that the exposure to ruminant trans fats mattered just aren't convincing, especially when he says the intervention was actually supplied with margarine.

      That Grimes, Schulz article you posted seems odd to me. I think they correctly identify the problems we all know about with the way that observational studies are reported in the media. Where I’m not really clear is why they think signal to noise is the underlying problem and strength of association should be the only consideration. The problem with observational research is unusually understood to be whether clear signal that exists is indeed an association, whether it is causal, or if it is the result of bias and confounding. Going back to the famous Hill paper that is cited, it’s clear that strength of association is great to have but should not be considered a sine qua non.

      Delete
    4. "Perhaps because you don't want to".

      Maybe you could explain what diets rich in nuts and olive oil, grains and legumes have to do with my comment. Just because I'm for saturated fat doesn't mean I'm against any of these foods.

      "Trying to downplay the well-known problems in the old trials by reminding that observational studies have problems".

      Huh, all I was saying was that you cannot rely on observational data alone or surrogate markers (and keep in mind their severe limitations). Where did I ever say that the old trials don't have limitations? In fact, skeptics have always pointed out the limitations of many of these dietary trials (some of which were multifactorial and didn't come close to isolating SAFA).

      It seems you took my statement out of context: While trials have limitations, I think it should not be forgotten how limited observational data is (http://www.ncbi.nlm.nih.gov/pubmed/22996110), especially with regard to diet where associations tend to be weak and inconsistent. Recommendations on diet should not be made on observational data alone, and RCTs should be required. Yes, trials have limitations, but this does not make observational data any more reliable.

      I also said:

      Short term trials on biomarkers are even more problematic since surrogates are not actual clinical endpoints (and limited only to a few known biomarkers), and reliance on them can even be dangerous.

      The HDL study you cite is one acute biomarker study (even worse is that it was based on in vitro data). Sorry, but that is just very poor data to be making such a claim, and this illustrates my point.

      "And that when you study the trials/meta-analysis which actually address the issue of what to replace SAFA with & how much & the context ... Well, you get just what is to be predicted: a benefit when replacing SAFA with PUFA".

      No, they don't. Are we going to go through this again? I'm Z.M. by the way.

      "But NOT because of the intervention per se"

      You can speculate that this is the case, but my point was that it is possible that the intervention is in fact ineffective. The burden lies on those making the claim that fat modification is beneficial, and speculating as to why trials fail is not the same as providing positive evidence for your claim. It seems that you are assuming that fat modification is beneficial, and therefore negative results must mean that the trial is flawed -- this is a biased approach. In other words, you are saying that the intervention could not possibly be ineffective, therefore negative results must mean that trial is flawed.

      Again, it seems that you didn't read my statement carefully: It should also be pointed out that sometimes the simplest explanation is the correct one, and the reason trials fail could simply be because the interventions are ineffective.

      "False dichotomies a la "SAFA isn't a problem but excessive n-6 is".

      I never made such an argument. Trials suggest that excessive n-6 intake and too little omega-3 is a problem, but on the other hand, trials do not suggest any harm or benefit with regard to SAFA and carbohydrates (of any type).

      Please read carefully what I do write because you constantly pin arguments on me that I never made. I'm not surprised though, since all my correspondences with you seem to be correcting misunderstandings.

      Delete
    5. Zahc,
      I'm well aware of the limitations of other types of research. The point of writing this is that we overly fetishize/romanticize RCT research and their role in research, when there are significant limitations to them. I've written before how Nutritional Sciences is inherently soft because of the limitations of all lines of research: http://nutrevolve.blogspot.com/2014/03/is-nutrition-soft-science.html . The best recommendations come out when preclinical and clinical data start to coalesce. I'm all for improving biomarker use in studies (using LDL-P instead of LDL-C, actually quantifying RCT instead of just measuring LDL).

      I updated the part on the SDHS to reflect that there's more of a controversy surrounding trans fats. But it doesn't change my point. It used a high n-6 safflower oil will little n-3, and had it substituted many different sources of fat, not just animal. It's inherently different than the other trials. This goes back to my point - Meta analyses of diet trials that don't do the same intervention are highly limited and inappropriate.

      Please point me in the direction of some trials that clearly show detriments of n-6 intakes in the recommended ranges - In both Ramdsen and Mozaffarien's meta-analyses on PUFAs replacing SFA, the n-6/n-3 ratio appeared unimportant. Prospective cohort studies simply don't support the notion that increased omega 6's increase risk of death from CVD like many will push: http://circ.ahajournals.org/content/early/2014/08/14/CIRCULATIONAHA.114.011590.abstract
      http://circ.ahajournals.org/content/early/2014/08/26/CIRCULATIONAHA.114.010236.abstract

      Delete
    6. Zahc - like Kevin, I also would like to know more about all these trials that "suggest that excessive n-6 intake and too little omega-3 is a problem". If you are talking about inflammation, I guess you are aware of meta-analyses of trials that show no effect of n-6 (i.e. linoleic acid) on inflammation markers (http://www.ncbi.nlm.nih.gov/pubmed/22889633).

      Delete
    7. "Huh, all I was saying was that you cannot rely on observational data alone or surrogate markers (and keep in mind their severe limitations). Where did I ever say that the old trials don't have limitations?"

      So that wasn't you trying to distract us from the issue at hand, huh? :-)

      "In fact, skeptics have always pointed out the limitations of many of these dietary trials (some of which were multifactorial and didn't come close to isolating SAFA)."

      ... and yet here you defend your stance by relying on ... them?

      Oh wait, are we going to hear about the "high quality" of Rose Corn Oil now? :-)

      "Short term trials on biomarkers are even more problematic since surrogates are not actual clinical endpoints (and limited only to a few known biomarkers), and reliance on them can even be dangerous."

      Well, please do explain then how this applies to the question at hand? How is reliance on e.g. LDL-C/-P dangerous?

      "The HDL study you cite is one acute biomarker study (even worse is that it was based on in vitro data). Sorry, but that is just very poor data to be making such a claim, and this illustrates my point."

      Hmm? As I didn't include a reference, would you kindly mention the study you think I mentioned? Please. 'Cause it sure as hell wasn't an in vitro study.

      "No, they don't. Are we going to go through this again? I'm Z.M. by the way. "

      Well duh? That was impossible to miss. And yes, if you want to. Perhaps this time you've actually read e.g. Skeaff and Miller ...

      "You can speculate that this is the case, but my point was that it is possible that the intervention is in fact ineffective."

      Then you would have to point out the mechanism WHY. As far as your n-6 scare goes, there is none: the inflammation case is mostly based on in vitro stuff and/or applies to VERY high intake, well outside the recommended range. Claiming that this is "da shit" is simply unfounded.

      "The burden lies on those making the claim that fat modification is beneficial, and speculating as to why trials fail is not the same as providing positive evidence for your claim."

      .... and you're - once again - ignoring the fact that providing an explanation as to why these old trials show what they show is quite enough here. Scientific argumentation: any given study is subject to criticism and any finding is to be taken with a grain of salt. The results are in line of what is to be expected due to the study design, results etc. etc.

      " It seems that you are assuming that fat modification is beneficial, and therefore negative results must mean that the trial is flawed -- this is a biased approach."

      I am indeed assuming that there is a small as there are a) well-known mechanisms why and b) evidence that when this is achieved according to the guidelines, there's c) benefit. And I haven't stated the latter but mentioned that there are clear reasons why the results were what was to be expected so please refrain from imagining things.

      "It should also be pointed out that sometimes the simplest explanation is the correct one, and the reason trials fail could simply be because the interventions are ineffective."

      This "simplest explanation" has to be proven, so please do so: still waiting you to explain away e.g. the LDL receptor findings.

      "I never made such an argument."

      But here you make it again:

      "Trials suggest that excessive n-6 intake and too little omega-3 is a problem, but on the other hand, trials do not suggest any harm or benefit with regard to SAFA and carbohydrates (of any type)."

      Based on current understanding that is a false dichotomy. Or are you seriously claiming e.g. that high sugar intake isn't problematic?

      Delete
    8. Mie, when are you going to actually read what I say?

      I wasn't "distracting" anything.

      I don't rely on anyone since I read the studies myself and read arguments from all sides.

      I never claimed that Rose was "high-quality".

      The study you mentioned was not in vivo, but on isolated HDL (Nicholls et al. 2006).

      I have read Skeaff and Miller but also all the other analyses (i.e. I do not cherry pick).

      I'm not promoting a "n-6 scare", but simply stating the hypothesis most consistent with clinical trials.

      There is no "false dichotomy", and there is little trial evidence that carbohydrates (simple or complex) are either beneficial or harmful.

      The "simplest explanation" statement was one acknowledging another possibility as illustrated by my words ("could"): It should also be pointed out that sometimes the simplest explanation is the correct one, and the reason trials fail could simply be because the interventions are ineffective.

      Delete
    9. Kevin, thanks for replying:

      Kevin: "Please point me in the direction of some trials that clearly show detriments of n-6 intakes in the recommended ranges - In both Ramdsen and Mozaffarien's meta-analyses on PUFAs replacing SFA, the n-6/n-3 ratio appeared unimportant."

      I cannot make a claim about recommended ranges since no trials have evaluated this, which is why I said "excessive omega-6". I said "the most consistent hypothesis" from dietary trials is that "excessive omega-6 and too little omega-3 is a problem -- saturated fat AND carbohydrates are likely not the problems".


      Kevin: "Prospective cohort studies simply don't support the notion that increased omega 6's increase risk of death from CVD like many will push"

      The evidence from prospective cohorts is weak and inconsistent and also illustrates my point about forgetting about the limitations of such data. It does not mean that omega-6 is harmless (http://www.ncbi.nlm.nih.gov/pubmed/21865817). I have never made a claim that omega-6 has proven to be harmful, hence the use of my word "suggest" and "hypothesis".


      Kevin: " I've written before how Nutritional Sciences is inherently soft because of the limitations of all lines of research"

      Yea, I agree that it is soft, but I think randomized dietary trials on hard endpoints should be given most weight.

      Delete
    10. "Mie, when are you going to actually read what I say?"

      It seems that after this one there's simply no point as you continue you avoid answering & commenting but parrot yourself.

      "I wasn't "distracting" anything."

      Already dealt with this one.

      "I don't rely on anyone since I read the studies myself and read arguments from all sides."

      ... and this from a person defending the meta-analysis discussed here by supposedly criticizing the same studies included in it.

      "I never claimed that Rose was "high-quality"."

      At Doc's Opinion, during our previous chat, you claimed that trials involving pure fat modification were of "high quality", did you not?

      "The study you mentioned was not in vivo, but on isolated HDL (Nicholls et al. 2006)."

      The dietary experiment was conducted in vitro, after which HDL was studied in vivo - methodologically good choice given the issue being inspected. That's not quite the meaning of "in vitro", is it?

      "I'm not promoting a "n-6 scare", but simply stating the hypothesis most consistent with clinical trials."

      And STILL waiting you to explain away e.g. the LDL receptor findings. Or quit weaseling as you do with Kevin's reply concerning the findings of various meta-analyses & n-6. No one's encouraging the intake of n-6 up to 14-15% of tot. E.

      "There is no "false dichotomy", and there is little trial evidence that carbohydrates (simple or complex) are either beneficial or harmful."

      If you wish to embarrass yourself further, do continue. I'm sure we'd all like you to explore the evidence why high sugar intake is ok. Or perhaps you don't think so after all - even though it logically follows from what you just stated ...

      Delete
  4. "and this from a person defending the meta-analysis "

    really? I'm not sure where I was supporting this particular analysis.

    "At Doc's Opinion, during our previous chat, you claimed that trials involving pure fat modification were of "high quality", did you not?"

    I did not, and illustrates again you are not reading what I say. I said the diet was BETTER controlled in the n-6 studies and two of the mixed n-3/n-6 studies, which is clear.

    "And STILL waiting you to explain away e.g. the LDL receptor findings."

    That's a surrogate. If the harder evidence shows no suggestion of harm with saturated fat or no suggestion of benefit for omega-6 PUFA then the LDL receptor hypothesis is total rubbish and not evidence of anything. This can be taken further with cholesterol lowering drugs also, since most of them fail to lower (and at times increase) CV/CHD death and total mortality despite decreasing cholesterol. You cannot rely on surrogates and it can be dangerous as I said before.

    "The dietary experiment was conducted in vitro, after which HDL was studied in vivo - methodologically good choice given the issue being inspected."

    My point is that this may have nothing to do with what is going on in vivo, which makes it a methodologically poor choice and poor basis for your claims.

    "No one's encouraging the intake of n-6 up to 14-15% of tot. E."

    and nowhere have I said anyone was.

    "If you wish to embarrass yourself further, do continue."

    The only embarrassing thing here is your lack of attention to what I'm saying and making up stuff along the way, and I never said "high sugar intake" (whatever that means) is ok, nor is it implied by my statements.

    ReplyDelete
    Replies
    1. "I'm not sure where I was supporting this particular analysis."

      Drop the dishonesty, will you? Why would you be writing here otherwise?

      Or are you just plain incapable of forming an argument/position? Or do you agree that the abovementioned meta-analysis de facto portrays the key problem here: the RCTs show attenuated results of the benefits of fat exhange?

      "I said the diet was BETTER controlled in the n-6 studies and two of the mixed n-3/n-6 studies, which is clear."

      Have you actually read e.g. Rose Corn Oil? If the mere fat exchange with no control for a variety of other factors is the synonym for "better control", I have to - once again - question your judgement.

      "That's a surrogate."

      Belittlement. That's an explanation to the previous findings in metabolic ward studies. And consequently, every kind of intervention that lowers LDL substantially and doesn't have any adverse effects on other risk factors brings a benefit in CVD/CHD risk.

      "If the harder evidence shows no suggestion of harm with saturated fat or no suggestion of benefit for omega-6 PUFA then the LDL receptor hypothesis is total rubbish and not evidence of anything."

      And yet again you choose not to mention that we know WHY the harder evidence - by which you means looking at fat modification studies without any consideration for study quality - doesn't show this. Please stop being dishonest and selective.

      "This can be taken further with cholesterol lowering drugs also, since most of them fail to lower (and at times increase) CV/CHD death and total mortality despite decreasing cholesterol."

      ... and here you fail to mention e.g. that looking at total mortality makes very little sense in e.g. primary prevention setting - or that CVD death isn't the only treatment goal but reduction in CVD events matters as well. Not to mention failing to account for e.g. the effect size and adverse effects of certain drugs.

      "You cannot rely on surrogates and it can be dangerous as I said before."

      Perhaps you'd finally explain HOW in this case. Please.

      "My point is that this may have nothing to do with what is going on in vivo, which makes it a methodologically poor choice and poor basis for your claims."

      No, of course not. The HDL functionality just TOTALLY changes after the dietary experiment. And of course the matter at hand could've been studied in vitro without any problems!

      "and nowhere have I said anyone was."

      And yet your "hypothesis" on n-6 being virtually the ONLY thing about diet that matters is based on studies where the intake was of this magnitude.

      "and I never said "high sugar intake" (whatever that means) is ok, nor is it implied by my statements."

      Still having problems with logic? Ok, redefine your statement "carbs don't matter" then, please, if carbs in some form then DO matter, after all.

      Delete
  5. "That's a surrogate. If the harder evidence shows no suggestion of harm with saturated fat or no suggestion of benefit for omega-6 PUFA then the LDL receptor hypothesis is total rubbish and not evidence of anything. This can be taken further with cholesterol lowering drugs also, since most of them fail to lower (and at times increase) CV/CHD death and total mortality despite decreasing cholesterol. You cannot rely on surrogates and it can be dangerous as I said before."

    It is very difficult to demonstrate significant effects on mortality, even with drugs, because of power. An effect on mortality is of course important to public health, but a lack of effect doesn't disprove that LDL is causally related to CVD. (That being said, newer metaanalyses of primary prevention trials with statins do show significantly reduced risk for cardiovascular events AND mortality (http://www.ncbi.nlm.nih.gov/pubmed/19022156, http://www.bmj.com/content/338/bmj.b2376). Even among people with no history of CVD at baseline (CTT Collaborators, 2012: http://www.ncbi.nlm.nih.gov/pubmed/22607822)).

    As an example, the MRC soybean oil included in Harcombe et al.'s was seriously underpowered. It could by design only show whether the risk was reduced by more than 50 %, which no one would expect.

    But several of the trials included did show an effect on cardiovascular events, like the LA Veterans' and the Oslo study.

    ReplyDelete
  6. Erik,

    "It is very difficult to demonstrate significant effects on mortality, even with drugs, because of power. An effect on mortality is of course important to public health, but a lack of effect doesn't disprove that LDL is causally related to CVD"

    Claiming lack of power or speculating as to why a trial failed is not evidence of anything, but simply a possible explanation. The other possible explanation which is the default one is that the intervention is ineffective. The burden is on you to prove that LDL per se is causal, not on me to disprove it (although there is a good argument against it). Furthermore, lack of power is an unlikely explanation since effects from trials are consistently around 1 (as it is for virtually all cholesterol lowering treatments with regard to CHD/CV death and total mortality), and this is also reflected in meta-analyses involving hundreds or over 1000 deaths.

    Whether statins reduce mortality in primary prevention is controversial because different analyses come to different conclusions and the effect is close to 1.

    "As an example, the MRC soybean oil included in Harcombe et al.'s was seriously underpowered. It could by design only show whether the risk was reduced by more than 50 %, which no one would expect"

    There was little effect on heart attacks (in the predefined endpoint) and mortality, and CHD/CV death was virtually identical. This is a case where arguing for lack of power is a poor one. The original conclusion was the correct one that the results "lend little support to this suggestion or to the suggestion that a diet of the kind used should be recommended in the treatment of patients who have suffered a myocardial infarction" - http://www.ncbi.nlm.nih.gov/pubmed/4175085

    "But several of the trials included did show an effect on cardiovascular events, like the LA Veterans' and the Oslo study. "

    Oslo could not test the effect of fat modification because it is was heavily confounded. LA Veterans was also confounded but to a much lesser extent and hence a better test. However, LA Veterans did not show a significant effect on cardiovascular events and had virtually no effect on total mortality (an endpoint Dayton himself stressed) despite including far more deaths than Oslo. The most consistent explanation for the differing results of dietary trials is omega-3 (marine fatty acids especially as in Oslo), and trying to attribute any effect to saturated fat would put you in major inconsistencies.

    ReplyDelete
    Replies
    1. "The other possible explanation which is the default one is that the intervention is ineffective."

      ... except that there are solid explanations as to why the trials didn't show the indicated result but NO explanations to support you position. So if you argue for the latter option, you'll have to explain away the well-known mechanisms (LDL receptor functionality, HDL functionality, safa being more pro-inflammatory than unsaturated fats) and findings in e.g. metabolic ward studies that show precisely what is to be expected on the basis of these. And findings in clinical trials showing consistent benefits in lowering LDL levels.

      ... but you haven't even tried.

      "The burden is on you to prove that LDL per se is causal, not on me to disprove it (although there is a good argument against it)."

      Oh? Let's hear that good argument then. What the heck, LDL levels only satisfy modified Koch's postulates so clearly they don't matter.

      (BTW, if you're pushing for that Colpo strawman of "modified LDL", don't bother - it's compatible with current understanding and in no way indicates that LDL isn't the key factor)

      "Furthermore, lack of power is an unlikely explanation since effects from trials are consistently around 1 (as it is for virtually all cholesterol lowering treatments with regard to CHD/CV death and total mortality), and this is also reflected in meta-analyses involving hundreds or over 1000 deaths."

      Is "around 1" your way of ignoring both statistically and clinically significant findings? Or are you trying to ignore that even a minor risk reduction translates into thousands of CVD events LESS per year?

      In addition, do you even understand "power" here? Cf. drug trials which can be easily 10 times as big and last much longer than most of these studies? This becomes of course more relevant the shorter the studies are.

      "However, LA Veterans did not show a significant effect on cardiovascular events and had virtually no effect on total mortality (an endpoint Dayton himself stressed) despite including far more deaths than Oslo."

      Oops. You failed to mention that combined endpoints showed that the intervention group in LA Veterans had a statistically significant smaller risk of fatal CVD events. Just a coincidence, I'm sure. :-)

      And of course the fact that Oslo showed that a substantial reduction (about -17% in intervention vs. about -3% in control) in tot. chol. levels resulted to significant decrease in CHD - without ANY statistically/clinically significant changes in other well-known risk factors, e.g. blood pressure and smoking (when inspected individually) - is PURE coincidence too. Of course. :-)

      Seriously. Whenever evidence points to the contrary, you a) ignore it, b) refuse to discuss and c) resort to blind acceptance of findings in certain RCTs. Then when you're confronted, you simply return to point a) and repeat.

      Religion, plain and simple.

      Delete
  7. Thanks for the post... when i heard about the Meta-Analysis i was a little suspicious and it seems that i was kind of right.

    ReplyDelete

Post a Comment

Popular posts from this blog

Want To Buy: A Placebo

A well-designed/performed, double-blind, randomized, placebo-controlled trial provides a high level of certainty about the effectiveness of an intervention. In scientific training, the need to utilize a placebo relative to your variable of interest is one of the first things you learn when designing an experiment. As many in the basic sciences and evidence-based medicine fields have become more interested in nutrition and its impact on health/biology (their interest is well-justified), there has been insufficient appreciation for the difficulty in performing nutrition research. This day 1 principle of "placebo-controlled" poses a particular challenge for many nutrition experiments: there is no placebo.  Consider an example that actually plagued causal inference in nutrition history: It was known that feeding diets high in saturated fatty acids was associated with higher LDL. Does that mean that saturated fat raises LDL? How would you design a study to show

On PURE

The PURE macronutrients studies were published in the Lancet journals today and the headlines / commentaries are reminding us that everything we thought we think we were told we knew about nutrition is wrong/misguided, etc. Below is my non-epidemiologist's run down of what happened in PURE. A couple papers came out related to PURE, but the one causing the most buzz is the relationship of the macronutrients to mortality. With a median follow up of 7.4 years, 5796 people died and 4784 had a major cardiovascular event (stroke, MCI). The paper modeled the impacts of self reported dietary carbohydrate, total fat, protein, monounsaturated (MUFA), saturated (SFA), and polyunsaturated (PUFA) fatty acid intakes on cardiovascular (CVD), non-CVD and total mortality; all macros were represented as a percentage of total self reported energy intakes and reported/analyzed in quintiles (energy intakes between 500-5000kcals/day were considered plausible..). All dietary data was determined by a