Nutrition Science, Part III – The Awkward Fit: RCTs and Nutrition Science

In the second part of this article series we discussed the utility, limitations, and misconceptions related to the prospective cohort design for nutrition science. As discussed in part I, the reductionist biomedical model and its gold standard randomized controlled trial (RCT) is ill-equipped for studying complex dietary patterns in a way that can effectively inform public policy. In this article we will examine the randomized controlled nutrition trial design in more detail.

..there are fundamental differences between the subject of inquiry for which the RCT model evolved to investigate (drugs) and the subject of inquiry to which it is now applied (food), making its rigid application problematic for generating clear results

One issue that has come to trouble nutrition science is the “…almost exclusive reliance on the RCT as the only type of evidence worthy of consideration.” (30). This is troubling, because the RCT is considered the only research design that can demonstrate strong causality. However, as described in part two, there are fundamental differences between the subject of inquiry for which the RCT model evolved to investigate (drugs) and the subject of inquiry to which it is now applied (food), making its rigid application problematic for generating clear results. Let’s examine some of the pertinent differences between the study of drugs and nutrients.

Drugs are designed to have a targeted mechanism of action, pharmacologic profile, and a specific endpoint or outcome measure (1). In contrast, foods and overall diets are complex and polyvalent in characteristics, acting through various pathways across multiple body tissues (1). The effect size of a drug is often measurably large, while nutrients tend to have more subtle effects.

A salient feature of nutrient action is that it always occurs within the biological range, in contrast to the dosing of pharmacologic drugs (30). This is important because both baseline nutritional status and the dose of a nutrient used in a trial may influence results (31). For example, consider the likely results from a deficient study population receiving a low dose of a nutrient, a deficient population receiving a high dose of a nutrient, or a non-deficient population receiving low or high doses of the nutrient.

It is also important because the “signal-to-noise” ratio may not be as high as observed with drugs, potentially rendering effects more difficult to detect (30,32). Large effect sizes from drugs mean smaller numbers of subjects are required to adequately power a drug trial. On the other hand, nutrient trials may require larger numbers of subjects for adequate power, resulting in greater cost. Ultimately, many nutrition trials are small and subject to the same criticisms of a small sample size, and thus lack of statistical power.

Drugs are often designed for the treatment of specific pathologies, while nutrition is a more  dynamic interaction. Healthy people consume diets too, which may influence disease processes long before the onset of overt pathology. Biomedical trials are often more “acute” interventions, and shorter duration trials allow for greater methodological rigor and tighter control. Diet-disease interactions are typically chronic in nature, and study designs for nutrition with the highest methodological rigor (e.g., metabolic ward trials) are unsuited as a long-term investigative tool, and are wholly unreflective of free-living conditions.

It is important to note that these differences are not a failure of science. Science exists to test the truth of phenomena in our natural world. Where the method of inquiry fails to generate results, it is – or should be – modified (33). The RCT model is based off a presupposition that the method of investigation is fit for the purpose of investigating a certain treatment (32).

Assumptions under the RCT Model

There are three underlying assumptions that increase the internal validity and conclusions of the RCT model: 1) uniformity, 2) independence of effects, and 3) well-defined treatment and placebo (32). Thus, if these assumptions cannot be made for nutrition interventions, we can make a strong case that the RCT model should be modified for the purpose of nutrition research.

The first, uniformity, assumes that proper inclusion/exclusion criteria and randomization procedures render individuals in the different arms of a trial effectively uniform, and thus effectively “exchangeable” (32). The intent is that the results are generalizable for the group under study — particularly in the context of drug trials where the effect of the drug is likely to be substantially larger than any inter-individual variation (32). However, there are situations where the assumption of uniformity does not hold.

…if we are unable to meet the assumption of uniformity, the results of a trial therefore lack both internal and external validity.

For example, trials of specific nutrient supplementation in older subjects with disease cannot be generalized to support sustained consumption of the nutrient early in life (8). Further, even if appropriate uniformity is established at the start of a nutrition trial, subjects in the intervention arm often receive ongoing support, while the control arm may also modify diet, thus impairing uniformity during the trial (8,32). Finally, baseline nutritional status may also differ significantly between individuals, and since nutrients are polyvalent and act synergistically across many tissues, it is difficult to presuppose uniformity (1,32). Thus, if we are unable to meet the assumption of uniformity, the results of a trial therefore lack both internal and external validity (8).

A second critical assumption in any RCT is independence of effects: that is, that the results have been caused solely by the intervention, without any interactions (32). This may be an impossible presupposition for nutrition to meet, an argument supported by the large failure rate of trials with antioxidants (34).

For example, in a study assessing the antioxidant capacity of apples, it was found that apple skin contained a greater concentration of polyphenolic compounds than the flesh alone: apples with skin contained double the antioxidant capacity, and the antioxidant activity of the whole apple was attributed primarily to polyphenols and not the traditional antioxidant action of vitamin C, despite containing high levels of vitamin C (35).

Similarly, almonds provide a rich dietary source of the fat-soluble antioxidant vitamin E, with 1 oz providing up to 7 mg. In addition, almonds are a source of monounsaturated fats and the presence of vitamin E, which protects unsaturated fats against oxidation, is an example of the nutrient composition of a food reflecting its biological makeup (36). In a population regularly consuming nuts, is an observed benefit associated with unsaturated fats or vitamin E? Is the benefit to do with effects on blood cholesterol from vitamin E, or blood pressure from monounsaturated fats?

Given the polyvalent nature of a food matrix encompassing multiple compounds at a macronutrient, micronutrient, and bioactive food component level, together with the multiplicity of biological actions, it is not possible to know every potential effect-modifying variable.

Biomedical purists arguing that the only design which should be employed in nutrition research are large, long-term, simple RCT’s seem to have derived this conclusion without accounting for the assumptions upon which the validity of RCTs is predicated: causal inference is not valid where an effect depends on the presence of uncontrolled additional variables (32). Given the polyvalent nature of a food matrix encompassing multiple compounds at a macronutrient, micronutrient, and bioactive food component level, together with the multiplicity of biological actions, it is not possible to know every potential effect-modifying variable (1,32). The implication is that no valid conclusion can be made from a nutrition RCT, as independence of effects cannot be assumed.

Thirdly, RCTs in the biomedical model presuppose that the intervention is well-defined, and is the sole cause of an observed effect compared to a placebo of “zero exposure” (6,32). This presents a dilemma for the study of nutrition. In a drug trial, it is desirable to eliminate all co-therapies to avoid confounding, while in nutrition the effect of a total diet is contingent on “co-therapies”, if we consider nutrients as such (1). Indeed, this is something to optimise in a whole diet (1). There is no effect of any nutrient in an absolute sense, and it is more likely the cumulative and interactive effects of multiple constituents of diet coalesce to influence health and disease (11,12,24,37). For example, the impact of calcium on promoting bone mineral density is influenced not only by vitamin D status, but also by dietary protein intake: higher levels of both the latter increases uptake of the former (38,39). Thus, having a well-defined intervention is not always possible.

There is no effect of any nutrient in an absolute sense, and it is more likely the cumulative and interactive effects of multiple constituents of diet coalesce to influence health and disease.

More problematic for this presupposition for the validity of an RCT, however, is the fact that there is no placebo for food. Everyone eats food, and there is no ‘nutrient-free state’ which can be applied as a true placebo (1). Consequently, nutrition trials tend to compare high to low intakes, however, this gives rise to ethical difficulties in that a true comparison of the effect of a nutrient would require giving a nutritionally inadequate level of a nutrient to the “control” group (1). The lack of defined placebo for food, and the difficulty in rendering dietary exposures into well-defined interventions, undermines this assumption for internal validity.

The critical feature of the assumptions underscoring the RCT design is that if any are untrue, the conclusions of the trial are invalid (32). This, coupled with the aforementioned differences between the nature of drugs compared to foods, indicates that the biomedical RCT is not a panacea for nutrition research (8). In fact, it appears to be unfit for the purpose, and to blindly insist on its application to nutrition is dogma.

In the next article in this series, we’ll take a look at how to improve the approach to nutrition research.

References

1. Heaney R. Nutrients, Endpoints, and the Problem of Proof. The Journal of Nutrition. 2008;138(9):1591-1595.
2. Vandenbroucke J. Adolphe Vorderman’s 1897 study on beriberi: an example of scrupulous efforts to avoid bias. Journal of the Royal Society of Medicine. 2013;106(3):108-111.
3. Mozaffarian D. Dietary and Policy Priorities for Cardiovascular Disease, Diabetes, and Obesity. Circulation. 2016;133(2):187-225.
4. Messina M, Lampe J, Birt D, Appel L, Pivonka E, Berry B et al. Reductionism and the Narrowing Nutrition Perspective. Journal of the American Dietetic Association. 2001;101(12):1416-1419.
5. Trepanowski J, Ionnidis J. Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How. Advances in Nutrition. 2018;Jul 1;9(4):367-377.
6. Satija A, Stampfer M, Rimm E, Willett W, Hu F. Perspective: Are Large, Simple Trials the Solution for Nutrition Research?. Advances in Nutrition. 2018;9(4):378-387.
7. Satija A, Yu E, Willett W, Hu F. Understanding Nutritional Epidemiology and Its Role in Policy. Advances in Nutrition. 2015;6(1):5-18.
8. Hébert J, Frongillo E, Adams S, Turner-McGrievy G, Hurley T, Miller D et al. Perspective: Randomized Controlled Trials Are Not a Panacea for Diet-Related Research. Advances in Nutrition. 2016;7(3):423-432.
9. Maki K, Slavin J, Rains T, Kris-Etherton P. Limitations of Observational Evidence: Implications for Evidence-Based Dietary Recommendations. Advances in Nutrition. 2014;5(1):7-15.
10. Willett W. Nutritional epidemiology. New York [etc.]: Oxford University Press; 2013.
11. Jacobs D, Steffen L. Nutrients, foods, and dietary patterns as exposures in research: a framework for food synergy. The American Journal of Clinical Nutrition. 2003;78(3):508S-513S.
12. Tapsell L, Neale E, Satija A, Hu F. Foods, Nutrients, and Dietary Patterns: Interconnections and Implications for Dietary Guidelines. Advances in Nutrition. 2016;7(3):445-454.
13. JAIN M, HOWE G, HARRISON L, MILLER A. A Study of Repeatability of Dietary Data Over A Seven-Year Period. American Journal of Epidemiology. 1989;129(2):422-429.
14. Post G, Vente W, Kemper H, Twisk J. Longitudinal trends in and tracking of energy and nutrient intake over 20 years in a Dutch cohort of men and women between 13 and 33 years of age: The Amsterdam growth and health longitudinal study. British Journal of Nutrition. 2001;85(03):375.
15. Mikkilä V, Räsänen L, Raitakari O, Marniemi J, Pietinen P, Rönnemaa T et al. Major dietary patterns and cardiovascular risk factors from childhood to adulthood. The Cardiovascular Risk in Young Finns Study. British Journal of Nutrition. 2007;98(01):218.
16. Joshipura K, Ascherio A, Manson J, Stampfer M, Rimm E, Rimm E et al. Fruit and Vegetable Intake in Relation to Risk of Ischemic Stroke. JAMA. 1999;282(13):1233.
17. Hooper L, Martin N, Abdelhamid A, Davey Smith G. Reduction in saturated fat intake for cardiovascular disease. Cochrane Database of Systematic Reviews. 2015;.
18. Mozaffarian D, Micha R, Wallace S. Effects on Coronary Heart Disease of Increasing Polyunsaturated Fat in Place of Saturated Fat: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. PLoS Medicine. 2010;7(3):e1000252.
19. Dehghan M, Mente A, Zhang X, Swaminathan S, Li W. Associations of fats and carbohydrate intake with cardiovascular disease and mortality in 18 countries from five continents (PURE): a prospective cohort study. Lancet. 2017;390:2050–62.
20. Wallström P, Sonestedt E, Hlebowicz J, Ericson U, Drake I, Persson M et al. Dietary Fiber and Saturated Fat Intake Associations with Cardiovascular Disease Differ by Sex in the Malmö Diet and Cancer Cohort: A Prospective Study. PLoS ONE. 2012;7(2):e31637.
21. Li Y, Hruby A, Bernstein A, Ley S, Wang D, Chiuve S et al. Saturated Fats Compared With Unsaturated Fats and Sources of Carbohydrates in Relation to Risk of Coronary Heart Disease. Journal of the American College of Cardiology. 2015;66(14):1538-1548.
22. Barnard N, Willett W, Ding E. The Misuse of Meta-analysis in Nutrition Research. JAMA. 2017;318(15):1435.
23. Hernan M, Robbins J. Estimating causal effects from epidemiological data. Journal of Epidemiology & Community Health. 2006;60(7):578-586.
24. Hill A. The environment and disease: association or causation?. Journal of the Royal Society of Medicine. 1965;58(5):295-300.
25. Aune D, Keum N, Giovannucci E, Fadnes L, Boffetta P, Greenwood D et al. Whole grain consumption and risk of cardiovascular disease, cancer, and all cause and cause specific mortality: systematic review and dose-response meta-analysis of prospective studies. BMJ. 2016;:i2716.
26. Malik V, Hu F. Sweeteners and Risk of Obesity and Type 2 Diabetes: The Role of Sugar-Sweetened Beverages. Current Diabetes Reports. 2012;12(2):195-203.
27. Mozaffarian D, Aro A, Willett W. Health effects of trans-fatty acids: experimental and observational evidence. European Journal of Clinical Nutrition. 2009;63(S2):S5-S21.
28. Mozaffarian D, Clarke R. Quantitative effects on cardiovascular risk factors and coronary heart disease risk of replacing partially hydrogenated vegetable oils with other fats and oils. European Journal of Clinical Nutrition. 2009;63(S2):S22-S33.
29. World Health Organisation. Diet, Nutrition and the Prevention of Chronic Diseases. Geneva: Joint WHO/FAO Expert Consultation; 2003.
30. Blumberg J, Heaney R, Huncharek M, Scholl T, Stampfer M, Vieth R et al. Evidence-based criteria in the nutritional context. Nutrition Reviews. 2010;68(8):478-484.
31. Morris M, Tangey C. A Potential Design Flaw of Randomized Trials of Vitamin Supplements. JAMA. 2011;305(13):1348.
32. Zeilstra D, Younes J, Brummer R, Kleerebezem M. Perspective: Fundamental Limitations of the Randomized Controlled Trial Method in Nutritional Research: The Example of Probiotics. Advances in Nutrition. 2018;9(5):561-571.
33. Engel G. The need for a new medical model: a challenge for biomedicine. Science. 1977;196(4286):129-136.
34. Steinhubl S. Why Have Antioxidants Failed in Clinical Trials?. The American Journal of Cardiology. 2008;101(10):S14-S19.
35. Eberhardt M, Lee C, Liu R. Antioxidant activity of fresh apples. Nature. 2000;405(6789):903-904.
36. Jacobs D, Gross M, Tapsell L. Food synergy: an operational concept for understanding nutrition. The American Journal of Clinical Nutrition. 2009;89(5):1543S-1548S.
37. Cespedes E, Hu F. Dietary patterns: from nutritional epidemiologic analysis to national guidelines. The American Journal of Clinical Nutrition. 2015;101(5):899-900.
38. Feldman D, Pike J, Bouillon R. Vitamin D. 2nd ed. San Diego: Academic Press; 2005.
39. Dawson-Hughes B, Harris S. Calcium intake influences the association of protein intake with rates of bone loss in elderly men and women. The American Journal of Clinical Nutrition. 2002;75(4):773-779