Never trust, always verify... even your allies
Never trust, always verify... even your allies
Practices for ending the dark age
This is a difficult post for me to write. I've thought long and hard about how to write it, and whether I should write it at all. Some people are going to be offended by it, because I'm critiquing people they admire and challenging ideas they believe in; others will no doubt accuse me of sowing division within the ranks of the 'COVID dissidents'.
So be it.
Given that I've spent the last few weeks delving into Steve Patterson's dark age thesis, discussing the factors that lead societies into dark ages, and conversely the attitudes and practices that mitigate against the proliferation of "shockingly bad ideas [that] become orthodoxy", I feel morally obliged to put my money where my mouth is. In last week's post, I shared some of Richard Feynman's thoughts on the mindset of the true scientist: namely, that he or she relentlessly attacks every hypothesis from every conceivable angle, to find its weaknesses and either modify it to strengthen its predictive value, or discard it if it is not fit for purpose.
As I have noted previously, this mindset is fundamentally unnatural to human beings. We like to be right; we prefer to win arguments than lose them; we value displaying loyalty to our friends and allies over honestly telling them that their opponent's arguments were stronger than theirs. But if we want to develop high-quality ideas that will help to pull us out of the present dark age, and give us the best chance of co-creating institutions and practices that promote human flourishing, we need to consciously build our mental muscles to overcome the sloppy thinking habits that produce bad ideas. As a practitioner, an educator, and a writer, I aim to help my clients, students and readers to develop rigorous thinking and research habits, while I hone my own.
With all that said, the impetus for this article was a post by the author of the The Forgotten Side of Medicine Substack, who goes by the pen-name A Midwestern Doctor. I have subscribed to this stack for quite some time, and find A Midwestern Doctor (henceforth abbreviated to AMD) to be exceptionally insightful. We have previously had productive interactions in the Comments section of both our stacks. However, as is inevitably the case, we have divergences of opinion on certain matters.
AMD's post, What Can Statins Teach Us About The COVID-19 Vaccines?, drew parallels between the behaviour of the medical-pharmaceutical-industrial complex with respect to the prescription of statins, and to COVID-19 injections. It made many excellent points about the dangers of statin drugs, and their lack of benefit for the vast majority of patients to whom they're prescribed, with which I wholeheartedly agree. (So does one of my favourite websites for quickly assessing whether the benefits of a medical intervention outweigh its risks, The NNT; check out their assessments of statins for persons in various risk categories here.)
However, I did notice a couple of statements which I judged to be questionable, and since AMD had explicitly stated his willingness to engage with commenters who disputed his views, I posted four statements that he made, that I judge to be false or misleading, and presented evidence against them. His response was not what I had anticipated, which disappointed me. I'm not going to dwell on that, because the purpose of this post is not to diss AMD, whom I respect despite this rather unpleasant interaction.
At this point, another reader of The Forgotten Side of Medicine, who goes by the marvellous moniker of
, weighed in with the following thoughtful comment, to which I have added a few quotation marks, for clarity (note that AMD had referred in his comments to the writings of Dr Malcolm Kendrick):"I'm just a layman, but one who's read all of Kendrick and a good bit more. I observe on both sides of the current exchange there's a good bit of professional jealousy in plain evidence. A problem all people have is a strong attachment to beliefs; they are not easily changed even when presented with conflicting evidence. Being a professional doesn't exempt one from such traits. This is even more a problem when evidence may point in different directions, as is surely the case with something as complex as nutrition and illness.
Nevertheless, I can offer Chuter some citations from Kendrick. You are free to research his sources as you see fit.
'Seven Countries Study' has 11 hits in Kendrick’s earliest work, 'The Great Cholesterol Con.'
'…Ancel Keys, and his famous Seven Countries Study. Keys looked at saturated-fat consumption in seven countries and found a straight-line relationship between heart disease, cholesterol levels and saturated-fat intake. The seven countries were:
• Italy
• Greece
• Former Yugoslavia
• Netherlands
• Finland
• USA
• Japan
Why these particular seven countries? He could have chosen another seven and demonstrated the exact opposite. Here are my seven countries:
• Finland [yes, I just noticed it's in both lists; but I confirmed that is how it appears in the ebook.]
• Israel
• Netherlands [my editorial insertion: again, Netherlands appears in both of Kendrick’s lists]
• Germany
• Switzerland
• France
• Sweden
What do you mean I can’t choose my own countries? That’s not fair. Keys did.'
(GCC, Ch. 5)
In 'Clot Thickens' Keys comes in for additional dressing-down. 'Have you ever heard of the Minnesota Coronary Study or the Sydney Diet Heart Study? Most haven’t. Both date to the early 1970s and were not published for decades. They only saw the light of day due to dogged investigators finding the original data. Both studies, done to high standards, showed that replacing saturated fat with unsaturated led to worse outcomes. And guess how [sic] was the lead on the former study?
I think it is important to mention that the lead investigator of the MCE study was the one and only Ancel Keys. I wonder why the study was not published…'
Ramsden et al. Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73). BMJ. 2016. https://www.bmj.com/content/353/bmj.i1246
For my own amusement, I discovered there is a web site called https://www.sevencountriesstudy.com/
Now, here I admit that I’ve done little exploration of this site’s underlying claims. I’m just an outsider. But the study began in 1956 and has continued to the present day. OK, it’s one of the very few long-term studies; surely it deserves its own web site. But recall the death-grip attachment to professional status and sincerely held beliefs I spoke of earlier? I suspect that those are a major factor here. Not to mention well over half a century of funding: salaries, probably 'contributions' from industries with vested interests, reputations at stake, and so on. My point is that only a fool would expect scientific impartiality at this site (or Kendrick’s book for that matter.) A truly impartial observer with no dog in the fight – which, if one is honest, is a pretty darned rare occurrence – would begin with the assumption that everybody has a hidden agenda and will skew the 'facts' to suit his own perspective. It’s necessary to weigh all available evidence on its own merits. There's a lot of money riding on the issue and who's right, what's true and what's not; isn't it reasonable to be just the least bit skeptical of any and all claims?
Anyway, hope I’ve been helpful. If you’re really curious about the anti-Keys arguments, Kendrick’s books are a good starting place."
I did indeed find Satan's Doorknob's comment helpful, because it, and the quotes from Malcolm Kendrick's books, confirmed my suspicions about the source of some of the factual errors in AMD's post on statins.
I replied thus:
"Thank you for contributing, Mr Doorknob (and I hope I haven't misgendered you). This is exactly the kind of calm, rational discussion of evidence, claims and counterclaims that I was hoping to provoke with my comment.
Your first excerpt from Malcolm Kendrick's books is the perfect illustration of the game of Chinese whispers that has been played with Ancel Keys' work, starting with a book written by Gary Taubes in 2007, called Good Calories, Bad Calories, in which Taubes conflated the Six Countries Study with the Seven Countries Study. The errors that Taubes made were then rehashed by Nina Teicholz, and since then have been repeated on countless blogs and popular books. Malcolm Kendrick has unfortunately rehashed this exact same argument in his book, despite it being previously dismantled by several authors.
You can read the full (and very interesting) story in a white paper titled 'Ancel Keys and the Seven Countries Study: An Evidence-based Response to Revisionist Histories', but here is the brief version, which I'll break up into separate answers for easier readability, and because I probably won't be able to address all your points in one sitting."
Before I go any further, I want to stress that Dr Malcolm Kendrick has been a consistent voice of reason throughout the entire manufactured COVID crisis, and his blog is well worth reading. But the fact that - after delving deep into the evidence - I agree with him on the damage wrought by bad COVID policy and dangerous medicines, doesn't mean that I do, or should, take everything he says on nutrition and cardiovascular disease as gospel truth. Instead, as Feynman urged, I subject his claims to the same blowtorch as anyone else's.
That's what I did in reply to Satan's Doorknob, and you can read the comment thread in its original form here. In this post, I'm presenting the arguments that I made in an easier-to-read format, with some light editing for clarity, some additional points that I neglected to include, and with the benefit of some images that I couldn't embed in the Substack comment section.
I’ll pick out each of Malcolm Kendrick's quotes from Satan's Doorknob's comment, and then give my response.
Quote #1 (source):
"…Ancel Keys, and his famous Seven Countries Study. Keys looked at saturated-fat consumption in seven countries and found a straight-line relationship between heart disease, cholesterol levels and saturated-fat intake. The seven countries were:
• Italy
• Greece
• Former Yugoslavia
• Netherlands
• Finland
• USA
• Japan
Why these particular seven countries? He could have chosen another seven and demonstrated the exact opposite. Here are my seven countries:
• Finland [yes, I just noticed it's in both lists; but I confirmed that is how it appears in the ebook.]
• Israel
• Netherlands
• Germany
• Switzerland
• France
• Sweden
What do you mean I can’t choose my own countries? That’s not fair. Keys did."
My response
1. The persistent conflation of the six countries graph and the Seven Countries Study
In 1953, Keys published a paper titled 'Atherosclerosis: a problem in newer public health', which you can read here. On p. 17 of this 22 page paper, which presented a wide assortment of comparative data on heart disease in various countries, there is a graph representing deaths from 'degenerative heart disease' plotted against the percentage of calories derived from fat (total fat, without regard to saturated vs unsaturated):
There are six countries on that graph - Japan, Italy, England/Wales, Australia, Canada and the US and hence, it is often described as the 'Six Countries Study', even though it was barely more than a sketch of ecological data and certainly does not qualify as a 'study'. These six countries were selected because they collected both dietary data and vital statistics in comparable ways (this was way before ICD codes standardised death certificate coding throughout the world). The dietary data that Keys used to produce this graph were national food balance data from 1949, supplied by the Food and Agriculture Association (FAO). Post-war food rationing was still in place in some of these countries, which is important because several researchers had noted a drop in coronary heart disease deaths during WWII, which correlated with food rationing and/or famines and blockades induced by hostilities.
It's important to point out that this was an ecological analysis, that is, a comparison of disease rates in different locations at a single time-point with a potential causative factor, which is considered a hypothesis-generating exercise. This is exactly how Keys presented it in his paper, as you can clearly see by reading the text before and after figure 2 on p. 17.
In 1957, Yerushalmy and Hilleboe published a critique of Keys' 1953 paper, 'Fat in the diet and mortality from heart disease; a methodologic note' in which they accused Keys of selecting countries with data that supported his hypothesis and ignoring countries whose data didn't support it. They claimed that data from 22 countries was available, and presented those data in this graph:
Among the countries which Yerushalmy and Hilleboe claimed Keys had intentionally excluded were Finland, Israel, Netherlands, Germany, Switzerland, France and Sweden, which is what makes it clear to me that Kendrick is drawing on the Taubes/Teicholz trope - that is, conflating the six countries graph with the Seven Countries Study - rather than "choosing his own countries", whether or not he acknowledges this in his book.
However, the data used by Yerushalmy and Hilleboe were from 1951-53, i.e. several years after the data Keys used, by which time post-war food restrictions were largely over. There are other reasons why Keys used the data from the six countries to produce his graph, which are discussed at length in pp. 24-29 of Ancel Keys and the Seven Countries Study: An Evidence-based Response to Revisionist Histories, so I won't clutter up this discussion with these additional arguments. I encourage you to read them and decide for yourself whether Keys' rationale was legitimate.
Furthermore, Kendrick is incorrect in stating that if data from Israel, the Netherlands, Germany, Switzerland, France and Sweden were included on Keys' graph, they would show the opposite relationship between dietary fat intake and heart disease to the one Keys proposed. There is still a positive correlation between these two factors, but it is weaker than the one Keys graphed from the data available to him in 1953.
2. The Seven Countries Study
Unlike the simple hypothesis-generating exercise of 1953, the Seven Countries Study (which included Italy, Greece, Yugoslavia, Netherlands, Finland, USA and Japan, while the six countries graph included Japan, Italy, England/Wales, Australia, Canada and the US) was, as Satan's Doorknob noted, a long-running prospective multi-country cohort study.
Keys' team at the University of Minnesota established strict protocols to ensure reliable and consistent data, trained the researchers from each country in these protocols, and then sent them back to their home countries to conduct the data collection.
The description of the conduct of the Seven Countries Study beginning on p. 9 of Ancel Keys and the Seven Countries Study: An Evidence-based Response to Revisionist Histories gives a good sense of the scope, magnitude and thoroughness of the process: teams of researchers in the field conducted measurements of blood cholesterol, blood pressure and BMI; performed electrocardiograms and glucose tolerance tests; took medical, smoking and physical activity histories; collected seven-day diet recall diaries backed up with seven-day weighed food records, and even conducted chemical analysis of foods eaten by participants. Strict diagnostic coding for disease events and cause of death was applied to make sure that results from each study centre were comparable.
Unfortunately, the US cohort of the study was discontinued early because of lack of funding, a problem which plagued the researchers throughout planning and implementation. Contrary to the (perfectly reasonable) speculation of Satan's Doorknob, there were no industries keen to fund this study.
The Seven Countries Study has generated hundreds of scientific publications, which you can browse here.
Quote #2:
"Have you ever heard of the Minnesota Coronary Study or the Sydney Diet Heart Study? Most haven’t. Both date to the early 1970s and were not published for decades. They only saw the light of day due to dogged investigators finding the original data. Both studies, done to high standards, showed that replacing saturated fat with unsaturated led to worse outcomes. And guess how [sic] was the lead on the former study?
I think it is important to mention that the lead investigator of the MCE study was the one and only Ancel Keys. I wonder why the study was not published."
My response
The Minnesota Coronary Experiment was originally intended to be a long-running intervention trial, in which dietary saturated fats (primarily from animal products) were almost entirely replaced with linoleic acid-rich corn oil. It enrolled residents of one nursing home and six state mental hospitals (9423 participants in total), a number close to the 10 000 participants that investigators had calculated would be required, in order to reach statistical significance.
However, the deinstitionalisation movement which resulted in the closure of many mental hospitals and discharge of long-term patients into the community, caused the failure of this study; almost three quarters of participants were lost within the first year and only about half of the remaining patients remained enrolled in the study for a full three years.
With so few participants left, the study had no hope of reaching statistical significance. However, Kendrick is incorrect in claiming that the results of the Minnesota Coronary Experiment did not "see the light of day" until the BMJ report published in 2016; they were indeed published in 1989 in a paper titled 'Test of effect of lipid lowering by diet on cardiovascular risk. The Minnesota Coronary Survey', with frank acknowledgement that the study failed to show benefit of the intervention for cardiovascular events, cardiovascular deaths, or total mortality.
Kendrick is also incorrect in stating that Ancel Keys was the lead investigator; the BMJ article states that Ivan Frantz was the principal investigator, and Frantz was the lead author of the 1989 paper, while Keys had no authorship credit.
Neither I, nor Malcolm Kendrick, nor anybody else knows why the additional data contained in the 2016 BMJ report, 'Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73)' were not published, because no reason was ever given by the investigators, to the best of my knowledge. Kendrick's obvious implication that Keys suppressed publication of these data is, as far as I know, completely unfounded.
Kendrick is also incorrect in stating that "replacing saturated fat with unsaturated led to worse outcomes" in this study. The BMJ article states that there was "no mortality benefit for the intervention group in the full randomized cohort or for any prespecified subgroup". In other words, there was no difference in outcomes. The finding of "22% higher risk of death for each 30 mg/dL (0.78 mmol/L) reduction in serum cholesterol" applied equally to both the control and intervention groups, as is made clear on p. 9:
"In survival analyses (table 4), there was a robust association between decreasing serum cholesterol and increased risk of death, and this association did not differ between the intervention and control group (P>0.16 for all serum cholesterol×intervention interactions)." [my emphasis]
In other words, participants whose cholesterol levels fell the most over the course of the study were the most likely to die, regardless of whether they were in the dietary intervention group or the control group. That's intriguing! Let's read on:
"Among both groups combined, a 30 mg/dL (0.78 mmol/L) decrease in serum cholesterol was associated with 22% higher risk of death from any cause (hazard ratio 1.22, 95% confidence interval 1.14 to 1.32) based on a Cox model adjusted for baseline serum cholesterol, age, sex, adherence to diet, BMI, and systolic blood pressure. The higher risk of death associated with decreased serum cholesterol seems to be driven by the subgroup aged ≥65. Among participants who were older than 65 at baseline, a 30 mg/dL decrease in serum cholesterol was associated with 35% higher risk of death (hazard ratio 1.35, 95% confidence interval 1.18 to 1.54), whereas among people aged under 65 at baseline there was no relation between the change in serum cholesterol and death (1.01, 0.88 to 1.16)."
The most plausible explanation for the association between declining cholesterol and increased risk of death, in older people but not in younger people, is most likely the terminal decline in cholesterol that occurs as death approaches. A study called 'Trajectory of Total Cholesterol in the Last Years of Life Over Age 80 Years: Cohort Study of 99,758 Participants' found that elderly people with the lowest cholesterol levels were the most likely to die within the study period, regardless of whether they were treated with statins or not (and in fact those with declining cholesterol levels despite NOT treated with statins were the most likely to die):
"Compared with TC [total cholesterol] values of 4.5-5.4 mmol/L, TC values <3.0 mmol/L were associated with higher mortality (statin treated hazard ratios 1.53, 95% confidence interval 1.43-1.64, p < .001; not treated, 1.41, 1.29-1.54, p < .001). A secular decline in TC values accelerated in the last 2 years of life. In the last quarter of follow-up, the adjusted odds of TC < 3.0 mmol/L for those who died, compared with surviving participants, were 3.33 (2.84-3.91, p < .001) for untreated and 1.88 (1.68-2.11, p < .001) for statin-treated participants."
"Conclusions: TC values show a terminal decline in the last years of life. Reverse causation may contribute to the association of lower TC with higher mortality in nonrandomized studies."
Reverse causation means (in this case) that declining cholesterol is the consequence of the deteriorating function that leads to death, rather than the cause of death. Anaemia, chronic inflammation, chronic renal and adrenal failure, nutrient deficiencies and hyperthyroidism (all common conditions in the final stages of life) all lead to reduced cholesterol production. Cancer, a leading cause of death, is also associated with reduced serum cholesterol, but cancer cells require cholesterol for growth and hence the relationship between low serum cholesterol levels and cancer is also attributable primarily to reverse causation.
The final comment I would make on the Minnesota Coronary Experiment is that this study used an intervention (replacement of saturated fats with corn oil, much of it in foods that contained high levels of trans fats which are now known to be the most dangerous type of fats with regards to coronary artery disease) that is not advocated by any school of nutrition thought these days, so it has essentially no relevance to the modern 'diet wars'.
What are the implications of these errors in Kendrick's books?
Do these errors and flaws in Kendrick's works invalidate his entire hypothesis regarding the cause of coronary artery disease? No, of course not. Like any of us, he can be wrong about some things, and right about others. Likewise, the fact that Kendrick makes incorrect statements about Ancel Keys doesn't mean that Keys was a saint or that everything he ever said or wrote should be read as holy writ. At least, though, Keys appears to have displayed the characteristics lauded by Richard Feynman:
"According to colleagues, however, Keys routinely did with such observations [of an association between total dietary fat intake and heart disease] what good scientists should do: turned them into testable hypotheses, not immutable convictions. He proceeded to test this hypothesis himself, and concluded long before the dawn of the ‘low fat’ diet era that total dietary fat quantity was unimportant, while the sources and quality of fat were important.... the written record and other first-hand accounts indicate that (a) Keys almost without fail carefully avoided any temptation to overstate the significance of his findings in published work; and (b) he based his impressions/conclusions on the overall mass of evidence, not just the SCS [Seven Countries Study]. Importantly, his views evolved over the course of his career in accord with the evolving evidence base." [my emphasis]
Ancel Keys and the Seven Countries Study: An Evidence-based Response to Revisionist Histories
Conversely, when I observe poor research techniques, misinterpretations of study findings, and copious use of informal logical fallacies - including ad hominem, straw man, neglected aspects and post hoc - then I know that I need to proceed with even more caution than I would usually employ when assessing a source of information. Kendrick's misstatements relating to Ancel Keys alert me that every statement that he makes has to be double-checked, and every source that he cites has to be read carefully in order to verify that it says what he claims it does.
As you have probably noticed, I am quite obsessive when it comes to checking sources and triangulating information. My own Substack articles are all densely referenced, because I want my readers to be able to check the sources of information that I'm drawing on, so that they can decide for themselves whether my interpretation is correct or not. I find it very frustrating when other authors don't do this, but instead simply make statements without providing me with any means by which I can verify them.
The take-home message
If we're going to drag ourselves out of the current dark age, we need to move past both our own egoic attachment to cherished ideas, and the primitive impulse to divide down tribal lines of those whose ideas we identify with vs those whose ideas are anathema to us.
All ideas, purported facts, hypotheses and theories need to subjected to rigorous interrogation. It's a good practice to regularly expose yourself to ideas with which you disagree, so you can challenge your own beliefs and understanding. You should also subject the ideas with which you agree to scrutiny, by searching for facts that contradict them, or alternative explanations.
That goes for my articles, by the way. I encourage you to let me know if you think I've got something wrong. Click on the links to source material that I provide, and decide for yourself whether I'm representing it accurately. Of course, I expect you to present your disputation calmly and with supporting evidence. We need to help each other to practise become more skilled, inquiring, critical thinkers, so we can generate the high-quality ideas necessary for us to thrive... and perhaps, even to survive the unprecedented challenges that we currently face.
Outstanding. I like this point especially. "All ideas, purported facts, hypotheses and theories need to subjected to rigorous interrogation. It's a good practice to regularly expose yourself to ideas with which you disagree, so you can challenge your own beliefs and understanding. You should also subject the ideas with which you agree to scrutiny by searching for facts that contradict them, or alternative explanations." I think that human pride can swell like a toad. I think group think is much stronger through time than we wish to believe. When tremendous sums of money align with friendly scientific interests frequently bad drives out good thinking. And sticks around badly training the next generation into thinking truth is not as important as naked self interest -strictly in a professional way of course this following the leader of the pack.
good article.
Yes, always question, because assumptions are the mother of all Fups.
And yes, we all make mistakes, but we all need humility to be able to accept this, to fix the mistake, and to ensure similar behaviour does not happen again.