We're Computerizing and Don't Need You

On the good intentions and bizarre conclusions of evidence-based medicine

There is a tempting story you can tell about the cardiology megatrials I’ve written about in the past few posts. If you focus on a single health outcome (say, mortality) and a single disease (say, infarctus du myocarde, aka heart attack), you can optimize the treatment program by continually running randomized clinical trials. Each trial will carefully compare two nearly identical treatment regimens that differ only in a single step. Professional societies will regularly communicate the latest results and guidelines. Piecemeal improvements to the standard of care will be broadcast at meetings and on websites to keep all clinicians up to date on the best plan for every patient.

This approach can be applied to multiple diseases. With enough buy-in, these trials can be scaled up to provide personalized adjustments to the standard of care. Patients’ demographic information, family history, and behaviors can then be considered, and professional societies can craft specialized plans for each demographic bucket.

Now, statistical calculations would quickly tell you that you need far more than 8 billion people to make this optimized dream a reality. But let’s leave that aside for a minute. There’s something bizarre and alienating about this view. In this pipe dream, care becomes nothing more than a computer algorithm. A patient becomes nothing more than a classification assignment. Healthcare becomes nothing more than optimizing actuarial tables. Is this dignified? Is this care? Is this what we want the medical system to be?

There was a reactionary and revolutionary movement in the 1990s that thought yes, healthcare should be nothing more than the application of clinical data: evidence-based medicine.

EBM remains incredibly influential. I mean, who wants medicine that isn’t based on evidence, right? Back in 2020, I also had a brief moment when I thought this was a brilliant way to conceptualize medicine. For mathematically minded people, EBM has the appeal of rigor. It provides what appears to be a set of clear rules for deciding on medical treatment. There is a hierarchy of evidence. A pyramid, if you will:

Expert opinion is the lowest form of evidence. Case studies are barely better. Epidemiological studies would be of slightly higher grade, but these are too easily fooled by confounding causes. A big step above, a gold standard, is the randomized controlled trial. And at the very top, we place an even higher grade on systematic reviews of all randomized controlled trials conducted on a disease.¹

Once you have this evidence in hand, you can make clear decisions. You don’t need expertise. You simply need a Meehlian formula: you plug all of the data you have about a patient into a computer. The computer spits out a treatment plan for the patient based on the highest grade evidence available. Healthcare is now solved.

You might think I’m caricaturing their position, but you can go back to the original position papers, and the quotes are even stronger than what I say. Here’s the opening paragraph of “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine,” written by a working group chaired by EBM pioneer Gordan Guyatt:

“A new paradigm for medical practice is emerging. Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research. Evidence-based medicine requires new skills of the physician, including efficient literature searching and the application of formal rules of evidence evaluating the clinical literature.”

The working group was explicit about expert judgment: “The new paradigm puts a much lower value on authority. The underlying belief is that physicians can gain the skills to make independent assessments of evidence and thus evaluate the credibility of opinions being offered by experts.” In their view, expert intuition and creative care caused more harm than good. Moreover, they thought a move to computerized decision making would improve patient outcomes, writing “A final assumption of the new paradigm is that physicians whose practice is based on an understanding of the underlying evidence will provide superior patient care.”

Computerization was exactly what they had in mind. David Sackett put together a prototype treatment computer, called The Evidence Cart, in the early nineties. Check him out making a diagnosis:

Sackett stuffed the evidence cart with a compendium of information: BestEvidence, the JAMA Rational Clinical Examination series, the Cochrane Library, multiple textbooks, and material compiled by his collaborators from journal groups and other assessments. Sackett’s crew would wheel the Evidence Cart into a patient room and evaluate each decision against the best evidence.

Did it work? Hilariously, the evidence that the Evidence Cart works is a tiny, single-center observational study, not an RCT. But the influence of EBM and the goal of making Sackett’s dream a reality was undeniable. Doctors now have supercomputers in their pockets at all times. They can pull out medical calculators, reduce patients to categories, and prescribe treatment plans based on the best evidence. Every doctor-patient visit is hooked up to a different computer that serves not only as a way for doctors to keep records of how their patients are doing, but also as a way to file them as statistics in actuarial tables at insurance companies and to enter their virtualized identity into the pool of future observational studies.

Has healthcare landed in a good spot? It’s complicated. Obviously, studies are good. But the idea that you can synthesize an optimal decision from the medical literature is crazy, whether you have modern LLMs or not. You can’t do an RCT of every possible option. We’ve already seen that expert knowledge is needed to decide which RCTs to do and to make sense of RCTs that seemingly contradict each other. And the medical literature does not tell you what to do with an individual patient. Patients are not just category labels. Care is more than robotic decision-making.

I’m more sympathetic to the iatrogenic argument put forward by the EBM pioneers. Many were medical conservatives. They thought that most treatments not only lacked evidence but were also harmful. You can look at the barbarism that riddles the history of medicine and certainly see that they had a point. Medical conservatives believe that we should err on the side of treating less not more. EBM was partially a battle against the intervention bias that still plagues medicine. “We have to do something” feels right, but is often harmful. This is why the CAST trial on anti-arrhythmia drugs is a poster child for the movement.

Though well-intentioned, evidence-based medicine is one of the best case studies of how the quantification trap leads to madness. We computerize everything to displace the influence of narratives. Mechanism and natural law don’t matter. The only thing that matters is optimizing outcomes. The pursuit of medical knowledge is only to optimize actuarial tables, not to understand the human condition. Medicine becomes a video game. This is the purest form of what Jean-François Lyotard called postmodernism. Statistical thinking run amok is the postmodern condition.

1. I’ve always found it completely incoherent that systematic reviews, which are observational studies, are graded higher than randomized trials. But, as this post argues, the whole system is more about ideology than logic.