I’ve been obsessed with playing guitar and making music since I was thirteen. Obviously, as a giant nerd, I approach this in the nerdiest way possible. So much of my playing is thinking about the mathematical relationships in patterns of intervals and shapes. I love challenging abrasive music that involves odd counting or microtones. I geek out on music technology, be it software or hardware. But in the decades I’ve been doing this, I’ve never come across someone who thinks I’d be able to get better at music through science.

Why are there no science-based guitar lessons? There are countless books with varied methods on the fastest way to learn or the best way to get better. There’s beautiful math you can nerd out on. Just like in weightlifting, people sell their lessons online on Instagram. But none of the musicians I follow ever put up PubMed screenshots. Instead, you find a psychopathic Spaniard who learns to play impossibly hard drum parts with one arm tied behind his back.

Don’t the laws of adaptation apply to music? It’s certainly the case that the more you practice, the better you get. Much of music practice is just sensorimotor learning. You are adapting both neural pathways and strengthening muscles. There’s psychophysical adaptation in syncing your limbs up with a click track. There are fine motor skills involved in hyperefficient picking. It’s also clear that if you practice too much, you’ll make your hands bleed or develop tendonitis. Surely Selye’s General Adaptive Syndrome applies to woodshedding.

I mean, the concept of progressive overload clearly applies to practice. The way you learn a song is to set your metronome to half time and work your way through the notes. Once you have it down, you increase the beats per minute by one. Each time you come back, you play faster than last time. Next thing you know, you are shredding Chopin.

And it’s not like there’s one perfect way to learn how to be the best at your instrument. There are wide disagreements about the best auxiliary exercises to improve your chops. There is an infinite collection of tutorials and courses designed to improve your skills, telling you which rhythms to practice, which strings to skip, which positions to memorize. And yet, no one thinks you need to run RCTs on these books to crown the best one.

You might say music exists more for art than for competition, and I’d applaud your idealism. But the music industry is a rough one, and it gets pretty cutthroat when you try to make a career out of it.

I write all this to ask what it is about the gym that makes us think that we can science it. I don’t have a good answer to this yet. I have a few partial answers, but none are particularly satisfying to me.

First, there is a clear connection to medicine, and for a wide variety of reasons we’ve decided that medicine needs to be based in science. Physical therapy can only argue for legitimacy in the healthcare sphere if it can be proven a cost-efficient therapy. That means it must be based in our post-modern science of efficiency, and hence we are forced to run RCTs.

The medical reasoning breaks down when it comes to weight lifting. A lot of the biggest names in the science-based lifting community are obsessed with hypertrophy, the fancy scientific word for increased muscle size. While it’s true that to be a strength athlete you need to have big muscles, most of these guys are selling programs for aesthetics. Dudes want to grow their muscles because looking jacked builds their self-esteem. The sport of hypertrophy, bodybuilding, is competitive body dysmorphia. Bodybuilding is an astonishingly unhealthy sport. It involves taking ungodly doses of performance-enhancing drugs, alternating binge eating with starvation, and walking around at dangerously low body fat levels. The pro athletes in this sport die at rates far higher than football players. It’s a mess. There is no good health-related reason for you to look like a modern bodybuilder.

But perhaps the universal desire of perfect aesthetics is a core part of why science is so alluring. Not everyone wants to be able to play 10 over 11 polyrhythms, but everyone wants to look hot. And since it’s so universal, the nerds want a way to stick it to the jocks and claim a masculine domain that hadn’t originally been theirs. The authority of science gives the nerds an institutional leg up in a world where all men want to stake their claim. Science-based training is to the gym as the analytics department is to professional sports teams.

However, when it comes to the strength sports, the science takes you about as far as the analytics takes a football team. Strength sports are not as quantitative as they look. There’s no optimal answer for any person to follow to become a championship Olympic weightlifter, even though the goal of that sport is to get the sum of two numbers as high as possible. Working hard, resting, and eating get you 80% of the way there. Science can’t fill in the rest. If you want to do heavy clean and jerks and snatches, you have to show up and do them.

Subscribe now

Last week in Zócalo Public Square, I wrote a piece on optimizing diets, adapted from the second chapter of The Irrational Decision.1 It’s one of my favorite stories in the book. Though it predates computers by several years, it’s a microcosm of the computer age and fits in seamlessly with today’s oddly dominant online culture of wellness optimizers. In a spat with USDA nutritionist Hazel Stiebeling about what sorts of recommendations are acceptable for the government to publish, the prickly economist George Stigler solved a complicated tableau by hand to find a rather unpalatable “minimum cost subsistence diet” of wheat flour and navy beans. Go read the essay, and then come back here and read a few fun addenda.

I tell the story of the diet problem in most of my book talks, and I always receive fun feedback. Jeff Linderoth sent me a hilarious reflection by George Dantzig, the inventor of linear programming, on his apparently futile attempt to find his own optimal diet to lose weight. Though Dantzig knew Stigler’s optimization problem had nothing but absurd and disgusting solutions, he figured he was adept enough at building linear programming models to patch Stigler’s simplistic assumptions with appropriate shaping of objectives and constraints. He used spare cycles of the IBM 701 at the RAND Corporation to churn out ever more innovative meal plans. But he kept getting bizarre recommendations, like drinking gallons of vinegar or consuming mass quantities of bouillon. He diligently refined the diet over the course of a week before his wife, Anne, got fed up:

Speaking firmly so that I would know who was boss, she said, “I have been studying the various menus the computer has been generating. There are some good ideas there that I can use. I’ll put you on MY diet. She did and I lost 22 pounds.

Steve Stigler—not only an amazing statistician and historian but also George’s son—attended my talk at the University of Chicago. Steve told me how this paper made its way out of academia and into national newspapers, ruffling feathers from coast to coast. George would receive angry letters scolding him about how “this is no way to feed growing boys.” I thought it was pretty clear from reading the original paper that George didn’t think anyone should try to eat his diet. He was trying to prove a point about the impossibility of optimal diets and the paternalistic nature of government recommendations. But people ended up taking him literally. Papers that start as sardonic jokes can surprisingly take on a life of their own.

Stigler’s paper is part of a broader conversation about the scope of government policy. The idea of a computable government was central to economic discussions during the Great Depression and throughout the Second World War. Experts and government officials argued about what is optimal, what can be planned centrally, what individuals should be allowed to navigate for themselves, and what sort of information is beneficial and which is coercive. These debates strongly influenced von Neumann, as you can see in his and Morgenstern’s engagement with contemporary economic debate in the introduction to their revolutionary book on game theory. For another fun example of the people building computers closely interacting with the people designing policy, here’s a 1958 photograph of a sharply dressed Claude Shannon at the Center for Advanced Study in the Behavioral Sciences at Stanford, taken by George Stigler on Steve Stigler’s camera.2

The diet debate also raises the uncomfortable central theme in Elizabeth Popp Berman’s book, Thinking Like an Economist. Everyone across the political spectrum is arguing about efficiency, as if that’s the only thing the government should think about. Left-wing technocrats (aka the Democrats) apply this sort of economic thinking to the utility of the population. Right-wing policymakers (aka Republicans) apply economic thinking to the utility of the individuals in that population. No matter their politics, everyone is thinking like an economist. The valence of the arguments and the parties making those arguments remain uncannily similar today.

Given the grand scale and ambition of the federal government, USDA dietary guidelines should be a fourth-order concern. But there’s something about worrying about what we should eat that galvanizes the popular imagination. It’s fun to walk through the original arguments about what should be in the food pyramid, especially given the weird steak-centric geometry being pushed by RFK’s cuckoo version of HHS. A steak every day sure sounds more appealing than a bean pie.

Subscribe now

In the noisy chaos of science-based wellness on social media, there are a few reasonable voices. One of my favorites1 is Greg Nuckols, a champion powerlifter who has been blogging about the science of strength training for well over a decade. Over the weekend, I read his 2015 books The Art of Lifting and The Science of Lifting, and they hammer home how a simple topic can be made overly complicated when we decide to write scientific papers about it.

Nuckols’ two-volume series is less than two hundred pages long and written in an engaging, bloggy voice. Though the books link out to systematic reviews and randomized trial reports, he doesn’t get bogged down in that material. Instead, he focuses on one simple modeling principle and its consequences: adaptation.

Though the human body is impossibly complex, its many interconnected systems deal with stressors in a surprisingly uniform way. Stressor here means all sorts of things: wounds, infections, temperature changes, environmental changes, activity changes. Something out of the ordinary. The model of homeostasis posits that bodies actively work to stay the same as much as possible. Most of our daily activities don’t stress the body, and we remain in a pleasant, steady state. However, the body changes and adapts if it gets pushed far enough out of its comfort zone. If you apply enough stress, the body initiates a response to prevent harm or death. There are limits to this response, and too much stress will cause injury or death. But in the right Goldilocks range, the body will reconfigure itself to resist the stress next time it sees it.

Exercise can be modeled as a stressor. It’s certainly stressful. It increases muscle tension and body temperature. It increases demand for oxygen and nutrients. It lights up your sympathetic nervous system and triggers the release of adrenaline. Your body responds to exercise similarly to how it fights off other stressors like injuries and infections. It panics and then marshals resources to make sure it sucks less the next time you go to the gym.

There are many models for the temporal behavior of the stress reaction. The most common mathematical model is the fitness fatigue model developed by Tom Calvert, Eric Banister, and collaborators in the 1970s. Notably, Calvert was an electrical engineer who introduced his physiology colleagues to cybernetics and impulse responses. The fitness-fatigue model is a parametric model of adaptation: stressors introduce bad effects (fatigue) from which the body works hard to recover quickly. It also introduces positive training effects, the adaptations you were chasing. The fitness-fatigue model posits that the positive effects decay more slowly than the negative ones. For the dynamical systems nerds out there: they model this with a simple second-order linear system. Hence, if you repeat this process of enough but not too much stress over time, you nudge your body to handle more and more stress.

This feels science-y, no? The general adaptation syndrome is a well-tested, clean model that certainly helps guide how to train athletes and implement physical therapy. This model hasn’t changed since 1980. It prescribes working hard and continuing to do so over time to increase adaptation. It predicts that training harder than last time is beneficial. These two together form the principle of progressive overload, a very simple concept that influencers try to make absurdly complicated. Moreover, this model of adaptation tells you that you need to recover to hasten the elimination of fatigue. It even suggests implementing tapering periods before competition, a common practice.

OK great, now let’s try to apply this model to practice. What is the right shape of the actual stress to give you the results you want? If you want to get stronger, which exercises should you do? How frequently should you do them? What weights should you use? How many repetitions should you do for each exercise?

Unlike most of the charlatans in the science-based fitness space, Nuckols is brutally honest about this: the answer to all of these questions is that we don’t really know. Or at least, we at best only have partial answers. You probably need to go above some nominal effort to induce adaptation. Because there is so much intersubject variability, the exact perfect amount is hard to pin down. But if you work hard and make sure to rest and eat, you’ll get stronger over time.

Nuckols thinks the broader scientific research on sports training tells us more than I think it does. From my reading of too many studies in this space, the ns are always really small, the data are always poor, and the assessments are always ill defined. Do we need thousands of studies to tell us that we should “keep it simple, stupid?” People obviously can go to the gym and get stronger.

But what’s interesting to me about how Nuckols writes is that he doesn’t consider “science” to be some combination of biomolecular pathways and randomized trials. Instead, his scientific view is one of simple predictive models and their limitations. For an athlete like Nuckols, the scientific view helps systematize what matters in training. Working hard matters. Working harder over time matters. But resting and recovery are also critical to good performance. The right balance might not be easy to quantify, but the qualitative arc and predictions based on principles of adaptation are helpful. The general adaptation model for training tells us that training isn’t that complicated if you think carefully about what you are going to measure, and how you are going to track progress. This cybernetic view of biomedical science—one closer to what I would call engineering—feels like a promising path for many other aspects of medicine.

Subscribe now

Of the many cursed segments of vertical video, few annoy me more than the cottage industry of lunatics ranting over screenshots of PubMed pages to claim authority for whatever therapy, training program, or peptide they are selling. These people are all full of shit. They promise to cure your pain, get you thin, and make you stronger. All through SCIENCE.

These influencers will tell you that the scientific is better than the not scientific, and might even go so far as to say that the remainder is pseudoscience or quackery. But we unfortunately live in a narrow-minded world where scientific too often means “proven efficacious in a systematic review of randomized controlled trials.” You’ll be hard-pressed to find anything in the realm of treating musculoskeletal limitations that fits the bill.

It’s hard to shoehorn these sorts of therapies into the starting requirements for randomized trials. We’d need to start with clean definitions of an intervention and an outcome. What would these be for the management of pain by physical therapy?

Let’s start with the intervention. If we are being dogmatic evidence-based practitioners, the intervention in any physical therapy trial is the invitation to have therapy. According to the intention-to-treat principle, the invitation is the only thing that can be randomized. We can’t look only at the people who comply with every instruction and make it to the end of the rigorous therapy regimen. The model patients who diligently comply might differ from those who don’t, and our statistical signal will be biased if we include only the former. The only way to avoid these selection biases is to count everyone who was randomized, regardless of what happens between randomization and the final assessment. Both committed patients and no-shows contribute to the measured average efficacy.1

But what does it even mean for patients to follow the protocol? Physical therapy is far more complex than taking a drug. There’s no simple unit of treatment applied. Each interaction with a physical therapist involves a conversation about how things have been going, a plan for moving forward, some sort of interactive intervention in the office, and a discussion about what to do once the patient goes home. Each step here introduces a new branch in a deep decision tree. And every PT I’ve interacted with has been different, even when performing similar range-of-motion tests or manual therapies. Moreover, the treatment of any session depends on the entire history of the treatment so far. On top of this, every physical therapist I’ve seen has assigned daily exercises to do between sessions. This is part of the treatment, too! There is no way to perfectly isolate and randomize a single component of these complex treatment protocols.

A multi-stage protocol is an exponentially large collection of interventions. I made this point on the blog a few weeks ago in the context of anticoagulant trials for heart disease: “If you want to compare the effect of three different timings and three different dosages of a single drug, you need nine arms in your trial. If you want to additionally see if a second drug is helpful, you need 18.” Physical therapy is arguably much more complex.2

What about the outcome? In drug trials, we might get grim, unambiguous, objective outcomes like mortality. In vaccine trials, we might get unambiguous outcomes, such as a PCR diagnosis. Unfortunately, in pain management, the outcome is necessarily subjective. You can measure changes in range of pain-free motion, but there is too much heterogeneity to definitively stake out what a good outcome would be. Instead, pain therapies are most commonly evaluated based on improvements on the Numeric Ranking Scale. Studies ask participants at admission how their pain is on a scale of 0 (no pain) to 10 (worst pain imaginable). They ask them again at the follow-up. Statistical protocol then dictates computing the mean of the differences in treatment and control and running a t-test. You can try to remove the heterogeneity in how people respond to these questions, but these adjustments are based on subjective clinician calls. No matter what you do, pain is hard to mathematize. Doing statistics on these “numbers” and coming away with strong conclusions is a fool’s errand.

Beyond the treatment and outcome, all sorts of investigator biases make randomized trials even messier. You can blind the patients and the clinicians who assess outcomes, but you can’t blind the people applying physical therapy. It’s impossible to say what effect this sort of bias has on the scientific record. Even when well-intentioned, a clinician who believes in PT can subtly give away the secret assignment to their patients during a session.

I’ve never read a single study in this space that’s been compelling, and I don’t know why we hope that a narrow view of therapy can help us out of it. This fuels the fire of debate with people using studies to attack each other’s practices. There are countless articles and videos castigating stretching, massage, or cupping as not backed by evidence. These are all denounced as pseudoscience by a medical establishment that prescribed OxyContin like candy for two decades. Boy do I have some bad news for people who think there is great evidence that opioids work for pain management.

If we want to understand best practices for “wellness,” we need a different language around it. Maybe this language will need to lean on biomechanical plausibility or biochemical pathways. That certainly wouldn’t hurt. But more importantly, the language will have to prioritize discussions of craft, practice, and the cultivation of expertise. Whatever the case, the narrow definition of evidence-based needs to be reimagined. Healthcare is far more than a collection of unambiguous interventions with unambiguous outcomes.

Subscribe now

Monday’s post highlighted how physical therapy sits at the heart of the tension between populations and individuals in modern scientific medicine. The field addresses one of the core problems people come to their doctors with: musculoskeletal pain or restriction. More often than not, these ailments can’t be treated by simple prescriptions of rest, drugs, or surgery, and primary care physicians don’t have solutions for these problems. So instead, they are often referred to physical therapy, which combines an uncomfortable mix of science and craft that eludes clean, evidence-based evaluation.

I’m a bit of a PT fanatic, and have had the privilege of working with five excellent physical therapists since I got a bit too overly serious about strength and conditioning in my forties. They were all very different, but all of them applied the same core principles. These principles highlight key aspects of what it means for a therapy to be individualized.

Every PT consultation begins with an assessment. Because there are so many moving parts and everything is so interconnected, it’s often hard to identify a single cause of a particular pain or restriction. Weakness in one area is balanced by strength in another. Pain in your foot might be caused by limited mobility in your hip. While everyone’s musculoskeletal system is connected in the same way, we have wide variability in the sizes, shapes, and positioning of our muscles and bones. Once we combine this with the variability of people’s physical interactions with reality, we find there’s no single simple answer for everyone. Each person is a weird biomechanical puzzle, and not every puzzle is particularly easy to solve. However, there is a set of therapeutic principles for making people better.

A therapist will test out the range of the different muscles and joints that could be connected to your symptoms. They try to find how far you can move without pain, which is a mix of qualitative (pain) and quantitative (degrees of rotation). Once they find which specific movements are restricted, they have a general process to fix all of them.

First, the therapist will open the restricted range using some sort of manual therapy. This is done in a single session and might involve stretching, deep-tissue massage, or even cupping (more on that later). Next, they give you exercises to gain control over that newly opened range. This is done between sessions with the homework exercises they assign you. After you can confidently access this new range, you load it, adding strength-bearing exercises to further strengthen the mobility you aim to increase. All of these are progressed over time, adding more intense manual therapy, more difficult exercises, and more weight.

Progression is the core principle, whether you’re treating tennis elbow or a sprained hip (I’ve had both). The difficulty of your physical therapy exercises will gradually increase with each session. It’s sort of obvious when you say it that way, no? This is all there is to the buzzy principle of progressive overload. It works because your body adapts to stress. When you add enough stress, it overcompensates and resists the stress more strongly the next time. Cascading this reaction over a long period in a controlled, thoughtful way leads to a clear and measurable outcome.

The complication is that bodies adapt more slowly than most of us would like. It might take weeks to start to see progress. Because progressive overload is slow, lots of people walk away from physical therapy convinced that their injury would have just gotten better anyway. They might be right, and I can’t argue their concerns away. But that’s why I’ve found it helpful to track progress over time. It might not be the PT prescription that lets me access that extra range or strength, but I feel more in touch with whatever it is that my body is doing. I’ve found that it’s worth targeting something even more ambitious than just “back to normal.” I now aim to come out of physical therapy more capable than I was before my symptoms started.

For many people, PT doesn’t work. I like it, but I might be particularly well suited for it. I love rituals. I love obsessing about niche minutiae. I love tracking and journaling. But recovering from injury and managing pain are not easy. Compliance with physical therapy might be a greater challenge than with other treatments. It’s a much bigger commitment than “take two pills a day with meals.” If you don’t like to spend 30 minutes a day on weird stretches, PT might not work for you. But that doesn’t mean it doesn’t work for someone else.

And this is why physical therapy is so easy to attack on scientific grounds. How do you run a controlled trial on a practice this complex and individualized aimed to treat subjective symptoms like pain? I’ll dig into the impossibility in the next post.

Subscribe now

Last week in the Washington Post, fitness columnist Gretchen Reynolds wrote about a “new way to treat sprains.” The common, catchy advice of RICE (Rest, Ice, Compression, and Elevation) was out, and new science had come to light: Doctors and scientists now concur with youth sports coaches that you should “walk it off.”

I’m barely being glib. Reynolds quoted research suggesting that reducing inflammation impairs the body’s natural healing mechanisms. By damping inflammation, scientists conjecture you are damping regeneration. This means that anti-inflammatory drugs like ibuprofen are also counterproductive, as they suppress key pathways that aid the healing process. Moreover, resting leads to muscular atrophy and loss of strength in tendons and ligaments. Putting it all together, rest and anti-inflammatory therapies compound to impede healing.

Reynolds cites expert studies on this topic (there are always studies), but to someone immersed in the culture of bro-science, I was surprised this advice was considered new and novel. Since getting a bit too obsessed with strength sports, I have injured myself multiple times, and I have never visited a physical therapist or watched a fitness YouTuber who recommends resting after an injury. Instead, most tell you to move as much as possible and to reduce inflammation only if it’s getting in the way of that movement. If you get injured, they advise you not to panic, to find what motion gets you moving, and to exercise around the limitations brought on by the injury. The place you sprained will be mobility-restricted, but you should access as much range as you can and keep things light, slowly adding work back as you go.

The working model that coaches and PTs share is that weight training and injury recovery are both processes for getting stronger. An injury is just a disadvantaged starting point. In both cases, the way to increase strength is to regularly stress the system with increasingly harder tasks. Every time, you hope to access more range or lift more weight. This is the principle of progressive overload.

Though now the backbone of most sports training and fitness programs, progressive overload originates in physical therapy. The common prescription of “three sets of ten” originates in the practice of physician Thomas DeLorme, who worked to rehab injured soldiers during World War II. His plan to heal, borne out by diligent clinical practice, became the foundation of the modern paradigm for getting stronger.

Though DeLorme’s progressive overload principle worked wonders in patients and athletes alike, standard medical practice unfortunately always needs a stamp of “science” to back it up and make it real. The practice of coaches, trainers, and physical therapists everywhere can’t justify a technique. That’s bro-science. You need some obscure microbiological pathway or a randomized experiment.

The problem is, the science of physiology is much murkier than you might expect. For years, I have been looking for a clean mechanistic description of why bodies adapt to training. Why do we get stronger and faster by progressively working harder at the gym? Reynolds’ column quotes UC Davis Professor Keith Baar applying his research to his own ankle sprain. Baar has long studied the molecular biological pathways that explain why training techniques like progressive overload work and why bodies adapt to stress. He co-authored one of my favorite surveys on adaptation to training, which lays out the myriad biological pathways and how different sports modalities shape them. My main takeaway from the survey is that we still don’t have a clear, comprehensive description of how the body adapts to exercise. The experiments all agree that if you progressively overload, then you get better. There are lots of things you can measure in a body under stress that point to something mechanistic going on. But they can’t pin down a precise mechanism that drives adaptation, and they certainly can’t figure out what an optimal training protocol would look like. In sports medicine, we don’t follow the science. Science follows the practice.

Coming back to inflammation, we can look at the evidence that ibuprofen inhibits muscle growth in strength training. Common gym wisdom is that ibuprofen, those sugar-coated orange pills often consumed like candy by gym addicts, are actually killing your gains. Bros tell you that you have to tough out the delayed onset muscle soreness associated with training. This was also the conclusion conveyed by the scientists quoted in Reynolds’ article.

Well, the science is of course far less certain than the YouTube advice. I could only find one study showing that taking ibuprofen inhibits muscle hypertrophy when strength training. In this study of 31 people aged 18-35, the treatment group took 1.2 grams of ibuprofen every day for eight weeks, and the researchers observed that they had less muscle growth than the control group. 1.2 grams is the maximum recommended over-the-counter dosage! This is a bit extreme. And yet, despite multiple attempts, no study has ever reproduced these results. There’s a fun fight between one of the failed replication teams and the original authors if you’re into those sorts of academic spats.

So who knows, team? It’s possible ibuprofen doesn’t affect training performance at all. I personally wouldn’t recommend eating it like candy. But what do I know? I’m not a scientist.

Subscribe now

My argmin.md file compels me to write a blog about blogging every July 1st. It’s my official Substack anniversary, and we’ve made it through year three. The process here is, for better or worse, set in stone and could be written as a skill file for an LLM agent. I listed the rules last year, and they haven’t changed since. I don’t expect them to change this year either.

In fact, going back over my posts, it seems like the most novel change was adding a new kind of content that got decidedly mixed reviews: football blogging. Of all the topics I’ve dabbled with on this Substack, there is none more polarizing. Some people really liked it! Others, let’s say, did not. I’m waiting until training camp gets going before I decide whether I’ll commit to another season of yelling about the absurdity of win probability.

The football blogging is related to one of the trickier things I haven’t been able to figure out in three years writing this blog. Argmin is a periodical with different authors. Those authors just all happen to be me. I realize you receive a notification from me whether I’m writing about a topic you care about or not. This is a problem I don’t really know how to address, but I suppose it is an issue with all Substack subscriptions. I don’t want to take on the bother of splitting this into four different Substacks, which I think would annoy you even more than it would annoy me. I hope the disclaimers at the top of the posts help you decide if you want to keep reading. I would love feedback on how to make that signposting clearer.

Relatedly, commenters are increasingly asking questions about things I’ve already talked about. This is totally expected, as I know I write more than anyone likely cares to read, but I worry about repeating myself—thus alienating a different collection of readers—when addressing them. The balance between repetition and clarity is delicate. Apoorva Lal said my blog has its own private dialect. If I don’t repeat myself somewhat, filling in the context of what my brain thinks obvious, the content will be even less approachable, and no one wants that.

Moreover, sometimes the only way to iron out a nonobvious point is by repeating it until it’s clear. I wrote in the afterword of The Irrational Decision that I used this blog “as a sketch pad, where I first workshopped much of the text that you read here.” This newsletter remains a place for me to play and write out first drafts. I get a lot out of putting ideas out there, even when they are way underbaked. “Talks and blogs are ephemeral, but interactions with the audience helped to shape and hone my arguments. All of those early interlocutors, in person and online, helped shape this final archival form. I’m thankful to all of you.”

In the spirit of learning from the audience, we can use this year’s reflection post as a chance for you to tell me what you’d change about the argmin magazine. Do you want more of any topic? Would you prefer a different format or cadence of posts? You can email me if you’d prefer not to comment (my berkeley.edu address is fine for that).

In the meantime, I can give you a preview of what to expect over the next year. There will definitely be more live course blogging. In the fall, I am planning an unhinged new class on forecasting, the tools we use to do it, and why we’re so obsessed with it. That should be a fun one. In the Spring, I’m apparently signed up to teach undergrad probability. This is for engineering students and based on the excellent text of Bertsekas and Tsitsiklis. Since it’s about engineering, we’ll have to grapple with some of the complicated metaphysics of what we think these probabilities mean in the context of building things. You, me, and the undergrads will all get confused together.

As for the remainder of this summer, I want to finish this series Against Optimaxxing. I’ve been talking about writing this forever, and I need to put in the effort to finish it. It would be an attempt at writing what Henry Farrell calls speculative nonfiction, seeking a reasonable language to imagine a future free from the vulgar positivism that runs our culture, politics, and industry. This blog series will try to establish a necessary vocabulary. My draft writing on this particular topic might lead nowhere. It might remain a private dialect. But I want to try to write it down.

Subscribe now

Yesterday on Good Morning America, NFL Hall of Famer Chris Johnson announced that he had ALS. Johnson, nicknamed CJ2K for his 2006 rushing yards in 2009, was one of the most electrifying players I’ve ever seen. He was impossibly fast. Every one of his carries was must-watch football, as you’d never know when he’d break a run for a touchdown. Johnson still holds the record for the most yards from scrimmage in a season, with his total of 2509 a comfortable 80 yards in front of second place.

Johnson retired in 2018, and, like most football players, disappeared from the national spotlight. So it was beyond heartbreaking to see him yesterday, at 40 years old, speaking to Michael Strahan with an eye-tracking device. ALS attacks the body’s nerve cells, progressively inhibiting a person’s ability to move their limbs, speak, and breathe. Though highly variable, the progression can be shockingly rapid. Johnson told GMA that he had been diagnosed less than a year ago. He wanted to share his story to bring more attention to ALS and the search for radical new treatments.

In 2023, Gideon Lewis-Kraus wrote a moving investigation into this search for The New Yorker. People with ALS and their families are, for quite obvious reasons, passionately dedicated to finding any reasonable treatments for the disease. The hope is to find therapies that at least slow disease progression and improve the patient’s quality of life. The challenge is this tends to require drugs, and ALS lies outside the sweet spot of modern drug discovery.

ALS is a rare but devastating condition, with around 5,000 new cases diagnosed in the United States every year. There is no clear established cause for ALS. Though it typically manifests in people’s late fifties or early sixties, Johnson’s case exemplifies how unpredictable its onset can be. Sometimes ALS progression takes years; sometimes it rapidly advances in months. There are no clear genetic signals for ALS, and family history isn’t predictive. There is a biomarker present in the nerve cells of almost all ALS patients, but current technology only lets us detect this biomarker after death.

Without access to a marker, measuring progression or reversal is a subjective challenge. Current ALS surrogates numerically assess bodily functions like speech, swallowing, breathing, and motor skills, and combine these different tests into a single number. For example, the ALSFRS-R score is the sum of 12 different assessments, each scored on a scale from 0 to 4, where 0 indicates total loss of function, and 4 indicates no loss of function.

All of this adds up to make drug testing a major challenge. It’s hard to define reasonable endpoints, and most trials aim to find a statistically significant difference between treatment and control in ordinal assessments made by clinicians. Though patients and their families urgently seek new possible therapies, there are not enough ALS patients to meet the harsh statistical standards of clinical trials. Moreover, drugs require trials, and the ethics of potentially denying the control group something beneficial are complex and fraught.1

Lewis-Kraus draws connections to the disease that set the standard for patient advocacy against institutionalized medicine: AIDS. AIDS activists in the 1980s were not only vocal in spreading awareness of the disease and its various treatments, but they actively engaged in scientific research and drug policy. It’s a remarkable example of a citizen group not only challenging, but shaping scientific expertise. To this day, AIDS activists remain go-to consultants for helping people and families affected by rare diseases to organize and advocate for the afflicted.

In hindsight, HIV treatment is similar to the treatment of other progressive diseases like ALS. None of the treatments cured an HIV infection, but they allowed patients to manage their infection. Today, HIV can be treated pharmaceutically with an excellent long-term prognosis. Though initially a terrifying death sentence, people with HIV infections can live a normal life with access to the proper medication.

The differences between AIDS and ALS highlight what makes a progressive, terminal condition treatable. AIDS was caused by an infection with the virus HIV. The first effective HIV treatment, AZT, targeted the virus by fooling it to bind to a byproduct of the drug rather than the person’s DNA manufacturing system. This eliminated HIV’s ability to replicate. Moreover, AIDS had an easily measurable surrogate marker to monitor progress. HIV killed a particular type of white blood cell, and counting these cells was a reasonable way to test if the drug was having an effect.

Sadly, while researchers are hopeful that we’re making rapid progress, we’re still missing many crucial causal pieces to the ALS story that might illuminate an appropriate target for the pharmaceutical industry. Johnson hopes that raising awareness of ALS might help us find the missing link in understanding and treating the disease. He wants to keep fighting. My heart goes out to CJ2K, his family, and everyone else suffering from this terrible condition. And I hope his inspiration will help us find the missing gap for a breakthrough treatment.

Subscribe now

My teacher and friend Dimitri Bertsekas passed away earlier this month. I just learned about this over the weekend, and I’m still processing my thoughts.

Dimitri was a hero of mine for so many reasons. I took my first and only class on optimization with him, and I was riveted by his clean presentation of convex analysis and mastery of the overhead projector. He took a liking to me, and I made it a point to find him for a chat whenever I’d visit MIT. He was always generous with his time and excited to exchange ideas, but he wouldn’t hesitate to harshly scold me when I’d present some result new to me that he had in fact written about twenty years earlier.

This was unavoidable because Dimitri did foundational work across mathematical optimization, penning landmark results in stochastic gradient descent, convex optimization, distributed optimization, dynamic programming, and reinforcement learning.

His work in reinforcement learning remains underrated. His collaboration with John Tsitsiklis, compiled in their book Neurodynamic Programming, was the first to show that most reinforcement learning algorithms were effectively approximating dynamic programming. Our contemporary, model-free mindset, rooted in Markov decision processes, derives from these initial insights. As far as actual practice goes, their book is more important to the modern way we implement reinforcement learning than Sutton and Barto’s.

Dimitri was also a passionate mathematical communicator. I own more of his books than any other mathematics researcher, but he wrote more than any other mathematics researcher in my field. Is there too much material? You could make the case! However, that he has definitive texts covering a broad range of optimization theory is remarkable.

In many ways, Dimitri was an original math blogger. He wrote exactly what he wanted to write, and he wrote frequently. He got fed up with publishers getting in the way of his process, so he started his own publishing company, Athena Scientific, shipping stacks of books out of his garage in Belmont, Massachusetts. This allowed him to write countless new editions and revisions of his work, reflecting trends in practice and incorporating new insights and simplified arguments. Though you’ll see plenty of repetition across his volumes, he knew that no book was a final draft. Each volume was a step towards broader understanding.

A decade ago, when he retired from MIT, I wrote a post of appreciation for his scholarly and pedagogical work. I’m going to reprint it here today, as it includes one of my favorite passages in his books about the tension between theory and practice. Right now, in our frenzied agentic era, we’re leaning heavily on only practice and vulgar empiricism to push out as many papers and products as we can before the bubble pops. Is there a role for theory at all anymore?

Dimitri argued that optimization theory is always a mix of qualitative and quantitative. The qualitative helps us understand what we know and don’t know. By gathering feedback from practice and constructing a working narrative, theorists can help engineers develop a language to describe what is possible and what can be improved. Theory provides a narrative scaffolding that helps us understand what to build and what to attend to when things break. Building stories of connections helps us streamline our processes and try things we might not have thought of.

There’s an ebb and flow between the theory and practice. I will revisit this ten years from now and see where the theories land.

Until then, rest in peace, Dimitri. Know you changed the way I think.

The following initially appeared as The Role of Convergence Analysis on June 10, 2016 at the old argmin blog.

This year marks the retirement of Dimitri Bertsekas from MIT. Dimitri is an idol of mine, having literally written the book on every facet of optimization. His seminal works on distributed optimization, dynamic programming, and Lagrangian methods remain the best references available. I had the privilege of taking Dimitri’s convex analysis course in grad school, and he would frequently burst into class beaming because he had stayed up until 2AM the night before simplifying an argument of Rockafellar’s down to elementary calculus.

My last post on Lagrangians was based on Chapter 3 of Dimitri’s Nonlinear Programming Book. Chapter 2 also happens to feature one of my favorite passages about the delicate balance between theory and practice in optimization. One of the trickiest parts about optimization (and a point I intend to repeatedly hammer on this blog) is realizing how many of the theorems are “qualitative” rather than “quantitative.” I wanted to just quote Dimitri’s text in full here, as I don’t think I could write it better. Best wishes to you in retirement!

The Role of Convergence Analysis by Dimitris Bertsekas

The following subsection gives a number of mathematical propositions relating to the convergence properties of gradient methods. The meaning of these propositions is usually quite intuitive but their statement often requires complicated mathematical assumptions. Furthermore, their proof often involves tedious ϵ−δ arguments, so at first sight students may wonder whether “we really have to go through all this.”

When Euclid was faced with a similar question from King Ptolemy of Alexandria, he replied that “there is no royal road to geometry.” In our case, however, the answer is not so simple because we are not dealing with a pure subject such as geometry that may be developed without regard for its practical application. In the eyes of most people, the value of an analysis or algorithm in nonlinear programming is judged primarily by its practical impact in solving various types of problems. It is therefore important to give some thought to the interface between convergence analysis and its practical application. To this end it is useful to consider two extreme viewpoints; most workers in the field find themselves somewhere between the two.

In the first viewpoint, convergence analysis is considered primarily a mathematical subject. The properties of an algorithm are quantified to the extent possible through mathematical statements. General and broadly applicable assertions, and simple and elegant proofs are at a premium here. The rationale is that simple statements and proofs are more readily understood, and general statements apply not only to the problems at hand but also to other problems that are likely to appear in the future. On the negative side, one may remark that simplicity is not always compatible with relevance, and broad applicability is often achieved through assumptions that are hard to verify or appreciate.

The second viewpoint largely rejects the role of mathematical analysis. The rationale here is that the validity and the properties of an algorithm for a given class of problems must be verified through practical experimentation anyway, so if an algorithm looks promising on intuitive grounds, why bother with a convergence analysis. Furthermore, there are a number of important practical questions that are hard to address analytically, such as roundoff error, multiple local minima, and a variety of finite termination and approximation issues. The main criticism of this viewpoint is that mathematical analysis often reveals (and explains) fundamental flaws of algorithms that experimentation may miss. These flaws often point the way to better algorithms or modified algorithms that are tailored to the type of practical problem at hand. Similarly, analysis may be more effective than experimentation in delineating the types of problems for which particular algorithms are well-suited.

Our own mathematical approach is tempered by practical concerns, but we note that the balance between theory and practice in nonlinear programming is particularly delicate, subjective, and problem dependent.

Subscribe now

On Friday, Midjourney, a generative AI company best known for automatically creating DeviantArt images from text prompts, announced it was pivoting to medicine. Their web copy opens with blunt honesty: “Today we’re gonna announce something a little weird and a little crazy, but also spectacular and filled with hope.” What follows is a cinematic trailer straight out of Ridley Scott’s Alien franchise, depicting Midjourney’s dream of a new whole-body ultrasound scanner that is “as powerful as MRI, and as casual as a trip to the spa.”

Their first scanning spa won’t be available until 2027, so I’m not sure why they decided to announce this on Friday. But it ignited a heated fight between tech boosters and medical practitioners, lighting up my mostly dormant Twitter timeline all weekend. I am embarrassed to confess that it was custom-made to engage me and my niche tastes. I couldn’t look away!

Let’s first be clear: everyone was fighting about vaporware. The actual technology is based on a 2026 Nature Biomedical Engineering paper. That paper, by a Caltech lab, is a nice engineering systems integration project combining a lot of technology that’s been around for decades. There is no fancy AI, and the imaging algorithm is written in Matlab and run on a cheap GPU (an RTX 3070). The paper doesn’t demonstrate any particularly cutting-edge imaging, presenting a few reasonable images of abdominal fat. Instead, the paper promises acceleration, claiming the ability to perform a whole-body scan in about a minute.

Midjourney thus promised something not yet demonstrated:

“In an ideal and near-term future, we take this information and watch how it changes over time. We compare it to the general population, we talk to doctors, nutritionists, coaches, trainers, and AI friends. We become more aware of our health and we improve our lifestyles. We make smarter, more proactive, more frequent decisions. And we live longer, healthier lives, better lives.”

Now, this text may seem clichéd and innocuous, but two communities on Twitter read this paragraph very differently.

On the one side were tech boosters, who seethed with vitriol against the medical establishment. They believe that all information is good, and more information can only improve decisions. They embrace some perverse version of Sutton’s Bitter Lesson that more data fed into more computing can solve all problems.

On the other side were medical doctors, most of whom also seemed to really hate the medical establishment! They’re on Twitter, after all. But they had compelling arguments against indiscriminate scanning.

First, they note that ultrasound is a limited imaging modality, and the new full-body system doesn’t eliminate those limitations. Ultrasound can’t see through bone or air, and has other fundamental limitations with contrast. You can’t fix a fundamentally physics-limited scanner with more frequent scans.1

Second, though conventional wisdom says that early detection improves disease outcomes, we have 75 years of evidence that is far from encouraging. The case has been most illuminating in oncology. Some cancers are untreatable no matter when they are detected. With advances in therapies, some cancers are treatable even if you catch them late. The correlation between early detection and cure is, unfortunately, much lower than cancer researchers had hoped in the 1950s. The medical establishment has tempered its expectations over the last 20 years, but the lay population hasn’t caught up.

Third, frequent scanning means you have to run more tests to follow up on what you see in a scan. This is a nightmare. Imaging is never perfect. Let’s say that you build a technology that is 95% accurate at detecting a disease that afflicts one in a thousand people. A crude probability calculation2 then says that for every hundred scans that come back positive, only 2 of them actually have the disease. What about those other 98 people? They are healthy people who will be subjected to the stress of potentially harmful follow-up tests. Biopsies aren’t fun!

The fourth lesson also comes from oncology. We don’t understand the dynamics of many progressive cancers. Even the best imaging technology still can’t tell the difference between harmful and harmless masses. Some growths look bad and appear to grow over time, but they never cause harm. On imaging, these look the same as tumors that will kill you in six months. Why should we be constantly raising alarms about information that doesn’t lead to actionable, beneficial interventions?

These arguments fell on deaf ears. The tech boosters were furious that the medical conservatives were against more information. They yelled that doctors are terrible at mathematics. They yelled that doctors are gatekeepers preventing people from accessing information that will make them better. They yelled that doctors don’t know innovation if it stares them in the face. One particularly vitriolic person wrote: “You guys thought that Lister and Pasteur both needed to be crushed, and for the most part, you haven’t changed.”

Unsurprisingly to me, the tech boosters didn’t present any scientific arguments. I only saw unfounded assertions about Bayes’ Rule and Blackwell’s theorem, and how “false positives aren’t a problem.” The Midjourney scanner is yet another project in the optimaxxing biohacking space, seeking technological means to wrest control of your body.

What the tech side of the argument came down to was this seemingly anodyne statement in the Midjourney blogpost: “You want as much data as you can get about your health as quickly and as cheaply as possible,” and that you should strive to maximize the “megabytes per second per dollar of information” you capture about yourself. Massive time series data would unlock new diagnostics, overcoming the limitations of single scans. All data is good data, and all data leads to improvement. This assertion is not backed by evidence. It is, as Midjourney wrote, hope.

I side with doctors on this one, though for a reason that they didn’t seem willing to advance. We shouldn’t obsessively overmedicalize ourselves. There is an inherent harm in always seeing yourself as a patient, ailing unless proven healthy. More information can make you sicker.

Subscribe now

The meme of the decade is maxximization: looksmaxxing, proteinmaxxing, moneymaxxing, longevitymaxxing. There’s nothing in the world that can’t be maxxed. All the kids are optimizing these days.

Now, it’s easy to tut-tut obsessive maxxers and see them as cautionary tales of culture gone wrong. But why is maxxing culture bad? We all respect people on humble searches for personal “improvement,” “development,” or “betterment” for themselves and their families. But isn’t “making something better” the same as maxxing? I’m coming around to thinking that the answer is no, and I don’t think it’s a matter of degree. The technical language of optimization—embraced by engineers and economists and influencing the rest of society—makes every quest for self-improvement into something grotesque.

In her phenomenal essay “The Ozempicization of the Economy,” Kyla Scanlon juxtaposes two different dietary interventions, the elimination diet vs. GLP-1 agonists, to draw out the limits of the language and mindset of optimization.

I had a bunch of friends who developed stomach issues in their twenties. Many of them went on the elimination diet for relief. For a month, they cut out everything from their diet except a few basic foods, like chicken breast, carrots, and rice. Then, they slowly added things back to their plate, maybe some basic vegetables, maybe some new meats. They journaled how they felt, seeing if they could find clear triggers for their symptoms. Inevitably, they would all find some culprits to avoid, and would establish a good baseline of what they could and couldn’t eat.

Elimination diets are excruciating. It takes months to get to a nearly normal diet that identifies a few problematic foods. And there’s no clear goal other than finding a broad set of accessible foods that you are happy eating.

Is the elimination diet a form of numerical optimization? What is the objective function? Is it “finding the largest array of foods I can eat while not feeling like shit?” How do you quantify “not feeling like shit?” When is it acceptable to stop? I don’t think it’s beneficial to view this process as maximizing a portfolio of food. Most people are far more concerned with the constraints on living a happy, normal life with a healthy relationship to food.

By contrast, GLP-1 agonists do feel like a miraculous optimization tool. If you take them, and you can tolerate the side effects, you’re going to lose weight. The number goes down for everyone.

Scanlon’s essay describes the madness of hoping everything will work like Ozempic. The maxxing community couches its searches for quick fixes in the language of rational science, but they appeal to the postmodern science I discussed last week. It’s a science that conflates “discovery” with a functional demonstration of utility. Interventions are judged by improvement over a baseline. It abandons explanations of how an intervention translates into an effect and replaces them with crude proofs of an impoverished notion of causality. An intervention causes an outcome if (and only if?) some randomized trial demonstrates a statistical difference when the intervention is compared to an appropriate control.

This is precisely the management view of epistemology Lyotard was calling out 50 years ago. A discovery is now a quantitative improvement of an outcome by a discrete move in a complex game. This not only rules out many other forms of discovery and knowledge-making, but it also restricts our attention to interventions that quantitatively improve outcomes for populations.

And it means the only equivocal interventions are the ones that work for everyone. For drugs like GLP-1 agonists, the effect is stark and undeniable. Ozempic is a drug that didn’t need a trial to demonstrate efficacy. It so unambiguously works for almost everyone who takes it. I’ve written about it here before. The p-values are zero. Everyone loses weight. It optimizes for everyone.

So perhaps the grotesque part, the unavoidable part of optimization culture, is the idea that there must be clear rules that work for everyone and guarantee large, measurable, quantified improvements and that every intervention must work like a miracle drug. The rare case of an impressive discovery like Ozempic convinces people that every problem can be solved by one simple trick.

This warps the questions we ask. It forces us into metrics and quantified scores. It warps the process itself. As Scanlon puts it, “The reason we can’t solve our problems is not lack of tools or information - it’s that the dominant method (add, optimize, measure) is the wrong method for the problem (figure out what’s poisoning you).”

Moreover, there is a grotesque endpoint in the shift from trying to be better to trying to be the best. It’s a shift from “as good as I can be” to “better than everyone else.” That doubled x-ed maxx implies there is a maxximum. A perfect destination that is as good as possible. It implies we’re all a peptide stack away from being better than everyone else.

But rejecting the extremes of maxxing culture doesn’t mean abandoning personal improvement. At the population level, bureaucratic pragmatism feels grotesquely crude. At the individual level, we celebrate people who seek to better themselves. Getting better often doesn’t need quantification. A person can get better at writing blogs or playing the guitar without validation from a numerical score. Moreover, we embrace difference at the individual level and acknowledge what works for one individual’s journey might not work for another. We even embrace relativism. Most of us accept that people can find what’s true for them personally by finding what works for them personally.

So what should we call trying to be better? What’s a word that juxtaposes against optimization? Tell me in the comments.

Subscribe now

Leif Weatherby, Tyler Shoemaker, and I have a new essay out today in The Ideas Letter about the creation of reality through pseudoscience, religion, and psychosis. Of course, it’s mostly about AI. Starting with the bizarre appearance of Antropic co-founder and mechanistic interpretability zealot Chris Olah at the side of Pope Leo as he read out his encyclical, we explore how commercial large language models automate the creation of reality.

The term of art for the process by which abstract concepts are made into material objects is reification. Marxist theorists think of the reification of social relations, where qualitative things like actions become commodified. This is the process by which labor becomes objectified and priced, and subsequently alienated from the person who does the work. Distinctly, historians and philosophers of science think of reification in terms of how correlational associations are named and become real through coordinated research programs. When your biggest companies all claim to be doing research, you can see how these two become one, with coordinated research becoming commodity itself.

LLMs are not only generated by a technological oligarchy, but also introduce a new twist into the process of reality generation. LLMs automate reification. LLMs are designed to produce explanations for us. We skip the steps in the collective creation of scientific concepts (aka thinking and arguing) and let LLMs do it for us through their linguistic association. LLMs entice you to skip the step of interpretation of outputs. Why not just ask Claude what it thinks?

As an illustrative example, we discuss Stephen J Gould’s historical analysis of the g factor in psychometrics research. In the social sciences, everything is correlated with everything. If you run some sort of correlational analysis on any data set, you’ll find something. Gould describes the process through which psychometricians decided that the correlational quantity that linked different measures of intelligence—the g factor—was real, and how they created a whole scientific research program to reinforce the reality of the g factor.

You know who still loves factor analysis? Mechanistic interpretability researchers. AI researchers have found themselves enamored with psychometrics as they work to prove their LLM toys are conscious. But they have new technology that Charles Spearman and his colleagues didn’t have a century ago: the LLMs themselves! Whereas 20th-century psychometrists had to think about causal stories to explain their strong correlational findings, interpretability researchers can automate this process. They can give an LLM the output of a factor analysis and ask it to explain the factors. Since it returns answers in natural language, this feels like discovery even though you have offloaded the process that would have been done by thinking about it to a machine. This process sets off a loop in which scientists and citizens alike give iterative credence to concepts generated by correlating text. We write:

“Models, in other words, kick off an associative chain of ideas by effectively auto-labeling queries. It’s like taking the principal components derived from that data about oak trees, boat speeds, cat whiskers, and Letterboxd reviews, and asking ChatGPT: “What do each of these artifacts mean?” ChatGPT will respond—and then keep the conversation going, bringing in more associations that more or less fit. But as we have already argued, this doesn’t only happen to amateurs who are easy to pathologize. How is this different from the standard methodology of interpretability research? In both the cases that might be dismissed as psychosis and the ones celebrated at AI conferences, interaction with LLMs induces mental friction. They create a feeling that discovery is there. By elaborating on what you put in through a context that the model has trained on, it is able to make connections that feel both correct and expansive, filling in the area around your thought—simulating the feeling that you are having a new thought. The model helps you refine the obscurity of your prompt through a chain of associations, and suddenly you have something. This is reification at work. And when the next link in a chain of thoughts comes along, it becomes hard to resist prompting the model again.”

LLMs are built to automate a complex web of reification. They are designed to tell us what we want to hear. As we write, “Trained on an enormous corpus of human writing, speech, and code, and tuned to refine responses around context and memory as user interactions unroll, a model of this kind is designed to provide the sense that one’s expectation is being exceeded.”

Artificial intelligence is a bizarre technology that participates in its own mythmaking. Anthropic is the most deeply invested in telling outrageous stories about its products, but no one in this space is innocent. As I’ve written before, I think that benchmarks get an unnecessarily bad rap. I certainly agree that you can’t “benchmark general purpose technology.” I agree that maxing out benchmark scores doesn’t mean a product will be better received by customers. But when we move away from benchmarking in artificial intelligence, we get stuck in storytelling. LLMs are designed and intended to convince us that their stories are real. When backed by a turbocharged capitalist engine, the reification becomes inevitable. Stories become commodities. Those commodities sell like hotcakes. And there is nothing more real than when the number goes up.

I’ll leave you there, as I need to go check on the value of my SpaceX shares. Read the whole article and let us know what you think.

Subscribe now

I was surprised by how divisive last Thursday’s post was. Some thought it was my best post, and others my worst. Let me convince you today that both assertions are true.

I understand why people are confused by the massive uncertainty in health-related decision making and enticed by evidence-based argumentation. When experts disagree, doesn’t that mean that someone is wrong? Shouldn’t there be a rigorous path to find the correct answer? Evidence-based medicine promised a reprieve from ambiguity through cold calculation. Its proponents still insist that biometry alone will lead us to universal truths.

Except, of course, it can’t. As I’ve been yammering on about, medical statistics tell us about population-level optimization and can’t say much about individuals. What a doctor believes about their patient comes from a complex synthesis of expertise and evidence. It never follows simply from a statistical decision-making computer program. EBM both displaces the physician as authoritative and displaces the patient into a statistical aggregate. Moreover, the conditions that establish population-level facts in a trial, like rigid inclusion criteria and strictly controlled experimentation, are simultaneously the conditions that render those facts inapplicable to any individual.

For anyone who’s read a bit of 20th-century French critical theory, that sounds an awful lot like a deconstructive moment. The founders of EBM thought of themselves as working in the pure spirit of Enlightenment rationalism. They were, in fact, unintentionally embracing postmodernism.

Postmodernism, fittingly, means many different things to many different people. Broadly speaking, it’s an embrace of the lack of universal truth, the fluidity of meaning, and the spirit of pluralism. Let me first explain some lessons worth drawing from the postmodernists, who have been far more right in their predictions than prognosticators ought to be. Then I’ll engage with those who receive postmodernism as nihilism and a rejection of plain facts.

Specifically, Jean-François Lyotard articulated how computerization led to an obsession with statistical optimization and an abandonment of grand shared narratives. He declared the postmodern condition to be the replacement of a collective search for universal truth with many disjoint searches for local optimality.

In the postmodern condition, scientific inquiry is reduced to asking whether interventions reduce costs, time, or deaths or increase shareholder value, productivity, or “well-being.” Knowledge is legitimated only if it can do something outside of itself. Research is funded only if it promises to improve performance. Science has to produce things that work. Whether or not any of these produce truth or understanding is irrelevant. Truth is justified through the attainment of ends.

Scaled-up statistical optimization is postmodern, be it in (evidence-based) medicine, economic development, or adtech. Once truth is arbitrated by number-go-up, you disregard explanation, mechanism, and experience in exchange for metric chasing. Lyotard warned us that if knowledge is governed by performance, then centralized technocratic power would get to decide what counts as knowledge and who gets to speak. In 1979, this needed a monograph. In 2026, it’s undeniable.

It is deliciously ironic that mathematical rationality—building statistical models and maximizing expected utility—is the epitome of Lyotard’s postmodernism. Those steeped in rationality would scoff and say that of course statistics isn’t postmodern.1 Rational calculation was supposed to be the purest form of modern reason. Instead, it has led to a fractured, virtualized, monetized cultural schizophrenia.

This is why the rationalist-adjacent crowd in Silicon Valley has been in a royal tizzy about the postmodernists and their antecedents. They have been fluffing jeremiads against “The Frankfurt School,” decrying postmodernism as the most harmful idea advanced against humanity, all while embracing attitudes toward reality that 20th-century postmodern theorists would have thought caricatures. Geoff Shullenberger has been writing great essays on this topic.2

Now, there’s a problem with pulling out the term postmodern. Lumping a bricolage of related ideas together under a catch-all also made a body of pressingly relevant thought easy to dismiss. I’m sure I’ve already lost people midway through this post because I chose to deploy it. The word “postmodern” tends to end conversations.

Indeed, that’s what happened in the 21st century. Scientists hated postmodernists who questioned their authority. Scientists also romantically put forward the idea that their theories explain universal truths about the fabric of reality and the technology we’ve built on top of it.3 In the 1990s, this led to heated public debates, culminating in the stupid controversy over Sokal’s hoax, in which a theoretical physicist got a fake paper filled with nonsense published in a postmodernism-friendly humanities journal. Postmodernism was thrown aside as unrigorous nonsense.

It was, of course, a pyrrhic victory. In the intervening decades, the academic STEM literature, with LLMs and perverse incentives, has Sokal-hoaxed itself to death. By contrast, the ensuing history has pretty much proven the postmodernists more than right.

So I’m choosing to use the term postmodern because it’s accurate and precise. The term explicitly captures the dialectic of mathematical rationality, the conflation of truth and utility, and the locality of efficacy. And though the French theorists wrote in an annoyingly inscrutable air of pessimism, I’d argue they offer glimpses of alternative political orders. It is perhaps only through productive disagreement that we might break free from the insidious, multiscale pressure of optimization.

Subscribe now

There is a tempting story you can tell about the cardiology megatrials I’ve written about in the past few posts. If you focus on a single health outcome (say, mortality) and a single disease (say, infarctus du myocarde, aka heart attack), you can optimize the treatment program by continually running randomized clinical trials. Each trial will carefully compare two nearly identical treatment regimens that differ only in a single step. Professional societies will regularly communicate the latest results and guidelines. Piecemeal improvements to the standard of care will be broadcast at meetings and on websites to keep all clinicians up to date on the best plan for every patient.

This approach can be applied to multiple diseases. With enough buy-in, these trials can be scaled up to provide personalized adjustments to the standard of care. Patients’ demographic information, family history, and behaviors can then be considered, and professional societies can craft specialized plans for each demographic bucket.

Now, statistical calculations would quickly tell you that you need far more than 8 billion people to make this optimized dream a reality. But let’s leave that aside for a minute. There’s something bizarre and alienating about this view. In this pipe dream, care becomes nothing more than a computer algorithm. A patient becomes nothing more than a classification assignment. Healthcare becomes nothing more than optimizing actuarial tables. Is this dignified? Is this care? Is this what we want the medical system to be?

There was a reactionary and revolutionary movement in the 1990s that thought yes, healthcare should be nothing more than the application of clinical data: evidence-based medicine.

EBM remains incredibly influential. I mean, who wants medicine that isn’t based on evidence, right? Back in 2020, I also had a brief moment when I thought this was a brilliant way to conceptualize medicine. For mathematically minded people, EBM has the appeal of rigor. It provides what appears to be a set of clear rules for deciding on medical treatment. There is a hierarchy of evidence. A pyramid, if you will:

Expert opinion is the lowest form of evidence. Case studies are barely better. Epidemiological studies would be of slightly higher grade, but these are too easily fooled by confounding causes. A big step above, a gold standard, is the randomized controlled trial. And at the very top, we place an even higher grade on systematic reviews of all randomized controlled trials conducted on a disease.1

Once you have this evidence in hand, you can make clear decisions. You don’t need expertise. You simply need a Meehlian formula: you plug all of the data you have about a patient into a computer. The computer spits out a treatment plan for the patient based on the highest grade evidence available. Healthcare is now solved.

You might think I’m caricaturing their position, but you can go back to the original position papers, and the quotes are even stronger than what I say. Here’s the opening paragraph of “Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine,” written by a working group chaired by EBM pioneer Gordan Guyatt:

“A new paradigm for medical practice is emerging. Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research. Evidence-based medicine requires new skills of the physician, including efficient literature searching and the application of formal rules of evidence evaluating the clinical literature.”

The working group was explicit about expert judgment: “The new paradigm puts a much lower value on authority. The underlying belief is that physicians can gain the skills to make independent assessments of evidence and thus evaluate the credibility of opinions being offered by experts.” In their view, expert intuition and creative care caused more harm than good. Moreover, they thought a move to computerized decision making would improve patient outcomes, writing “A final assumption of the new paradigm is that physicians whose practice is based on an understanding of the underlying evidence will provide superior patient care.”

Computerization was exactly what they had in mind. David Sackett put together a prototype treatment computer, called The Evidence Cart, in the early nineties. Check him out making a diagnosis:

Sackett stuffed the evidence cart with a compendium of information: BestEvidence, the JAMA Rational Clinical Examination series, the Cochrane Library, multiple textbooks, and material compiled by his collaborators from journal groups and other assessments. Sackett’s crew would wheel the Evidence Cart into a patient room and evaluate each decision against the best evidence.

Did it work? Hilariously, the evidence that the Evidence Cart works is a tiny, single-center observational study, not an RCT. But the influence of EBM and the goal of making Sackett’s dream a reality was undeniable. Doctors now have supercomputers in their pockets at all times. They can pull out medical calculators, reduce patients to categories, and prescribe treatment plans based on the best evidence. Every doctor-patient visit is hooked up to a different computer that serves not only as a way for doctors to keep records of how their patients are doing, but also as a way to file them as statistics in actuarial tables at insurance companies and to enter their virtualized identity into the pool of future observational studies.

Has healthcare landed in a good spot? It’s complicated. Obviously, studies are good. But the idea that you can synthesize an optimal decision from the medical literature is crazy, whether you have modern LLMs or not. You can’t do an RCT of every possible option. We’ve already seen that expert knowledge is needed to decide which RCTs to do and to make sense of RCTs that seemingly contradict each other. And the medical literature does not tell you what to do with an individual patient. Patients are not just category labels. Care is more than robotic decision-making.

I’m more sympathetic to the iatrogenic argument put forward by the EBM pioneers. Many were medical conservatives. They thought that most treatments not only lacked evidence but were also harmful. You can look at the barbarism that riddles the history of medicine and certainly see that they had a point. Medical conservatives believe that we should err on the side of treating less not more. EBM was partially a battle against the intervention bias that still plagues medicine. “We have to do something” feels right, but is often harmful. This is why the CAST trial on anti-arrhythmia drugs is a poster child for the movement.

Though well-intentioned, evidence-based medicine is one of the best case studies of how the quantification trap leads to madness. We computerize everything to displace the influence of narratives. Mechanism and natural law don’t matter. The only thing that matters is optimizing outcomes. The pursuit of medical knowledge is only to optimize actuarial tables, not to understand the human condition. Medicine becomes a video game. This is the purest form of what Jean-François Lyotard called postmodernism. Statistical thinking run amok is the postmodern condition.

Subscribe now

You might have noticed a pattern in last week’s blog history of the megatrials in cardiology. All the breakthrough trials I covered were essentially drug trials of anticoagulant interventions like aspirin, streptokinase, and heparin. This, of course, wasn’t just a coincidence.

Though randomized trials are often pitched as the ultimate arbiters of causality, you need to have a good reason to do one. You can’t (and don’t) just propose a random trial with a control group. Not only must there be considerable uncertainty about the outcome to get a trial approved, but there must also be some reason to do the trial in the first place. There has to be some intuition that leads some investigator to propose some intervention, and they have to have enough faith in that intervention to devote massive resources to proving it’s efficacious. Though randomized trials sit far above expert opinion in the hierarchy of clinical evidence, expert opinion necessarily must still drive the operation.

So what was the core hypothesis driving the series of anti-coagulant megatrials? Whether clots were a primary cause of heart attacks was a hot debate in cardiology well into the 1980s. The prominent theory for much of the 20th century posited that it was arterial hardening and valve narrowing that caused heart attacks. In this model, blood clots were caused by the heart attack and not the other way around. This felt consistent with the evidence, as most of those who died from heart attacks didn’t have blood clots in their autopsies.

It wasn’t until 1980 that a paradigm-shifting study showed that blocked arteries were overwhelmingly common in the early stages of heart attacks. The investigators corroborated earlier findings that the clots were less prevalent the following day. This suggested that some biological mechanism was naturally breaking down the clots during heart attacks, and suggested a mode of attack for treatment. Cardiologists spent a decade finding new and creative ways to break down blood clots in the heart and intervene as early as possible, leading to marked improvements in survival.

The large pragmatic drug trials were important for building practice. They did find a fairly robust suite of therapies for treating heart attacks. Could an individual cardiologist have determined a better regimen? Possibly. But the megatrial infrastructure of cardiology trials proved that, for the right clinical hypothesis, professional societies of trialists could maximize population outcomes to achieve a reasonable standard of care with 3-6 times lower mortality than the prior one.

Now, I also mentioned a trial that wasn’t about coagulation: CAST. As a reminder, the CAST trial studied the value of drugs that quelled irregular heartbeats in heart attack patients. The trial found that these therapies led to a major increase in mortality, and it ended the practice. But why were doctors convinced that this was a good idea in the first place?

I suppose it might sound reasonable to make a sick heart look “healthier” by messing with its rhythm. But what’s the model behind why that would be a good idea? I’m sure there is some grand rounds somewhere that lays out the full hypothesis, but I couldn’t find one, and the literature associated with the CAST trial doesn’t provide it. Instead, the trial report cites a bunch of correlational studies showing that arrhythmia is correlated with mortality in heart attack patients. The main driving hypothesis seems to be this statistical correlation. Expert opinion on anti-arrhythmia drugs was decidedly mixed before CAST, and that’s part of the reason the trial went forward.

It’s worth dwelling on one remarkable feature of the CAST trial. The investigators were allowed to compare against a placebo. Without the placebo arm, the bad practice of treating arrhythmias would not have been ended. It remains a heated debate in medical ethics as to when it’s acceptable to compare an intervention to a placebo rather than to the standard of care. There is a strong case for placebos. A cascade of RCTs has a great deal of path dependence. Every trial slightly adjusts the standard of care along a single axis. This is what optimizers call random search, and there is nothing to prevent it from ending up in a disadvantageous local minimum. Thus, trials against the standard of care might keep you in an iatrogenic region of medical policy. It is quite possible that the standard of care needs to be completely reimagined to achieve the next major therapeutic improvement. As evidence, this was exactly what was needed to force cardiology to shift to anticoagulant therapy. CAST remains a classic reminder that it is not just ethical but often imperative to test against placebos.

Though often held up as a poster child for the megatrial, the full history of cardiology looks an awful lot like the rest of science and engineering. There isn’t a single magic bullet that automatically led to improved therapies. There was a mixture of expert opinion and institutional debate. Small studies inspired huge randomized trials. The megatrials were guided by biological plausibility and mechanistic intuition. The RCT was clearly an important part of improving practice, but it was part of a complex curation of expertise, rather than a cut-and-dried gold standard.

However, some people stubbornly maintain that RCTs are the only pure way to get to the truth of what works. This belief led to one of the weirder moves in medical history that’s still with us today: The rise of the postmodern thinking of evidence-based medicine. This is the topic of my next post.

Subscribe now

I’m always looking for examples where we need statistical reasoning and significance tests to change care, and I’m surprised no one has shoved cardiology in my face before.1 Cardiologists can make a powerful case that their field has been completely revolutionized by turning to randomized controlled trials to guide their practice.

In the late 1980s, cardiologists ran massive clinical trials to improve the standard of care for heart attacks. The GISSI trial enrolled 12,000 patients and found a 2% reduction in hospital mortality when treating heart attack patients with the anti-clotting medicine streptokinase. ISIS-2 (17,000 patients!) found that adding aspirin as an additional anti-clotting agent resulted in an additional 2% reduction in mortality.

These percentages were absolute percentages. With neither treatment, 12% of the patients died within five weeks of the heart attack. With aspirin and streptokinase, that percentage dropped to 8%. For those who care about these things (like me and three other people on the internet), the p-values in these studies were all on the order of 1 in a million. The trial size here mattered a lot. Had the trials been 10 times smaller, the same event rates would not have passed a standard p<0.05 significance threshold. These trials are textbook cases for RCT gold standard evangelists.

Moreover, the megatrial culture in cardiology has famous examples of rooting out harmful practices. The most famous study is the CAST trial of the early 1990s. Standard practice had been to pharmacologically suppress arrhythmias after heart attacks. Using drugs to make the heart look more “normal” seemed like a good idea. The CAST trialists enrolled 1,500 patients and shockingly found significant harm. 5% of patients died within 10 months on the anti-arrhythmia treatment, whereas only 2.5% died on placebo. The confidence intervals were narrow, and the p-value was again tiny. Something that felt reasonable—suppressing unusual heartbeats—was deemed harmful, and the practice was ended.

What can be said of the net benefits of these treatments after 40 years? Though robust, the effect sizes in these trials are all small, ranging from 1% to 3%. How can we be sure that the effects are cumulative and that modifying the standard of care is actually helping cardiology patients?

Regrettably, we now have to turn to epidemiology. No IRB will authorize an RCT of the old methods—essentially bed rest and oxygen—against the current therapeutic regimen of angioplasty, clot reducers, blood thinners, beta blockers, and statins. That would be akin to doing an RCT with a control group assigned to blood letting. But the improvement in survival of heart attacks is undeniable. Estimates suggest that the current death rates have dropped from somewhere in the range of 15-20% to about 4-5%. That’s quite astounding. 50 years of improving practice have accumulated a 3-6 fold reduction in deaths. You couldn’t ask for something better. Cardiology is a compelling and fascinating case study of the power of outcome optimization.

Why was there so much success in cardiology? You could argue that heart attack is a nearly ideal case for this sort of trial-based optimization. The endpoint of “death” is the most unambiguous in medicine. The adverse endpoint occurs fairly quickly (within weeks), as opposed to, say, oncology, where treatments can take years to assess. Moreover, heart attacks are unfortunately very common. The silver lining of their commonality is large pragmatic trials are relatively easy to assemble. Large trials are essential when the effect sizes are only 1-2%.

Now, even though cardiology is a poster child for evidence-based medicine, it’s important to note how the actual advancement of practice was not simply by chaining together a sequence of massive RCTs. First, not every trial was as unambiguous as GISSI, ISIS-2, and CAST. Two trials in the 1990s assessed the relative value of streptokinase and tPA, two anticoagulant agents. The GUSTO trial enrolled 41,000 patients and found tPA reduced death by 1%. The GISSI-2 trial enrolled 12,500 patients and found no difference between tPA and streptokinase. They contradicted each other! Post-trial analyses concluded that the trials had administered tPA differently, and this explained why GUSTO found a benefit. It was a careful study of the trial after the fact that suggested the true benefit of the drug. This analysis required an appeal to what pharmacological knowledge, and wasn’t simply adjudicated by randomized experimentation. Moreover, a second post-trial analysis concluded that tPA increased the risk of stroke. The story is complicated. Mega-trials alone can’t solve practice.

Even the unambiguous trials were already pointing to a major challenge with RCTs. As a treatment regimen becomes more complex, ironing out the fine details requires an exponentially increasing number of RCTs. If you want to compare the effect of three different timings and three different dosages of a single drug, you need nine arms in your trial. If you want to additionally see if a second drug is helpful, you need 18.

Finally, not every guideline of practice comes from randomized trials. Fewer than 10% of the American College of Cardiology/American Heart Association clinical guidelines are backed by large RCTs. I was surprised to learn that there is no convincing RCT showing that bed rest is harmful. We have ended the practice of long-term bed rest regardless. So how cardiologists make recommendations to their patients remains complicated. How does this sort of therapy relate to individual doctor-patient experiences? That’s the next question I’m hoping to answer.

Subscribe now

In passing, I mentioned a new book by C. Thi Nguyen, The Score, which asks the question: Why are numerical scores fun in video games yet oppressive in social metrics? Or, more succinctly: Why do we love games and hate rules?

The Score is simultaneously a philosophy book, a self-help book, and a gentle introduction to the contemporary academic study of institutions. I applaud Nguyen for recognizing that a philosophy of games and rules needs to engage with ethnographic disciplines to make sense of why metric chasing is core to our current condition. His book makes the tension between individuals and populations more visible for those less willing to immerse themselves in the vast literature of science and technology studies.

Nguyen clearly summarizes the work on bureaucracy, statistics, and rules by scholars like Lorraine Daston, Theodore Porter, and anarchist anthropologist James Scott, whose classic Seeing Like a State is beloved by both the left and the reactionary right. But there’s another anarchist anthropologist who I think has already solved the core dilemma of The Score: David Graeber.

Graeber was not only an academic but a dedicated political activist. He was one of the central figures of the Occupy Movement, credited with helping coin its iconic slogan, “We are the 99%.” Though best known for his books Debt and Bullshit Jobs, my personal favorite is The Utopia of Rules. I like to think the subtitle was microtargeted to me: On Technology, Stupidity, and the Secret Joys of Bureaucracy. The book is a collection of five essays that, though readable separately, together provide a unified answer to Nguyen’s central question.

Graeber frames games and bureaucracy as offering similar utopian fantasies of fairness and equality. In games, everyone plays by the same rules. The outcomes are transparent. We know what it means to win and lose. Since everything is written down, we can all evaluate if we think it’s fair and argue for rule changes if it’s not. But bureaucracy promises the same thing. It has no shortage of written rules! We only have all of those rules to maintain transparency and fairness. We want everyone to be treated equally under the law, don’t we?

Score-based games let us escape in temporary fantasy, and yet, in their complex scoring systems, Graeber writes, they “reinforce the sense we live in a universe where accounting procedures define the very fabric of reality.” Video games and bureaucratic mechanisms are two reflections of the same reality. Ironically, the score-based games that Nguyen celebrates close off the imagination of alternative forms of governance.

Still, we love our board games and video games, and we hate going to the DMV. Why is that? What explains how getting enmeshed in a battle with HR or the IRS is terrifying and emotionally crippling? What explains why our popular imagination paints bureaucracy in surrealist, existential nightmares like in The Trial, Brazil, or Andor?

For Graeber, the main difference between games and bureaucracy is what he calls structural violence—“forms of pervasive social inequality that are ultimately backed up by the threat of physical harm.” Graeber argues that bureaucratic systems of endless, stupid paperwork are the defining example of structural violence in our society.

“Now, I admit that this emphasis on violence might seem odd. We are not used to thinking of nursing homes or banks or even HMOs as violent institutions—except perhaps in the most abstract and metaphorical sense. But the violence I’m referring to here is not abstract. I am not speaking of conceptual violence. I am speaking of violence in the literal sense: the kind that involves, say, one person hitting another over the head with a wooden stick. All of these are institutions involved in the allocation of resources within a system of property rights regulated and guaranteed by governments in a system that ultimately rests on the threat of force. ‘Force’ in turn is just a euphemistic way to refer to violence: that is, the ability to call up people dressed in uniforms, willing to threaten to hit others over the head with wooden sticks.”

It is this threat of force that makes bureaucracies so stupid. To see why, begin with violence itself. It is one of the only forms of human interaction that requires no interpersonal interpretation. We know exactly what happens with the conditional command: “Cross this line and I’ll shoot.” When you issue this edict, you are running a mechanical algorithm of violence that needs no understanding of the person who is coming at you. If they cross the line, you shoot them. Both you and your counterparty have a precise expectation of what happens after you pull the trigger.

This lack of interpretation is afforded only to those who have power and can actualize violence without repercussion. This consequent ability to harm makes those in power lazy. And the structures built to maintain these power structures become lazy with them.

Indeed, bureaucratic rules let those in power remain oblivious to what’s actually happening to everyone else. The stupidity of bureaucracy is a feature for those in power. It removes their need to understand those who are subject to the rules. Graeber’s argument is completely consistent with the views of stout institutionalists who tout the benefits and necessity of bureaucracy. Part of the benefit of bureaucratic systems is how they abstract complexity up a hierarchy, allowing people at each level to act without knowing all the details of what’s happening below them.

However, Graeber foregrounds the people at the bottom who are erased by quantified summaries. The erasure of those individuals into statistics means that those in power have no need to do the interpretive labor of thinking what it must be like to be them. Those who set the rules are privileged to not have to think about those forced to abide by them. By sharp contrast, those who have to deal with the rules are obliged to empathize with the powerful. They must constantly imagine how the powerful might act and react to avoid the persistent threat of violence. This is how normally intelligent people are forced to act like idiots when dealing with bureaucratic procedures.

Summarization and bureaucratization do not have to be stupid. You can read my architecture lecture from a few weeks back to see how well-designed hierarchical rule systems can create amazing outcomes. Graeber, to his credit, doesn’t disagree.

“To put it crudely: it is not so much that bureaucratic procedures are inherently stupid, or even that they tend to produce behavior that they themselves define as stupid—though they do do that—but rather, that they are invariably ways of managing social situations that are already stupid because they are founded on structural violence.”

Bureaucracy is stupid when it is used as a system of deempathization and structural violence. When rule-based systems reduce the interpretative labor of those in power, “such procedures come to partake of the very blindness and foolishness they seek to manage.”

Now, I don’t think you’re going to reach Ezra Klein and Derek Thompson listeners with Graeber’s far-left radicalism. You’re definitely not going to reach the staunch institutionalist liberals of Bluesky who hate David Graeber with every ounce of their being. I don’t mean this cynically: I think Nguyen wants to reach out to both of those audiences, and that’s his prerogative.

However, I think we are best off not forgetting Graeber and the left-wing movements that arose in the wake of the financial crisis. Though the stock market is soaring, it’s hard to argue the world is in a better place today than it was after 2008. The economist Joseph Stiglitz’s so-called 1% has lost some of its power, but only because it has ceded it to the 0.001%. There’s less faith in institutions than ever. The financialization and gamification of everything have put us all in the awkward position where every aspect of our lives is now connected to a risky set of dehumanizing rules. The radical critiques from the 2010s don’t have simple answers for our current polycrisis, but ignoring them walls off imagining better worlds.

Subscribe now

I had a bit of a throwaway line in the last post about how maximizing the welfare of populations requires the erasure of individuals. You might ask why? Individuals are units in a broader population. A population is a group of individuals. Improving the welfare at scale must improve the welfare of the units.

Except we all know this isn’t true. Maximizing averages doesn’t say anything about the outcomes of any particular individual. In fact, decisions that maximize averages often harm some of the individuals in the total sum. Individuals in a community always have some shared interests, but they have plenty of disparate interests too. Which interests are maximized is a political decision that necessarily leaves other interests neglected. Our metrics and measures can have broad societal value while still making many unhappy.

And how should those individuals make decisions about their own lives? You may be able to convince yourself that Mathematical Rationality makes sense for a bureaucratic state or company. However, it’s much harder to make the case for being a mathematically rational individual.1 Quantification, abstraction, hierarchy, and statistics can help organize and steer decision making at scale. But if one of the core goals of quantification is legibility for intersubjectivity, why do you need numbers to make sense of your personal experience? Why is it useful to see yourself like a state?

Nothing motivates my research more than this tension between the population and the individual. It’s been my main focus since 2020.2 But getting traction on these topics has been an uphill battle. Try telling someone in the human-facing sciences that you want to study the epistemology of case studies. It’s so easy to fall into the cracks of crankhood.

Now, of course, scientists are incapable of seeing the pure irrationality of science. Blindly applying population results to individuals requires a lot of faith. We have formal scientific language to understand population averages. This language is incoherent when directed back towards individuals. Take our gold standard of causal inference, the randomized controlled experiment. These trials can estimate the fraction of people in an experimental cohort who would benefit from taking a drug. But let’s say you do a trial with 600 people and find that 20% of the control group has a bad outcome, and 10% of the treatment group has a bad outcome.3 What does this say about my outcome? Unfortunately, that result alone says nothing. We’d like to argue that the intervention reduces my risk by a factor of 2. But what even is individual risk?

Despite this bizarre inability to really say anything about individual benefit, when you try to come up with a non-statistical, non-quantized language to say precise things about people, you are relegated to the bucket of pseudoscience or, if you can bench enough, bro-science. Anecdotes are not data. Your miracle cure is always a fluke. Your personal experience is trumped by this impenetrable 500-page systematic review. Anyone who disagrees with the consensus of experts is being an irrational contrarian.

However, a lot of practices people find beneficial are immune to the postmodern lens of the randomized trial. It’s really hard to do RCTs on organ transplantation. You can’t do RCTs for physical therapy. You certainly can’t do them for massage or chiropractic. It’s hilarious that we have convinced ourselves that we can do this for psychology, despite decades of embarrassing “scientific” failures. And when you start looking at “sports science,” you realize how silly it is to try to put a scientific corset on all of human experience. No randomized trial explains Victor Wembanyama.

I could pick on more than just the RCT. Individuals don’t exist in the calculus of rationality. None of the pillars of mathematical rationality I talk about in The Irrational Decision make much sense for individual people. I could give similar spiels about game theory, statistical prediction, or optimization.

Optimization, in particular, is tough to grapple with. I got a lot of questions about personal optimization in the conversations about my book. Many people find it useful to think about their lives as a collection of optimization problems. If you want to strive for the best, that means some number must go up, right?

Don’t get me wrong, I love optimizing too! Do I obsess about my home coffee setup, my exercise program, my writing schedule? You bet I do. Is that bad? Does that mean I’m just a cog in the capitalist machine? These questions form the basis of a conversation worth having.

Now, here’s an annoying paradox. If we want a language to talk about individual experience, it has to have some element of intersubjectivity. This is where the quantification trap comes in. The mimetic power of the quantification trap means that a shared language for discussing individual experience is always at risk of being contaminated by scientific quantification. But it doesn’t have to. Most people share their experiences without numbers and charts. We obviously share experience through art, music, and literature. These are all shared languages, too.

For the next bit on this blog, I want to find language to talk to each other about individual experiences. I want to write about this weird tension between the quantification trap and the individual. People figure out how to do amazing things without consulting the scientific literature all the time. How can we talk about commonalities without reducing them to numbers or statistics? I initially thought this would be the topic of the final chapter of The Irrational Decision, but I realized it was far too sprawling and unwieldy to fit. It will have to be its own book. Some day.

In the meantime, I’m going to try to type it out on here.

Subscribe now

If we want to make decisions in a complex society, we need a shared language. Experts on the ground must summarize complex situations in their communication with decision makers decoupled from the field. They need to make their experiences legible to those they report to.

The easiest way to make situations legible is to quantify them. To count things, record figures in tables, compute statistics, and make charts. Quantification sorts complexity into simple bins, simplifying communication both up and down the chain.

When we speak in such quantified numerical summaries, our statements feel objective. We believe that appropriate quantification isn’t be subject to the whims and opinions of an individual field worker. By agreeing upon standards, quantified measurements are now scientific facts.

Once we have objectivity, we have authority. Making decisions based on objective facts is obviously in the best interest of everyone else, and we impose threats of chastisement, ostracism, or violence upon those irrational individuals who disagree.

And once these numerical summaries that we made out of whole cloth to simplify communication become authoritative, they become real. They become things we should strive to maximize.

This is the quantification trap.

The quantification trap is social-scientific canon. You could build this story entirely out of texts written before the year 2000. The role of quantification, measurement, and legibility in statecraft is laid out by James C. Scott’s Seeing Like a State (1998) and Alain Desrosières The Politics of Large Numbers (1993). Theodore Porter’s Trust In Numbers (1995) highlights the turn to quantification in pursuit of standardization and objectivity. The blind optimization of decontextualized metrics is core to Jean-François Lyotard’s characterization of The Postmodern Condition (1979).

Twenty-five years into the twenty-first century, I don’t think you should have to run a Science and Technology Studies sidequest to recognize the quantification trap. It’s obvious and almost trite when we say it out loud. It’s trendy to talk about how metrics and benchmarks are bad and to prattle on endlessly about Goodhart’s, Campbell’s, or Murphy’s Laws. And yet, we continue to organize ourselves around statistical summaries. Is the quantification trap an inevitable part of scale? Is it an inevitable part of efficiency? Is it an inevitable part of the dismal hierarchy of bureaucratic power? The great puzzle of our contemporary condition is why it’s so hard to escape.

Part of the puzzle is that making society computable has dramatic benefits paired with every cost. The constant tension in mathematical rationality lies in the interplay between its sweet spots and its limitations. The quantification trap creates an intersubjectivity for collective action. Mathematically rational governance lets systems and hierarchies see, but also makes it easy to maintain their control. It facilitates posing clear questions and objectives, though crowds out nuance and multiplicity. It creates a shared understanding of standardization but removes the discretion of experts. It lets us speak about maximizing the average welfare of populations, but erases individuals.

If there are such clear trade-offs with quantification, why do we always tend to side with “the data?” The acceleration of computation has made the quantification trap exponentially more contagious. As computers became ubiquitous, the quantification process became inevitable and invisible. We don’t think about how we are tethered to unfathomable computing machines. They’re just part of who we are now. Our devices measure us all the time, recording time-on-site and click-throughs. Everything has a like button. All of these measurements are churned upon by data scientists hoping to hit their personal promotion metrics, regardless of whether the instrumentation means anything. The quantification trap is built out of an invisible fabric of computation.

I feel like I say this in the book, but never say it in the Irrational Decision. The book articulates the role of mathematical computation, optimization, and statistics as scaffolding in the elaborate quantification trap. To understand why we optimize what we optimize, it’s helpful to look at the history of computational methods and language boxing us in. The path from legibility to authority goes straight through computation and computerization. Quantification transforms experience into machine-readable data and a small number of interventions and outcomes. Decisions can only be automated once we throw away the messy, uncomputable parts. We maximize averages because it’s a convenient way to model uncertainty.

Now, I am by far not the only person to talk about the quantification trap. I wrote about it today because I felt I needed this placeholder after the last few weeks of talking about my book. However, if you want a reading list from the past 25 years, I can write us an impossibly long bibliography. Even in the past year, crossover books like Healy and Fourcade’s The Ordinal Society and Nguyen’s The Score have articulated the same conundrum.

It’s good that more people are talking about this. What we count, compute, and optimize is a political decision. Counting flattens complexity, and the choice of what is left is a question of power. The virality of the quantification trap forecloses better futures. We can’t strive for them if we can’t see the gilded cage we’re in.

Subscribe now

My least favorite part of writing books is their permanence. Minutes after I spend time with a hardcopy, I start seeing a bunch of things I should have added, removed, or modified. I’m not talking about typos—I expect and accept those will be abundant—but rather ways I’d have written the book differently if I could do it again. In particular, this last week of Irrational Decision events crystallized a few things that I wish I had commented on, but maybe couldn’t have.

I’ll spend the next few blog posts going over the questions raised that I wish I had addressed in the book. I was unhappy with my extemporaneous replies, and I’m hoping blogging might bring me more satisfactory answers.

I’ll start with the trillion-dollar elephant in the room: language machines. In our conversation on Tuesday, Lily Hu asked me how LLMs denote a shift away from the mathematical rationality of The Irrational Decision. For a long time, artificial intelligence systems were built by solving problems differently from how people solved them. For example, computers are better at chess than humans, but they do not approximate the way master chess players play. But LLMs, especially the chatbot interaction model that captured global attention and rocketed the valuations of these companies into the stratosphere, uncannily seem to act and respond like us. This seems like a departure from the cold rationality of computation.

The chat interface leans on the ambiguity of language to make for a satisfying interactive experience. Joseph Weizenbaum famously lamented that when his colleagues tried to make a computer appear intelligent, they leaned on psychological parlor tricks. A machine that summarizes in iambic pentameter appears intelligent. A machine that solves a linear program in millions of variables is merely mechanically calculating.

So what does it mean to lean on language machines for decision making? This is a tricky question because LLMs are unfathomably complex software artifacts. If you use them under the harness of a coding agent, you can get them to optimize all sorts of things. For example, they are incredible at making code faster, outperforming any autotuning tool I’ve ever tried. You can easily get them to implement a Bayesian rational choice engine for you.

Certainly, the rationalist weirdos in charge at Anthropic have demanded that Claude be trained to recite back summaries of dogma from the lesswrong.com comments section. So LLMs can and will parrot back the tenets of mathematical rationality to you. But natural language is not rational language. Indeed, the language of mathematical rationality is a Bayesian language game, always working to box out the unmeasurable and unquantifiable. It demands language without ambiguity, but of course, language is always ambiguous, fluid, and evolving.

You can try to squeeze out the ambiguity by prompting the machines to reply in logically structured language, like software or mathematics. Yet, even when you try to get the chat interfaces to give you rational answers, you still find perplexing mistakes that aren’t even logical, let alone rational. I liked this glaringly obvious one from last week.

These sorts of errors remain vexingly common, but are becoming increasingly difficult to spot.

The coding machines I mentioned above, with their ability to execute, check errors, and optimize, feel like they are more rational. However, they force programmers to cosplay as project managers. Software engineers have to prompt in natural language that is kind, inspiring, and encouraging. There are massively popular GitHub repos that help you find the right phrasing to make sure your eager agents have the right artificial mindset so they don’t make mistakes. This doesn’t seem very mathematically rational either.

Moreover, in the broader context of decision making, the vast majority of people do not have LLMs write optimization code to compute the decisions for them. Instead, they treat the chat like a consultant, and get interactive text back, sometimes with charts and figures for quantified comfort. Being advised by these outputs as to how to structure your life or business is closer to a magic eight ball than to a linear program. Writing in imprecise language and having a system trained to respond nicely certainly isn’t mathematical rationality. It’s what the early mathematicians tried to excise from decision making. What a weird turn.

But if we step back from the chat box, it’s not hard to place LLMs on a linear axis of mathematical rationality. The people who build these things religiously believe in mathematical rationality. They endorse the Bostrom-Russell nonsense that intelligence is attached to singular objective functions, and our robots will kill our dogs to make us coffee. Alignment researchers claim we just have to find the right objective function for post-training, and then we’ll have perfect human companions to build us a utopia of abundance.

Similarly, the guts of LLMs are the same computational pillars I describe in the book. LLMs seem like us, but they are still built upon the very unnatural model of statistically summarizing language and code via maximum likelihood estimation. People don’t do this.

Language machines run on complex software, hardware, and network infrastructure, but none of it would look alien to a computer engineer from 2006. Just like every computing system of the past two decades, they are tuned by optimizing engagement, maximizing userbase preferences in glorified A/B tests. The advances are charted by the same sort of rational, average-case benchmarking we’ve been doing in machine learning for decades. We make them better by blindly optimizing without understanding.

And let us not forget that these products were built upon fraud, theft, and rent extraction. They are enriching a small group of incredibly annoying people who claim a mantle of mathematical rationality and are promising to subjugate the rest of us in unemployment work camps. Though I started this blog lamenting what I didn’t cover in The Irrational Decision, maybe I got this one right: That we built this wildly irrational technology with mathematical rationality corroborates my argument.

Subscribe now