The Least Agentic People Alive

The arms race of deferring agency and the dark acquiescence to robotic bureaucracy.

I’ve started and stopped four posts this week and couldn’t figure out why I was stuck. Finally this morning, in a delightful retelling of an Alan Moore comic, Henry Farrell struck the nail in my head.

Henry describes Moore’s society where a supergenius benevolent dictator named Abelard Snazz designs giant police robots to fight crime. The robots get too zealous and start bothering ordinary citizens. So Snazz deploys criminal robots to keep the police busy. The cycle continues until people are forced to flee the planet.

Henry, always keen to show us what sci-fi tells us about our lived experience, describes how this automation is playing out in our lives, with people heading to chatbots to deal with bureaucracies, and governments planning new AI systems to deal with the onslaught of ChatGPT protestations. We know where this ends. And this week has seen way too many stories of this abandonment of agency and succumbing to AI.

I drafted a blog about our annual complain-fest about the academic wreckage that is the NeurIPS conference. Everyone is taking to social media to complain that people are shirking their reviewing responsibilities. They scoff at new limits of 25 papers per author per conference. They lament that AI papers and referee reports are now better than those submitted by people. They say no one reads any of these papers. And yet, everyone continues to submit papers, review papers, and be area chairs.

It is completely unclear what everyone wants from this process anymore. With every purported fix, the process gets more onerous, and the problems only get worse. More papers are submitted, more hair is torn out in frustration. Every day I’m reminded of what Kevin Baker wrote about this acceleration of publishing and perishing: “Systems can persist in dysfunction indefinitely, and absurdity is not self-correcting.” But Kevin’s next sentence is more damning. “Whether the acceleration produces collapse or adaptation or simply more of the same is not a question about the technology, and it won’t be answered by debates about capabilities.“

I drafted a blog post about open letters from employees of tech companies. They warn of all of the potential harms of the technology they are building. They beg for the government to regulate them. They decry their helplessness as they collect their 7-figure remuneration. Apparently they can’t see a path to do anything themselves.

These people want us to trust them, but then they keep admitting they are committing crimes. They write about their software as a person, trying to deflect from their own incompetence in security standards. They demand strict testing of open models while bragging about their laughably sloppy practices. They demand that the government come in and regulate them, because they can’t get out of their own way.

Even in esoteric places, we see a deferral to the AI. The lore laundering machines are now good enough to prove actual theorems, finding the key clever steps needed to take the existing literature and close “major” open problems.¹ There’s a lot to say about what’s happening here. It’s still lore laundering, and mathematicians are rightfully angry that the models remain horrible at proper attribution. But the new models are generating much more clever stitches to glue the proofs together.

OpenAI has been running a distributed denial-of-service attack on unsolved math problems, and their internal tooling has finally been able to solve some very hard ones. Only three people in the world know what a non-sofic group is and why it’s important,² but the new OpenAI Astra model has managed to construct one. The mathematical progress has created an arms race in mathematics, where people are telling mathematicians that they need to do all their work with ChatGPT now or be left behind. Or even worse, that there’s no need for mathematicians anymore. What this means for mathematics and applied mathematics more generally remains to be seen. But I guarantee you that people are going to quickly tire of theory papers on arXiv that are exclusively transcripts of professors having conversations with ChatGPT.

As Henry writes, all of these cases are arms races. As with all arms races, the momentum can feel insurmountable. I’m surrounded by people throwing up their hands as if they have no agency. But sometimes it’s important to step back for a second and see that where we’re heading is not inevitable. We do have choices here.

Researchers don’t have to send papers by the tens of thousands to conferences where no one reads anything. They can collectively design better systems to generate, evaluate, and share knowledge. Employees can use the power of their labor to steer their companies, rather than pleading for someone else to step in. Mathematicians can continue to think, understand, and teach. We all have the power to do something else.

1. There are no major open problems in math in the sense that the rest of the world would notice once they were solved. The only thing that happens when someone solves them is that mathematicians open new problems.

2. None of them work at OpenAI.