Indeed the Neurips arms race is pathetic but very funny. And I have zero sympathy for any complainers; I just laugh. The solution is trivial, and takes two steps:
1) Just stop submitting, and disengage. That is what I did. It is very easy. I used to enjoy the conference (from about 1990-2015) because I liked talking to people there, but after 2019 I vowed never again (even now it is heading to Sydney because of work I did many years ago to get it there!). I have turned down every request to be a reviewer, area chair, senior area chair, and even senior program chair (a couple of years ago) and am delighted I did. And I don't submit there any more.
2) Stop judging people in terms of whether they are named as authors on papers published there (note my careful wording). Alas, my professorial colleagues (world-wide I mean) are the worst offenders here. This too is pathetic...
[An alternative (only slightly flippant) suggestion which I have made to a Neurips board member is to decouple the papers from the conference: by all means have the conference, for people to confer, but do not tie it to any publication; separately, have a publication venue (aka a journal). Demand submitters to pay handsomely for each submission. Use the vast profits to pay for nice food etc for the conference attendees who would then attend for the sake of conversations. The conference would no doubt shrink which would be a fine thing, and it might be fun to attend again...]
Regarding your other points, well Geoff Hinton reckoned there would be no radiologists any more (I am very glad he was wrong, having interacted with them to my considerable benefit). And other technological superstars are equally terrible at making predictions (there is a large literature on this, not just an opinion, but a well documented pervasive pattern ... look at all the refs in chapter 3 of https://acola.org/sites/default/files/2026-03/saf05-Technology%20and%20Australia's%20Future%20report-full-17sept%202015.pdf ). So when I read the latest predictions, I just recall the performance of prognosticators of times past.
My free advice: Say no to Neurips. And say no to stupid chatbots, and get on with life. Easy :-)
One thing we need to address is what junior people should do. They have more pressures, but they also have more agency then us older folks. If they go away, all of academia does too!
Indeed! Thinking of them was a primary motivation for expressing my opinion so bluntly. There are many things we can do: for example, stress the value of intrinsic rather than extrinsic rewards (well documented advantages are demonstrated by Deci and Ryan); declare our (old folks) disdain of judging by appearance in a venue (sign the DORA to signal your stance); encourage them to practice articulating what they are doing and why it matters (rather than relying on a venue where it shows up as a certificate of quality); encourage them to be proud of what they do, rather than what people say of it, and have the courage and gumption to take the long view; teach the value of good scientific conversations, so that when they go to conferences, they can make the most of the experience rather than trying to impress; decouple funding approval for conference attendance from having a paper there (the normal scheme implicitly says to young folks that the reason for going is only to transmit rather than to receive); explain to them that many of the "pressures" are self-inflicted (e.g. anxious and tawdry comparisons with others); telling them that the bar is low -- they can easily do better than our generation (because they have seen our collective mistakes); and leading by example in not being a complete ranker.
Over 30 years ago Hennessy and Patterson led this Natl academy report. (They included me as a pre tenure professor on the committee.) Might we revisit it and suggest a shift in tenure and promotions to encourage collective agency? “Academic Careers for Experimental Computer Scientists and Engineers (1994)”
Such a revisiting is long overdue! I was introduced to this report by Jeff Ullman (who pointed me to the subsequent short memo "Best Practices Memo
Evaluating Computer Scientists and Engineers For Promotion and Tenure"). Ironically, the National Academies report makes much of the "artefact" (as what _experimental_ computer scientists produce). One could articulate what other areas produce (not everyone makes artefacts), but of course the key point is that the end goal is not just a paper, especially one whose value is described solely in terms of its venue, which was the trigger for my initial comment.
What I find bizarre is that over the years I have tried to get academics to seriously engage in the question "what is it we are really looking for in hiring and promotion?". I even wrote a big report on this 6 years ago ... never published. The astonishing thing is the vigor with which I am always responded along the lines that "we don't need to discuss this, we all already know ...". Except they don't.
I want to work on new processes for science in the era of AI if anyone wants to join. And no I don’t think the existing ML conferences will adapt soon enough. They’ll take years of pain that I don’t want to bother trying to get involved with.
Yes. Me too! I have things I'm working on behind the scenes which are still in rough draft form. But maybe I should share those drafts here... I post a lot of my rough drafts here.
In 2014, I suggested to Philip Stark that senior people have more responsibility than junior people to change evaluation structures. He, too, said he was working on it.
I gotta say, Philip has done amazing, tangible things. He's singlehandedly responsible for the university acknowledging the sham metric that are our student teaching evaluations.
Feels like a cargo cult to me! Math is not about proving theorems or writing papers. We don’t stop running because cars are faster. Isn’t it all about the joy of figuring things out?
It seems that writing was easier to automate than reading - or, what Tao, in a strange gastrointestinal metaphor calls (proof) digestion - I wonder why that is...
Is this true? LLMs can automate both "writing" (generation?) and "reading" (verification?) under some definition right now, but for a human to understand the output of any automated writing they obviously have to read it themselves
Yes, each of these, but individual refusal remains difficult in corporations as well as universities, esp when careers still depend safety and publication, respectively. What collective actions could change the incentives without asking junior employees and researchers to sacrifice themselves first
Would something like Twitter but only for paper reviews work? Each researcher can curate their feed based on what they're experts in and also get feedback organically.
Bluesky community aside, AT Protocol seems like a pretty solid foundation for sharing and discussing papers. More promising than desperately trying to resurrect Twitter or Last.fm or w/e.
Great piece, Ben. So many opportunities to escape the vortex to the bottom and work in orthogonal directions to escape the positive feedback cycle of absurdity. Step one is to name it and remind us that new ways are possible.
While true, this ignores the consequences of doing something else. The individual consequences of choosing not to publish (or submit) can include perishing, which eliminates the author's power to continue publishing with the affiliation of the institution that executed the perishing---or maybe any future institutional affiliation.
The institutional power structures (deans, hiring committees, P&T committees) that promote/require publishing will likely advocate for scaling the productivity systems (e.g., AI reviewers) over reflection upon why we publish followed by policy changes.
If Kurt Vonnegut were still alive, he would be smiling at how his novel, "Player Piano," reflects contemporary society quite well.
Are we in any way surprised at the arms race we are seeing in so many domains? It is the result of poor incentives ("publish or perish" gone mad) coupled with the voracious engine of increasing profits.
Plus our more general metric of "more stuff = needs met". This was probably true up until about the mid twentieth century. Now we are choking on stuff. The Curse of Plenty!
There was little choice after WWII, especially in Europe. This made the apparent success of Russia's command economy seem like a good solution to meet the demands of the economy. But our freer market proved far more successful in providing variety to meet different needs and tastes, albeit still with unmet needs and some market failures. Nationalized or monopolist industries still had the flavor of command economies.
The freer market machine, driven by profits, has proven relentless in generating more. Amazon highlights the overproduction of such choice in consumer goods quite well. How much of that production goes from production to landfill without ever getting bought and used? Plastic waste dwarfs what we were facing back in the early 1970s when awareness of the problem was directing concern about the problems of dealing with it. Worse, vested interests have stymied action to decarbonize the global economy. Ironically, what should have been easy for a command economy to switch production has failed due to nations like the USSR collapsing due to complexity and turning to freer market economies. Post-Soviet Russia became a major petro-state.
I was born in the 1950s and grew up in England in a very comfortable middle-class area of London. By the latter half of the 1960s, we had a color TV, and I had the old B&W TV in my bedroom. Today, in the USA, I am quite poor, yet I have 6 flat-screens, 2 for TVs and 4 for computers, and a 7th that needs to be recycled.. Inexpensive screens with larger display sizes than the old CRTs have become almost "throw-away" products.
In 1954, the SciFi writer, Fred Pohl wrote the short story, "The Midas Plague". It was a topsy-turvy world where the poor were on a treadmill to consume production output, while the upper-status people got to consume far less. Keeping up with production is a fast-approaching problem. Recall how "keeping the economy going" was prioritized over staying safe during the COVID-19 pandemic.
Wisdom does not need as many words as knowledge, but only understanding can create wisdom, and it is, unfortunately, a derivative of learning knowledge. But learning knowledge does not necessarily lead to wisdom.
I suspect those who do not play the AI arms race might find they have an insurmountable advantage in a future iteration where those who purport to be credentialed gatekeepers in their hierarchies are but naked kings wholly reliant on the mediocre recursive knowledge amalgam spat out of AI.
The agency you're referring to is lacking in those structures because for a while now we've been replicating hierarchies where those who are promoted do not embody ethical virtues. Talk is cheap etc. I dare submit that ethically minded intelligent people progress more arduously in unethical environs and are more likely to exit fraudulent games, thinning the quality of those players who remain.
Personally I didn't pursue a PhD over a decade ago because it initially seemed pointless, and later, when I finally found a worthwhile question to consider doing a PhD on, the idea of passing between three and five years of my life in university was a turn off. I'm not suggesting I would be a who knows what, but do consider the analytical inputs about what's wrong with the gamified model of university this essay is critical about is biased by those who are within the game and doesn't ask for the opinions of those who took a look and decided to walk away.
But it's not. Your definition of lore laundering is that the results were already part of mathematical folklore, just hadn't been written up and peer-reviewed. That is not the case for at least the unit distance problem, the Sofic thing, the Jacobian counter-example.
The irony of no one understanding why Oppenheimer regretted his contributions for the exact same logic. The irony of building 'agents' to build things with abandon, robotically, with no awareness that it is easy to fall into doing so oneself.
Hallelujah!
Indeed the Neurips arms race is pathetic but very funny. And I have zero sympathy for any complainers; I just laugh. The solution is trivial, and takes two steps:
1) Just stop submitting, and disengage. That is what I did. It is very easy. I used to enjoy the conference (from about 1990-2015) because I liked talking to people there, but after 2019 I vowed never again (even now it is heading to Sydney because of work I did many years ago to get it there!). I have turned down every request to be a reviewer, area chair, senior area chair, and even senior program chair (a couple of years ago) and am delighted I did. And I don't submit there any more.
2) Stop judging people in terms of whether they are named as authors on papers published there (note my careful wording). Alas, my professorial colleagues (world-wide I mean) are the worst offenders here. This too is pathetic...
[An alternative (only slightly flippant) suggestion which I have made to a Neurips board member is to decouple the papers from the conference: by all means have the conference, for people to confer, but do not tie it to any publication; separately, have a publication venue (aka a journal). Demand submitters to pay handsomely for each submission. Use the vast profits to pay for nice food etc for the conference attendees who would then attend for the sake of conversations. The conference would no doubt shrink which would be a fine thing, and it might be fun to attend again...]
Regarding your other points, well Geoff Hinton reckoned there would be no radiologists any more (I am very glad he was wrong, having interacted with them to my considerable benefit). And other technological superstars are equally terrible at making predictions (there is a large literature on this, not just an opinion, but a well documented pervasive pattern ... look at all the refs in chapter 3 of https://acola.org/sites/default/files/2026-03/saf05-Technology%20and%20Australia's%20Future%20report-full-17sept%202015.pdf ). So when I read the latest predictions, I just recall the performance of prognosticators of times past.
My free advice: Say no to Neurips. And say no to stupid chatbots, and get on with life. Easy :-)
One thing we need to address is what junior people should do. They have more pressures, but they also have more agency then us older folks. If they go away, all of academia does too!
Indeed! Thinking of them was a primary motivation for expressing my opinion so bluntly. There are many things we can do: for example, stress the value of intrinsic rather than extrinsic rewards (well documented advantages are demonstrated by Deci and Ryan); declare our (old folks) disdain of judging by appearance in a venue (sign the DORA to signal your stance); encourage them to practice articulating what they are doing and why it matters (rather than relying on a venue where it shows up as a certificate of quality); encourage them to be proud of what they do, rather than what people say of it, and have the courage and gumption to take the long view; teach the value of good scientific conversations, so that when they go to conferences, they can make the most of the experience rather than trying to impress; decouple funding approval for conference attendance from having a paper there (the normal scheme implicitly says to young folks that the reason for going is only to transmit rather than to receive); explain to them that many of the "pressures" are self-inflicted (e.g. anxious and tawdry comparisons with others); telling them that the bar is low -- they can easily do better than our generation (because they have seen our collective mistakes); and leading by example in not being a complete ranker.
Over 30 years ago Hennessy and Patterson led this Natl academy report. (They included me as a pre tenure professor on the committee.) Might we revisit it and suggest a shift in tenure and promotions to encourage collective agency? “Academic Careers for Experimental Computer Scientists and Engineers (1994)”
Such a revisiting is long overdue! I was introduced to this report by Jeff Ullman (who pointed me to the subsequent short memo "Best Practices Memo
Evaluating Computer Scientists and Engineers For Promotion and Tenure"). Ironically, the National Academies report makes much of the "artefact" (as what _experimental_ computer scientists produce). One could articulate what other areas produce (not everyone makes artefacts), but of course the key point is that the end goal is not just a paper, especially one whose value is described solely in terms of its venue, which was the trigger for my initial comment.
What I find bizarre is that over the years I have tried to get academics to seriously engage in the question "what is it we are really looking for in hiring and promotion?". I even wrote a big report on this 6 years ago ... never published. The astonishing thing is the vigor with which I am always responded along the lines that "we don't need to discuss this, we all already know ...". Except they don't.
I want to work on new processes for science in the era of AI if anyone wants to join. And no I don’t think the existing ML conferences will adapt soon enough. They’ll take years of pain that I don’t want to bother trying to get involved with.
Yes. Me too! I have things I'm working on behind the scenes which are still in rough draft form. But maybe I should share those drafts here... I post a lot of my rough drafts here.
In 2014, I suggested to Philip Stark that senior people have more responsibility than junior people to change evaluation structures. He, too, said he was working on it.
I gotta say, Philip has done amazing, tangible things. He's singlehandedly responsible for the university acknowledging the sham metric that are our student teaching evaluations.
Feels like a cargo cult to me! Math is not about proving theorems or writing papers. We don’t stop running because cars are faster. Isn’t it all about the joy of figuring things out?
It seems that writing was easier to automate than reading - or, what Tao, in a strange gastrointestinal metaphor calls (proof) digestion - I wonder why that is...
Is this true? LLMs can automate both "writing" (generation?) and "reading" (verification?) under some definition right now, but for a human to understand the output of any automated writing they obviously have to read it themselves
Yes, each of these, but individual refusal remains difficult in corporations as well as universities, esp when careers still depend safety and publication, respectively. What collective actions could change the incentives without asking junior employees and researchers to sacrifice themselves first
1) The senior people have agency, too, even though they act like they don't.
2) if the junior people go away, the whole thing collapses.
I know systems are maddening to navigate, but I still believe collective action is possible.
Would something like Twitter but only for paper reviews work? Each researcher can curate their feed based on what they're experts in and also get feedback organically.
Sometimes I feel that's what Twitter and Bluesky are trying to be.
Though the community can be annoying, the curation tools on Bluesky for what you describe are pretty great.
Bluesky community aside, AT Protocol seems like a pretty solid foundation for sharing and discussing papers. More promising than desperately trying to resurrect Twitter or Last.fm or w/e.
Great piece, Ben. So many opportunities to escape the vortex to the bottom and work in orthogonal directions to escape the positive feedback cycle of absurdity. Step one is to name it and remind us that new ways are possible.
"We all have the power to do something else."
While true, this ignores the consequences of doing something else. The individual consequences of choosing not to publish (or submit) can include perishing, which eliminates the author's power to continue publishing with the affiliation of the institution that executed the perishing---or maybe any future institutional affiliation.
The institutional power structures (deans, hiring committees, P&T committees) that promote/require publishing will likely advocate for scaling the productivity systems (e.g., AI reviewers) over reflection upon why we publish followed by policy changes.
I hear you, but those power structures are made of people too! They can also adapt to new means of evaluation and promotion.
If Kurt Vonnegut were still alive, he would be smiling at how his novel, "Player Piano," reflects contemporary society quite well.
Are we in any way surprised at the arms race we are seeing in so many domains? It is the result of poor incentives ("publish or perish" gone mad) coupled with the voracious engine of increasing profits.
Plus our more general metric of "more stuff = needs met". This was probably true up until about the mid twentieth century. Now we are choking on stuff. The Curse of Plenty!
>The Curse of Plenty!<
Once called "Afluenza" (1990s?).
There was little choice after WWII, especially in Europe. This made the apparent success of Russia's command economy seem like a good solution to meet the demands of the economy. But our freer market proved far more successful in providing variety to meet different needs and tastes, albeit still with unmet needs and some market failures. Nationalized or monopolist industries still had the flavor of command economies.
The freer market machine, driven by profits, has proven relentless in generating more. Amazon highlights the overproduction of such choice in consumer goods quite well. How much of that production goes from production to landfill without ever getting bought and used? Plastic waste dwarfs what we were facing back in the early 1970s when awareness of the problem was directing concern about the problems of dealing with it. Worse, vested interests have stymied action to decarbonize the global economy. Ironically, what should have been easy for a command economy to switch production has failed due to nations like the USSR collapsing due to complexity and turning to freer market economies. Post-Soviet Russia became a major petro-state.
I was born in the 1950s and grew up in England in a very comfortable middle-class area of London. By the latter half of the 1960s, we had a color TV, and I had the old B&W TV in my bedroom. Today, in the USA, I am quite poor, yet I have 6 flat-screens, 2 for TVs and 4 for computers, and a 7th that needs to be recycled.. Inexpensive screens with larger display sizes than the old CRTs have become almost "throw-away" products.
In 1954, the SciFi writer, Fred Pohl wrote the short story, "The Midas Plague". It was a topsy-turvy world where the poor were on a treadmill to consume production output, while the upper-status people got to consume far less. Keeping up with production is a fast-approaching problem. Recall how "keeping the economy going" was prioritized over staying safe during the COVID-19 pandemic.
Excellent reference!
it's the subprime mortgages all over again
https://www.youtube.com/watch?v=GU4Zl2IYSrA&t=42s
Wisdom does not need as many words as knowledge, but only understanding can create wisdom, and it is, unfortunately, a derivative of learning knowledge. But learning knowledge does not necessarily lead to wisdom.
I suspect those who do not play the AI arms race might find they have an insurmountable advantage in a future iteration where those who purport to be credentialed gatekeepers in their hierarchies are but naked kings wholly reliant on the mediocre recursive knowledge amalgam spat out of AI.
The agency you're referring to is lacking in those structures because for a while now we've been replicating hierarchies where those who are promoted do not embody ethical virtues. Talk is cheap etc. I dare submit that ethically minded intelligent people progress more arduously in unethical environs and are more likely to exit fraudulent games, thinning the quality of those players who remain.
Personally I didn't pursue a PhD over a decade ago because it initially seemed pointless, and later, when I finally found a worthwhile question to consider doing a PhD on, the idea of passing between three and five years of my life in university was a turn off. I'm not suggesting I would be a who knows what, but do consider the analytical inputs about what's wrong with the gamified model of university this essay is critical about is biased by those who are within the game and doesn't ask for the opinions of those who took a look and decided to walk away.
I'm late to the party here, and the robot police story certainly does the trick, but for a very memorable real life example I love this one: https://www.foxnews.com/science/india-fights-monkeys-with-bigger-monkeys
> It’s still lore laundering
But it's not. Your definition of lore laundering is that the results were already part of mathematical folklore, just hadn't been written up and peer-reviewed. That is not the case for at least the unit distance problem, the Sofic thing, the Jacobian counter-example.
Excellent post, thank you. I've written some similar thoughts.
The irony of no one understanding why Oppenheimer regretted his contributions for the exact same logic. The irony of building 'agents' to build things with abandon, robotically, with no awareness that it is easy to fall into doing so oneself.
Excellent piece!