Position papers are best when they are more like op-eds (and not the crappy NY Times ones). Some journals have a "perspectives" or "commentary" category which is where those go. One could argue that Donoho's frictionless reproducibility and Breiman's two cultures are in that vein. One of my favorite short papers is Shannon's "The Bandwagon" because it's nice and salty.
The dreck survey papers I find are like a bad setting of the Credo in a mass: tendentious and just regurgitating without commentary or even synthesis. More salt needed!
(*) Vaughan Williams has a great setting of the Credo in his Mass in G -- double choir, overlapping, gets through the text in record speed.
1. They are not about "people should do this thing". They are about "people should *not* do this thing". Which makes "so just do it yourself" a difficult advice to follow.
2. They are, in some sense, reviews. They collect instances from other research that would be hard to motivate for inclusion in an experimental report, and yet do not need to have the full scope of a survey.
3. Why not a blog post? Because we felt the community needed to hear this at an academic venue. People who read blogs are not a representative sample of people who write papers and follow certain practices, and yes published papers get more respect. [Also, a cynic may say that CV goes brrr when position papers are legit at conferences so why choose a non-brrr option. The 2025 one even got nominated for best paper, by the way]
I think I had more but I forgot. Awaiting your pushback.
I believe in position papers, at least in principle. In CS, we often have no venue for thoughtful, peer-reviewed discussion of broader topics. In mainstream sciences they have Review papers, Commentaries, Perspectives, etc., and each of these serve valuable functions; some of the best and most-useful papers I've read (or that I keep going back to) fit in these categories. And writing position papers / Review papers has both clarified my thinking and allowed me to present a broader view. These paper cover topics much broader than a single experiment or algorithm, they allow discussion of entire approaches to understanding a problem and the kind of work people should be doing. (Social media and blog posts also plays a role for all of these things, I sometimes I wonder if my blog posts get more traction than the papers that I've turned them into, but I still believe in archival peer-reviewed publication.)
I agree with everything you write here, and think the arXiv is making a major mistake by requiring peer review in advance for survey papers.
I also think that having a place for editorial pieces is healthy, and other communities encourage this. Let me say something nice about statistics (gasp!), which does this regularly.
CS does have journals that feature perspectives, such as CACM. But maybe these aren't flashy enough? I guess the question is whether we need more venues like this.
Whatever the case, I don't think having a feeding frenzy, free-for-all 600-submission peer-reviewed conference track with a 6% acceptance rate is a solution.
I am a clinician with some research background and I miss good old times when it was easy to know what is good on the Internet. There was Pubmed with clear gatekeeping rules and papers that someone at least tried to check. It had a floor on quality. There was a handful of my favorite personal blogs that were transparently subjective. I followed them dutifully.
And all of this now feels like a very distant past. Pubmed quality floor is now more like a quality quagmire. You open a search and can't tell if you're looking at legitimate research, paper mill output, or pharma-funded consensus theater.
Thoughtful, accountable medical blogs are painfully rare. People either promote their papers on social media or simply quit because why bother? And there is no shortage of undereducated and overhyped writers and podcasters and widespread adoption of AI is only making things worse.
And there is MedRxiv. The worst of both worlds. It looks like PubMed (DOI, formatted, proper citations) but has the quality control of... a blog? Less? At least bloggers put their reputation behind their words. MedRxiv papers can be methodologically garbage, get debunked, updated, or withdrawn, but they've already been cited 50 times.
So we gained speed at the price of legibility. A medRxiv preprint, a predatory journal article, a pharma-backed consensus and a legitimate trial or a solid review all look the same. They all have DOI, the get cited, they claim authority. People that used to chill out and write on their personal webpage now grind for visibility because that's what counts now.
I think that there is no going back. We can't uninvent preprint servers. We can't resurrect the blog ecosystem now that professional incentives have moved elsewhere. We can't restore PubMed's quality floor once predatory journals have learned to game the indexing. We can't rebuild clear categories once everyone has realized that blurring them serves their immediate interests - pharma gets "educational" CME funding, paper mills get citations, researchers get CV lines.
When I have a client with unprocessed grief I often start by saying that it is natural to mourn not only the loss of a person but also the loss of an entire world that used to be and now there is no going back. I miss good old times when it was easy to know what is good on the Internet.
Agree with most of what you’re saying but I guess the point of position paper is to raise the high-level problem that can’t be done by individual researchers / groups alone. People write to advocate for collective effort of AI community and beyond to address it instead. Making this appeal at an academic conference may better help to attain attentions than in a personal blog page, if that person isn’t as well-known as Prof. Recht yet?
I am all in favor of calls for collective action by scientists and engineers. I'm not convinced that calls to collective action should be peer reviewed.
I agree that most position papers should either (a) be made obsolete by doing the research or (b) be blog posts like this one. But there is a role for a well-supported position paper, particularly for discussing methodological issues and presenting roadmaps. ArXiv seems to be a good place to distribute community documents such as roadmap reports and studies from the CCC/CRA/NASEM
Not bad, but where's the novelty in your position?
Seriously though, I would be interested to know better the historical practices. Haven't scientists/mathematicians always had position papers? Probably less formalised, though sometimes via invitation or a lecture that was written up, though definitely with less regularity it seems.
It would perhaps be similar to how they already had (well considered) comment threads, for example in the royal society journals that have comments sections from other academics (e.g. https://www.jstor.org/stable/2985061).
Yes. As I replied down thread to Aaron, one of the field of statistics most laudable practices is their tradition of commentary.
I think computer science backs itself into weird corners with its commitment to conference culture. I don't think you can build a thoughtful commentary culture on top of a horribly noisy conference reviewing system.
Very interesting observations about arXiv that I was unaware of. For those of us unable to have access to journal papers, arXiv can act as an approximate, pre-peer review version of a published paper. Not perfect, but legal, and acts as another open source publication platform. Authors who do get published get the best of both worlds - the recognition of a published paper in a respectable journal as well as wider readership from the arXiv and bioRxiv platforms. Permanent availability is important in the digital world.
That the platform is being gamed was inevitable, just as journal publishing was gamed in various ways, including the garbage journals that just provide "proof of publication". Given the profitability of academic publishing, wasn't this inevitable? I think Robert Maxwell made a fortune going down this route. Journals could do themselves a favor and stop demanding an absurdly high fee to send electronic copies of papers, when a newstand price of the current journal is a fraction of the price. (And just how is the guarantee of releasing papers/data after a period enforced?)
I predict that the traditional review article will be replaced by topic-focused living reviews -- web sites that organize the papers in an area and are kept up to date. LLM technology may even allow us to build semi-formalized representations of the state of knowledge in an area. ArXiv is also experimenting with applying LLM technology to streamline the moderation process. If we can automatically screen for citation spam and LLM slop, we could resume releasing suvey papers articles without requiring prior peer review.
ArXiv is also an archive. Web pages come and go; URLs are not reliable long-term links, especially ones tied to an employer. ArXiv assigns your paper a DOI and promises to keep serving your paper "forever".
again, that it should be an archive is part of the complex sociology of academia. being a doi server was a secondary function until recently. and it's downstream from a general deterioration of the public internet.
I don't care about position papers, but a total ban on reviews and surveys makes me sad. A well written review paper is so useful when you want to learn something new.
This is not a total ban. If a survey/review has been accepted after peer review, arXiv will consider it for release. I released several of these this weekend.
Even if not a total ban, this requirement of peer review for survey papers is a sad net loss for the academic community. Survey papers can be as valuable as "research papers" and are critical for reframing and advancing fields.
But are LLM-generated annotated bibliographies valuable? Many of them conform to the standards for systematic reviews, but they lack any useful synthesis. Low quality reviews turn out to be the first form of scientific research to be fully automated. When I can get an up-to-date bespoke systematic review with the press of a button, is there any benefit to publishing it as a PDF on arXiv?
Position papers are best when they are more like op-eds (and not the crappy NY Times ones). Some journals have a "perspectives" or "commentary" category which is where those go. One could argue that Donoho's frictionless reproducibility and Breiman's two cultures are in that vein. One of my favorite short papers is Shannon's "The Bandwagon" because it's nice and salty.
The dreck survey papers I find are like a bad setting of the Credo in a mass: tendentious and just regurgitating without commentary or even synthesis. More salt needed!
(*) Vaughan Williams has a great setting of the Credo in his Mass in G -- double choir, overlapping, gets through the text in record speed.
I'll put myself out there and link to my own two position papers:
https://aclanthology.org/2025.naacl-short.25/
https://aclanthology.org/2023.findings-emnlp.1012v1.pdf
You may notice a couple of things:
1. They are not about "people should do this thing". They are about "people should *not* do this thing". Which makes "so just do it yourself" a difficult advice to follow.
2. They are, in some sense, reviews. They collect instances from other research that would be hard to motivate for inclusion in an experimental report, and yet do not need to have the full scope of a survey.
3. Why not a blog post? Because we felt the community needed to hear this at an academic venue. People who read blogs are not a representative sample of people who write papers and follow certain practices, and yes published papers get more respect. [Also, a cynic may say that CV goes brrr when position papers are legit at conferences so why choose a non-brrr option. The 2025 one even got nominated for best paper, by the way]
I think I had more but I forgot. Awaiting your pushback.
Why is the 2025 paper not a research paper? What distinguishes it as a position paper?
I'd say the novel empirical content is minimal and not at the center of presentation. It wouldn't hold up a "main track" paper on its own.
I believe in position papers, at least in principle. In CS, we often have no venue for thoughtful, peer-reviewed discussion of broader topics. In mainstream sciences they have Review papers, Commentaries, Perspectives, etc., and each of these serve valuable functions; some of the best and most-useful papers I've read (or that I keep going back to) fit in these categories. And writing position papers / Review papers has both clarified my thinking and allowed me to present a broader view. These paper cover topics much broader than a single experiment or algorithm, they allow discussion of entire approaches to understanding a problem and the kind of work people should be doing. (Social media and blog posts also plays a role for all of these things, I sometimes I wonder if my blog posts get more traction than the papers that I've turned them into, but I still believe in archival peer-reviewed publication.)
I agree with everything you write here, and think the arXiv is making a major mistake by requiring peer review in advance for survey papers.
I also think that having a place for editorial pieces is healthy, and other communities encourage this. Let me say something nice about statistics (gasp!), which does this regularly.
CS does have journals that feature perspectives, such as CACM. But maybe these aren't flashy enough? I guess the question is whether we need more venues like this.
Whatever the case, I don't think having a feeding frenzy, free-for-all 600-submission peer-reviewed conference track with a 6% acceptance rate is a solution.
```
Every paper has to have a blog and a twitter thread and an arXiv post.
```
Hi Ben, how far are we from adding "Announcement of Series A funding" to the list?
I understand that this comment isn't adding anything of substance to the discussion.
But these are the kind of quips that I felt readers of Argmin would softly chuckle at (I know my friends did).
I am a clinician with some research background and I miss good old times when it was easy to know what is good on the Internet. There was Pubmed with clear gatekeeping rules and papers that someone at least tried to check. It had a floor on quality. There was a handful of my favorite personal blogs that were transparently subjective. I followed them dutifully.
And all of this now feels like a very distant past. Pubmed quality floor is now more like a quality quagmire. You open a search and can't tell if you're looking at legitimate research, paper mill output, or pharma-funded consensus theater.
Thoughtful, accountable medical blogs are painfully rare. People either promote their papers on social media or simply quit because why bother? And there is no shortage of undereducated and overhyped writers and podcasters and widespread adoption of AI is only making things worse.
And there is MedRxiv. The worst of both worlds. It looks like PubMed (DOI, formatted, proper citations) but has the quality control of... a blog? Less? At least bloggers put their reputation behind their words. MedRxiv papers can be methodologically garbage, get debunked, updated, or withdrawn, but they've already been cited 50 times.
So we gained speed at the price of legibility. A medRxiv preprint, a predatory journal article, a pharma-backed consensus and a legitimate trial or a solid review all look the same. They all have DOI, the get cited, they claim authority. People that used to chill out and write on their personal webpage now grind for visibility because that's what counts now.
I think that there is no going back. We can't uninvent preprint servers. We can't resurrect the blog ecosystem now that professional incentives have moved elsewhere. We can't restore PubMed's quality floor once predatory journals have learned to game the indexing. We can't rebuild clear categories once everyone has realized that blurring them serves their immediate interests - pharma gets "educational" CME funding, paper mills get citations, researchers get CV lines.
When I have a client with unprocessed grief I often start by saying that it is natural to mourn not only the loss of a person but also the loss of an entire world that used to be and now there is no going back. I miss good old times when it was easy to know what is good on the Internet.
Agree with most of what you’re saying but I guess the point of position paper is to raise the high-level problem that can’t be done by individual researchers / groups alone. People write to advocate for collective effort of AI community and beyond to address it instead. Making this appeal at an academic conference may better help to attain attentions than in a personal blog page, if that person isn’t as well-known as Prof. Recht yet?
Two position papers that I like:
https://arxiv.org/abs/2407.16711
https://arxiv.org/abs/2404.08592
I am all in favor of calls for collective action by scientists and engineers. I'm not convinced that calls to collective action should be peer reviewed.
I agree that most position papers should either (a) be made obsolete by doing the research or (b) be blog posts like this one. But there is a role for a well-supported position paper, particularly for discussing methodological issues and presenting roadmaps. ArXiv seems to be a good place to distribute community documents such as roadmap reports and studies from the CCC/CRA/NASEM
Not bad, but where's the novelty in your position?
Seriously though, I would be interested to know better the historical practices. Haven't scientists/mathematicians always had position papers? Probably less formalised, though sometimes via invitation or a lecture that was written up, though definitely with less regularity it seems.
It would perhaps be similar to how they already had (well considered) comment threads, for example in the royal society journals that have comments sections from other academics (e.g. https://www.jstor.org/stable/2985061).
Yes. As I replied down thread to Aaron, one of the field of statistics most laudable practices is their tradition of commentary.
I think computer science backs itself into weird corners with its commitment to conference culture. I don't think you can build a thoughtful commentary culture on top of a horribly noisy conference reviewing system.
Very interesting observations about arXiv that I was unaware of. For those of us unable to have access to journal papers, arXiv can act as an approximate, pre-peer review version of a published paper. Not perfect, but legal, and acts as another open source publication platform. Authors who do get published get the best of both worlds - the recognition of a published paper in a respectable journal as well as wider readership from the arXiv and bioRxiv platforms. Permanent availability is important in the digital world.
That the platform is being gamed was inevitable, just as journal publishing was gamed in various ways, including the garbage journals that just provide "proof of publication". Given the profitability of academic publishing, wasn't this inevitable? I think Robert Maxwell made a fortune going down this route. Journals could do themselves a favor and stop demanding an absurdly high fee to send electronic copies of papers, when a newstand price of the current journal is a fraction of the price. (And just how is the guarantee of releasing papers/data after a period enforced?)
I predict that the traditional review article will be replaced by topic-focused living reviews -- web sites that organize the papers in an area and are kept up to date. LLM technology may even allow us to build semi-formalized representations of the state of knowledge in an area. ArXiv is also experimenting with applying LLM technology to streamline the moderation process. If we can automatically screen for citation spam and LLM slop, we could resume releasing suvey papers articles without requiring prior peer review.
ArXiv is also an archive. Web pages come and go; URLs are not reliable long-term links, especially ones tied to an employer. ArXiv assigns your paper a DOI and promises to keep serving your paper "forever".
again, that it should be an archive is part of the complex sociology of academia. being a doi server was a secondary function until recently. and it's downstream from a general deterioration of the public internet.
I don't care about position papers, but a total ban on reviews and surveys makes me sad. A well written review paper is so useful when you want to learn something new.
sure, but why do they have to be posted to arxiv?
For discoverability? And other other thing you mentioned in your note - the death of personal homepages.
The death of personal webpages: another thing that makes me sad!
This is not a total ban. If a survey/review has been accepted after peer review, arXiv will consider it for release. I released several of these this weekend.
Even if not a total ban, this requirement of peer review for survey papers is a sad net loss for the academic community. Survey papers can be as valuable as "research papers" and are critical for reframing and advancing fields.
But are LLM-generated annotated bibliographies valuable? Many of them conform to the standards for systematic reviews, but they lack any useful synthesis. Low quality reviews turn out to be the first form of scientific research to be fully automated. When I can get an up-to-date bespoke systematic review with the press of a button, is there any benefit to publishing it as a PDF on arXiv?