Tenure was intended to protect faculty seeking to opt in to risky reforms and initiatives, but ironically it also protects those who wish to opt out. I've seen more of the latter than the former. Maybe Ben can buck the trends.
I love and strongly support workshops/"microconferences". They are one of the best things about academia. One of the main benefits I see is that the room size gets small/focused enough, so people stop talking past each other with their contributions (as they often do at large conferences) and instead talk to/with each other. And the handful of times I've been a discussant (or had a discussant of my own work) at workshops, I feel it's also really raised the bar in terms of people talking to/with each other.
I'm a little unsure about the idea of publishing proceedings for workshops/microconferences. Most of all, I think the non-proceedings nature of workshops is key to drawing in a diverse set of people. For all the flaws of existing publication models, if a workshop is a publication venue then all of the sudden you have the economists, political scientists, legal scholars, etc. no longer willing/interested in joining/presenting, because they need tokens in their own community currency, even if some subset of fields buy in to the new currency model being proposed here. The whole token/currency thing is of course a major issue with academia, but I just worry that adding publications would hurt one of the few Good Things academia has going, and it would be really hard to reform currency norms in All The Fields. So maybe proceedings would be a good optional thing? Maybe something like the "invitation to a special issue" mechanism for an (overlay?) journal?
Worth discussion: ACM EC has a forward-to-journal mechanism, which gets CS, OR, and Econ folks together for a coherent conference but lets them still go harvest their respective tokens. The EC reviews get passed to the journal. And EC also organizes a "Highlights beyond EC" session, which lets people request to present work recently/already published "elsewhere" but relevant to the community. Both of these mechanisms help keep the "conference" and "publication" goals separate, which I think has been good.
Finally, seconding Deborah's comment, I think one of the best contributions tenured folks can make is organize such events and to feature/promote the work of junior/untenured scholars at such events. Ramesh Johari gave me roughly this advice when I got tenure, which I found wise and generous. I've tried to model good behavior on this front.
Schloss Dagstuhl (https://www.dagstuhl.de/en) explicitly couples microconference infrastructure and support with an open access publishing framework. Within my community, a great deal of work gets done at Dagstuhl.
Very interesting. Have you come across Adam Mastroianni's concept of science as a strong-link problem? https://www.experimental-history.com/p/science-is-a-strong-link-problem?hide_intro_popup=true. (Apologies if you've blogged about it before and I missed it. But you really can't complain, you write a lot of blog posts ... almost suspiciously many for a guy concerned about over-production ... )
The idea is that, from the point of view of scientific progress, we actually don't care about how many bad papers are written. Bad papers are usually forgotten pretty quickly, and good papers have a way of being found. Citations are a crude metric, but how many papers with over 1,000 citations are actively bad? Not many, I'd wager.
An ecosystem which encourages lots of mediocre publications might be bad for other reasons (eg, maybe it's hard to evaluate phd candidates), but it's not immediately obvious that it's a problem in the long run.
Agreed. And while the infrastructure (and funding) provided by places like Banff, IPAM, Simons, .. is certainly nice, it is surprisingly easy to just do it yourself (with most attendees funding themselves). It also is much quicker: maybe 2-3 months lead time, compared to waiting ~18 months from a proposal to an event with one of those organizations. (This is a modest example of a microconference series, 2 ad hoc meeting and 2 via Banff - https://willperkins.org/frontiersworkshops.html ).
The Dagstuhl and Oberwolfach type conferences in TCS used to also have “bump sessions” (maybe they were called rump!) where people proposed open problems, noodled over difficulties, etc. Sometimes these micro workshops even “published” open problems from these bump sessions. They were quite useful and engaging!
Another model I’d like to advocate for is to have members of a PC present the papers they selected (see my rant about paper reviewing and big conferences) and then an audience discussion. Like what the statisticians do but in person and with a publication resulting for the authors.
Micro-conferences and workshops are the most productive, but any conference where all the participants can fit in one lecture room can work. After that, as you say, the most useful function is as a job fair.
Conversation can be scaled, I've found the "unconference" format - bunch of sticky notes with opinions and discussion groups - to work well towards this end. (Still don't like the the word "conference" to be included tho as it still represents the old format/way of sharing ideas.)
Great emphasis on evaluation as sustained conversation rather than mass triage, just wondering how to keep these gatherings genuinely porous to people outside the organizers’ existing networks.
There's a fantastic substack I read (https://caseorganic.substack.com/p/inside-the-very-human-origin-of-the) on the origin of the term "Artificial Intelligence" and why no one liked a grumpy old Norbert Weiner. I think something that's ruined conferences, from what elders in the field tell me, is the unassailable might of money at these places. I would assume that for such microconferences to work, there would be a need to create some distance similar to McCarthy's rebranding of "cybernetics".
Microconferences are great! The conference -> journal path is also a nice way to support community building. (The bonus is that conference attendees make natural reviewers: they've heard the talk, maybe). ESIF AI+ML is somewhat the Econometric Society's version of EC. Which is nice to have a venue for exchange of ideas for communities that publish in completely opposite ways but study the same or similar problems. OR's Applied Probability Society is also experimenting with forward-to-journal options from ML workshops.
There are a number of practical problems still. One is that microconferences scale very poorly for in-person get-togethers; any person has a fixed travel budget but each work/project might belong to multiple distinct microconferences. Another is budget. A "microconference" that we all like in causal inference is ACIC. It used to rotate around universities and expanded in the past few years to a national stage. But word on the street is that the convening society might be a bit broke. Staying indie for microconferences is cheaper, but labor intensive. The final one is about expanding the club, rather than contracting it. I've been a (grateful!) attendee of "family reunion" workshops and feel a bit out of place to not be a dynastic member of the tree.
Anyway, microconferences are great. Many conversations I have at microconferences begin with "this is so much nicer than a megaconference". I hope more people organize them, and also to anyone reading the comments section please invite me to speak at your next microconference : )
It is possible you are suggesting something else but honestly your post to me reeks of elitism. Almost sounds like the current folks in power in the USA who want to go back to a 1950s white men only research world. I may be misinterpreting things but who decides who is allowed or not. It's an awful level of gatekeeping and I don't see this as a way to fix the existing problems. Just feels like a head in the sand "we are the smart ones leading the field and we know whom to invite" idea. And all this talk about some people with the most ideas is so western-centric notion of having a hero or two to worship rather than a better eastern centric notion of appreciating the collective efforts and allowing everyone a chance to build (though I'll give you that all societies need leaders and most humans prefer to be followers). Indeed, those who are very smart or have a lot of resources (think transformers) are the ones who will hit any field running but research also happens broadly and collectively. The NSF model of giving everyone a chance (at least until recently) and being welcoming is so much better than the DARPA model of pre-deciding that a Berkeley faculty is better than someone else not at Berkeley.
The intersection of microconferences and slop is interesting. On one hand, increasing the amount of time real people are talking to each other is great for filtering out slop from useful thinking.
However, I am concerned that microconference submissions could turn into slop magnets. While double-blind is gold standard for review, single-blind might actually work better here. If you know there are a handful of people, potentially impactful to your career, who will definitely read what you write, that could make you less likely to submit slop.
If you can work that out, this could be really cool! More feedback iterations on research and opportunities for collaborative ideation could lead to fascinating results.
And tenured folks need to take the lead to de-risk for pre-tenured.
This. It's a great idea, but will go further if some people who are of a certain standing take it up.
Tenure was intended to protect faculty seeking to opt in to risky reforms and initiatives, but ironically it also protects those who wish to opt out. I've seen more of the latter than the former. Maybe Ben can buck the trends.
I love and strongly support workshops/"microconferences". They are one of the best things about academia. One of the main benefits I see is that the room size gets small/focused enough, so people stop talking past each other with their contributions (as they often do at large conferences) and instead talk to/with each other. And the handful of times I've been a discussant (or had a discussant of my own work) at workshops, I feel it's also really raised the bar in terms of people talking to/with each other.
I'm a little unsure about the idea of publishing proceedings for workshops/microconferences. Most of all, I think the non-proceedings nature of workshops is key to drawing in a diverse set of people. For all the flaws of existing publication models, if a workshop is a publication venue then all of the sudden you have the economists, political scientists, legal scholars, etc. no longer willing/interested in joining/presenting, because they need tokens in their own community currency, even if some subset of fields buy in to the new currency model being proposed here. The whole token/currency thing is of course a major issue with academia, but I just worry that adding publications would hurt one of the few Good Things academia has going, and it would be really hard to reform currency norms in All The Fields. So maybe proceedings would be a good optional thing? Maybe something like the "invitation to a special issue" mechanism for an (overlay?) journal?
Worth discussion: ACM EC has a forward-to-journal mechanism, which gets CS, OR, and Econ folks together for a coherent conference but lets them still go harvest their respective tokens. The EC reviews get passed to the journal. And EC also organizes a "Highlights beyond EC" session, which lets people request to present work recently/already published "elsewhere" but relevant to the community. Both of these mechanisms help keep the "conference" and "publication" goals separate, which I think has been good.
Finally, seconding Deborah's comment, I think one of the best contributions tenured folks can make is organize such events and to feature/promote the work of junior/untenured scholars at such events. Ramesh Johari gave me roughly this advice when I got tenure, which I found wise and generous. I've tried to model good behavior on this front.
Schloss Dagstuhl (https://www.dagstuhl.de/en) explicitly couples microconference infrastructure and support with an open access publishing framework. Within my community, a great deal of work gets done at Dagstuhl.
Very interesting. Have you come across Adam Mastroianni's concept of science as a strong-link problem? https://www.experimental-history.com/p/science-is-a-strong-link-problem?hide_intro_popup=true. (Apologies if you've blogged about it before and I missed it. But you really can't complain, you write a lot of blog posts ... almost suspiciously many for a guy concerned about over-production ... )
The idea is that, from the point of view of scientific progress, we actually don't care about how many bad papers are written. Bad papers are usually forgotten pretty quickly, and good papers have a way of being found. Citations are a crude metric, but how many papers with over 1,000 citations are actively bad? Not many, I'd wager.
An ecosystem which encourages lots of mediocre publications might be bad for other reasons (eg, maybe it's hard to evaluate phd candidates), but it's not immediately obvious that it's a problem in the long run.
Agreed. And while the infrastructure (and funding) provided by places like Banff, IPAM, Simons, .. is certainly nice, it is surprisingly easy to just do it yourself (with most attendees funding themselves). It also is much quicker: maybe 2-3 months lead time, compared to waiting ~18 months from a proposal to an event with one of those organizations. (This is a modest example of a microconference series, 2 ad hoc meeting and 2 via Banff - https://willperkins.org/frontiersworkshops.html ).
The Dagstuhl and Oberwolfach type conferences in TCS used to also have “bump sessions” (maybe they were called rump!) where people proposed open problems, noodled over difficulties, etc. Sometimes these micro workshops even “published” open problems from these bump sessions. They were quite useful and engaging!
Another model I’d like to advocate for is to have members of a PC present the papers they selected (see my rant about paper reviewing and big conferences) and then an audience discussion. Like what the statisticians do but in person and with a publication resulting for the authors.
I always thought it as "rump" :). but bump just sounds more polite.
Micro-conferences and workshops are the most productive, but any conference where all the participants can fit in one lecture room can work. After that, as you say, the most useful function is as a job fair.
Conversation can be scaled, I've found the "unconference" format - bunch of sticky notes with opinions and discussion groups - to work well towards this end. (Still don't like the the word "conference" to be included tho as it still represents the old format/way of sharing ideas.)
Great emphasis on evaluation as sustained conversation rather than mass triage, just wondering how to keep these gatherings genuinely porous to people outside the organizers’ existing networks.
There's a fantastic substack I read (https://caseorganic.substack.com/p/inside-the-very-human-origin-of-the) on the origin of the term "Artificial Intelligence" and why no one liked a grumpy old Norbert Weiner. I think something that's ruined conferences, from what elders in the field tell me, is the unassailable might of money at these places. I would assume that for such microconferences to work, there would be a need to create some distance similar to McCarthy's rebranding of "cybernetics".
Microconferences are great! The conference -> journal path is also a nice way to support community building. (The bonus is that conference attendees make natural reviewers: they've heard the talk, maybe). ESIF AI+ML is somewhat the Econometric Society's version of EC. Which is nice to have a venue for exchange of ideas for communities that publish in completely opposite ways but study the same or similar problems. OR's Applied Probability Society is also experimenting with forward-to-journal options from ML workshops.
There are a number of practical problems still. One is that microconferences scale very poorly for in-person get-togethers; any person has a fixed travel budget but each work/project might belong to multiple distinct microconferences. Another is budget. A "microconference" that we all like in causal inference is ACIC. It used to rotate around universities and expanded in the past few years to a national stage. But word on the street is that the convening society might be a bit broke. Staying indie for microconferences is cheaper, but labor intensive. The final one is about expanding the club, rather than contracting it. I've been a (grateful!) attendee of "family reunion" workshops and feel a bit out of place to not be a dynastic member of the tree.
Anyway, microconferences are great. Many conversations I have at microconferences begin with "this is so much nicer than a megaconference". I hope more people organize them, and also to anyone reading the comments section please invite me to speak at your next microconference : )
It is possible you are suggesting something else but honestly your post to me reeks of elitism. Almost sounds like the current folks in power in the USA who want to go back to a 1950s white men only research world. I may be misinterpreting things but who decides who is allowed or not. It's an awful level of gatekeeping and I don't see this as a way to fix the existing problems. Just feels like a head in the sand "we are the smart ones leading the field and we know whom to invite" idea. And all this talk about some people with the most ideas is so western-centric notion of having a hero or two to worship rather than a better eastern centric notion of appreciating the collective efforts and allowing everyone a chance to build (though I'll give you that all societies need leaders and most humans prefer to be followers). Indeed, those who are very smart or have a lot of resources (think transformers) are the ones who will hit any field running but research also happens broadly and collectively. The NSF model of giving everyone a chance (at least until recently) and being welcoming is so much better than the DARPA model of pre-deciding that a Berkeley faculty is better than someone else not at Berkeley.
what
I have myself paid dearly for the "collection of papers" model of graduate education
The intersection of microconferences and slop is interesting. On one hand, increasing the amount of time real people are talking to each other is great for filtering out slop from useful thinking.
However, I am concerned that microconference submissions could turn into slop magnets. While double-blind is gold standard for review, single-blind might actually work better here. If you know there are a handful of people, potentially impactful to your career, who will definitely read what you write, that could make you less likely to submit slop.
If you can work that out, this could be really cool! More feedback iterations on research and opportunities for collaborative ideation could lead to fascinating results.