Microconferences

A proposal to create alternative systems for generating, evaluating, and sharing knowledge

I ended my last post with a call for action. I stated, without evidence, that if we’re unhappy with the status quo, we can choose to do something else. I’m hoping this semester I can instead lead by doing (sorry that’s corny) and be a weird freak out in public on this blog. Let me start today by sharing something I’ve been working on: a plan to create alternative systems for generating, evaluating, and sharing knowledge.

While academia has exploded in size in the post-war period, the ideal size of a conference has remained constant. Small workshops with one or two dozen participants have consistently launched big ideas, forged multidisciplinary connections, and set new standards across fields. Look at the history of artificial intelligence (a marketing term I continue to loathe with every ounce of my being but one that is unfortunately useful given its current cultural dominance). The field was conceived at a small conference held at Dartmouth. Societal implications and multidisciplinary connections were hashed out at the Macy Conferences. The ICML conference began as a small workshop of AI researchers at CMU, a splinter group that reclaimed and rebranded the term machine learning. NeurIPS began as small workshops organized at Caltech, called “Hopfest,” where John Hopfield brought together neuroscientists, physicists, and computer engineers to rethink information processing. These workshops each had 10 to 30 people, were cross-disciplinary, aimed at bridging disciplinary boundaries, and focused on brewing creative, new ideas.

Machine learning conferences like ICML and NeurIPS now draw tens of thousands of participants. Somewhere along the way, computer scientists convinced themselves that conference proceedings should be valued as highly as journal publications in other disciplines. A combination of industrial interest and metric-maximizing left AI conferences as an undignified mix of job fairs and “vetting” of “publications” for CV padding. The evaluation is indistinguishable from coin tossing, and no particular result ever stands out. It’s more likely that a mishap at an industry party will get attention than a paper will change the field. This situation is unfortunate and untenable.

Of course, the small workshops—mocking scale, let’s call them microconferences—live on. A professor with a little initiative can put together an ad hoc meeting. There are non-profit institutions dedicated to hosting these sorts of events, including mathematics research institutes like the Institute for Pure and Applied Mathematics at UCLA, computing research institutes like the Simons Institute at UC Berkeley, and institutes that grew to define an entire field like the Santa Fe Institute. However, microconferences are considered luxury goods, something everyone wants to do all the time, but is only allowed to do after their paper writing, teaching, reviewing, and committee work. For some reason, in the modern calculus of the curriculum vitae, a refereed conference paper is more valuable than inspiring a new research direction at a small meeting. This is absurd.

The funniest part about the quantification trap is how we forget that the quantities were ad hoc and artificial in the first place. What makes a Science paper count as prestigious is nothing more than the salivation of a bunch of annoying academics who want Science papers for themselves. If enough of us just want “well-curated microconference proceedings,” then we can value these just as highly.

How could we go about increasing the value and worth of these meetings? I don’t propose constructing an elaborate counting mechanism to have microconferences register as yet another notch on a publication list. And I know we can’t break down our current system of bureaucratic accounting overnight. Deans will still rely on stodgy metrics, scholars will need to produce legible artifacts to appease said deans, and historical institutional prestige will still signal more than it should. Nonetheless, I’d like to offer an initial path for fields to return to the days when small-scale but intense evaluations were valued more than large-scale but cursory selection processes.

Valuing the outputs of small workshops wouldn’t be a departure from our academic traditions. Most of our publication system traces back to reading papers before the Royal Society of London. A paper was only considered published after it had been read aloud and debated in person. Oftentimes, people read other people’s papers and debated these. Lots of our modern systems try to carry on this legacy of debate. In statistics, a main contributed paper is accompanied by “discussions” by other authors and a rejoinder by the contributed paper’s authors. In economics, the working paper serves as a way to gather feedback before official publication, which can take years. Many fields use “special issues” to compile talks from their conferences into proceedings. These are all great ideas! We should draw from the best of these traditions.

One of the hardest things to avoid is making hard-and-fast rules. Mission statements are always forgotten in giant bureaucratic systems, leaving behind a fossilized set of rules and conventions that people assume have been there since time immemorial. These systems then invite the same old gaming and goal displacement. Fully aware that this is the case, let me first state some values I’ve compiled in discussions with friends and colleagues. I’ll offer a few guidelines after.

1. What: Microconferences are where the best ideas are debated. Some things do not and cannot scale.

2. Who: Microconferences should bring together people who didn’t know each other before. You can invite friends, but you can’t only invite your friends. Sometimes you need to get different people in the room and see what happens.

3. When: The turnaround time from conception to microconference should be as short as possible.

4. Organization: While a parent institution for these microconferences can streamline logistics for microconference organizers, it should not require a permanent home or dedicated staff. Its leadership should be minimal and rotate over time. It should be easy to organize a microconference.

5. Output: Not all outputs of a microconference need to be a publication, but sharing findings and insights with others is part of what academics do. While not everything needs to be written down, and ephemeral conversation is important, participants should err on the side of more documentation.

For this final bullet, I envision a documentation structure similar to what organizations like the aforementioned IPAM and Simons Institute currently maintain: a webpage. The Proceedings of Machine Learning Research (PMLR) provides an ideal model. Inspired by the minimal elegance of the Journal for Machine Learning Research, Neil Lawrence cooked up a GitHub repo to compile and curate conference proceedings. It does exactly what is needed and nothing more. The main page will cleanly list past events, and you’ll be able to search by title or participant. Individual microconference webpages could include a list of participants, a photo of the gathering, and a documentary record that links to slides, videos, or papers. What is archived and how it is archived would be up to the organizers.

In the style of arXiv overlay journals, these pages could signify the “blessing” of a particular version of arXiv papers by the participants, demarcating that a paper has been legitimately peer-reviewed. People might dismiss such peer review as cronyism, but if you see the names of participants, you can judge for yourself what credence you want to give to the review.

I am not claiming that microconferences solve all problems. One glaring example is the widespread belief that computer science graduate students need papers. What is a PhD if not a bunch of papers stapled together? Yikes. The obsession with using conferences as a proxy for grad student education is a mistake we are paying for dearly. This proposal does not fix the problem. That said, having microconferences all over the place would be nothing but a boon for graduate students. There will always be one near their university that they can apply to attend. And nothing in this model prevents them from organizing their own microconferences.

Whatever the case, the current situation is untenable. Conferences are impossibly overrun, and their publications fully depreciated. A framework for microconferences offers something else. It is not a replacement. It is a new path to credentialing and cultivating expertise. Will it also end up a bureaucratic mess in a few years? Probably so! But I hope you’ll join me in trying it out before declaring defeat.