Who is the public?

Readout of the Public Feedback for AI Workshop, part 1

Hi there, argmin readers! As the fall semester picks up, posting volume will, too. So I’m going to commit to writing short descriptive headers to help you sort through the different threads.

Today’s post is by Jessica Dai, writing up her thoughts on the microconference on “public feedback for AI and beyond” that she ran at UC Berkeley in August. -Ben

---

Much of the conversation on A(G)I’s “risks and opportunities” centers on as-yet unrealized near- (or far-) future possibilities. Yet AI is already changing the world; its societal impact is already being felt in real time by real people. There’s a yawning gap between “AI elites” — frontier labs and academics and the online AI commentariat — and everyone else, in both beliefs and priorities. I think this is bad — medium- and long-term outcomes will also depend on how society experiences the near-term.¹

My vision for dealing with this is something I’ve started calling “public feedback for AI.” The logic goes something like this:

Proposition 1. “The public” has interesting and important things to say about their experiences with AI, but are not typically listened to by decision makers.

Proposition 2. “Evaluations” — and aggregated information, more broadly — are useful, in the sense that they can influence consequential decisions.

Corollary. AI evaluations from public feedback can be a meaningful way to “do something” about the emergent misalignment between those who control AI development and literally everyone else.

I’ve spent the past few years working on this from various angles — trying to articulate why this might be a good idea, methods work for concrete audit instantiations, finding empirical evidence that user-generated content can tell us something useful about AI safety. But the more I think about these problems, the clearer the gaps in my own knowledge; there are many people who are smarter than me, who have worked on related problems for longer than I have, or who have new and different perspectives than those I’m exposed to in my local circles. And while there are a wide range of research questions that I find intellectually interesting, this clearly can’t remain a thought experiment.

Thus, microconference. (You can see a blurb, agenda, and list of attendees at publicfeedback.ai.)

I had a lot of fun at this workshop because while there were some fundamental philosophical questions threaded throughout the two days, everyone was ultimately interested in doing things beyond sending PDF preprints around to one another. And I’m really grateful for everyone’s engagement, because it helped me refine my thinking around each of the foundational ‘propositions’ underpinning the broader vision.

This write-up will be structured around the two ‘propositions’ I outlined above (rather than following the agenda directly). Additionally, I’ll stick closely to what was actually discussed at the workshop, so there is certainly lots of relevant literature that I’m not including here. Editorializing here is mostly mine, and if something is stupid or upsetting, blame me and not whichever speaker I’m referencing!

The first major question I wanted to cover is a core tension that ‘Proposition 1’, as written, glosses over: Who is the public?

While I won’t attempt to do armchair democratic theory here — though if there are any political philosophers reading this, I would love to chat — having clearer and better-motivated answers to this question seems important, because who is considered relevant also directly affects what concerns are substantively important, and also how their views can be understood.

Users?

I started us off on Monday by sharing a few plots from my paper analyzing r/ChatGPT, where we argue that anonymous Reddit data can help us understand broader societal impact; the claim is that the population of “r/ChatGPT posters” can help us simultaneously access a true “population of ChatGPT users” and something about the topics they cared about. I think this was broadly true for the time period we analyzed, but “ChatGPT users” is a very particular choice for defining a “public.”

Nevertheless, if we think AI impacts are largely realized through their users, then usage data is critical to measuring that impact. To this point, the first half of Shayne Longpre’s talk covered the newly launched AI Observatory project, a heroic effort to pull together as many real transcripts as one might reasonably find available on the internet, and to release and analyze them together as a corpus of usage data. Still, there are fundamental challenges with trying to reconstruct proprietary data from the outside; I’m cautiously optimistic about Anthropic’s (very new! Hot off the presses!) data release program.

Study participants or poll respondents?

A common adjective I’ve seen people use to describe chatlogs and other usage data is naturalistic: it reflects the real context in which users were engaging with the AI product. But this data has limitations; it is purely observational, which makes representativeness concerns all the more salient, and cannot provide context beyond the interaction itself (e.g., motivation or outcomes of usage). Instead, the typical approach to “human data collection” in academic contexts is to run more structured studies: recruitstudy participants and ask them to answer survey questions or complete tasks. While this allows researchers the ability to study more detailed questions, these studies have their own limitations; Serina Chang’s talk traced this tradeoff explicitly, with naturalism and control substituting directly for one another.

One issue that came up in discussion was about who these study participants were. Why should we expect that the data they generate is in any way “representative” of the constructs that researchers are trying to measure? A typical academic study promises a pay rate of up to $10/hour (the current standard recruitment platform is Prolific); this is below minimum wage in 30 US states. While $10 USD can be significant in a global context, we were unsure how to reason about the impact of this pay rate on the people who choose to participate.

This is a fundamental problem shared by public opinion polling (which is one reason why Ben hates it), and while it’s probably best not to read into more than one sigfig of poll results, there is reason to believe polls can be useful for coarse-grained judgments of vibe — especially when the vibes are off. Emma Pierson opened her talk with poll results showing the divergence between AI experts/ Claude users, who are generally positive about AI, and a demographically representative sample of the American population, who are generally negative. Jasmine Sun’s datacenter reporting was also motivated in part by national polls showing new datacenter construction polling on par with, or worse than, nuclear and coal. This stood in stark contrast to the SF tendency to dismiss datacenter opposition as merely a problem of misinformation.

Anyone who wants to report?

The approaches described above begin by recruiting people, then gathering their feedback within the explicit context and scope of a study (or poll). An alternative is an inversion of that paradigm: waiting for people to organically share feedback of their own volition, so that ‘the public’ is itself defined by the act of submitting a report. The second half of Shayne’s talk covered a flaw- reporting workflow that lets anyone create a report for an AI-related flaw, vulnerability, or incident, broadly construed, then sends it off to relevant parties. In terms of reporting accessibility, this system rhymes with a public incident reporting portal administered by the California Office of Emergency Services. As Deb Raji explained, this was created as part of SB53’s implementation. While this legislation was mainly directed at companies to self-report severe incidents (more on that later), there is also a provision for enabling public reporting.

Neither of these portals have shared the submissions received thus far, but they have similar design decisions that I suspect also lead to similar challenges (Algorithmic Justice League also has a harm reporting portal, and also hasn’t shared findings, as far as I can tell). Since literally anyone with the link can submit reports, these portals are likely susceptible to ‘low-quality’ submissions or spam. On the other hand, I would guess that overall submission volume is low, since awareness that these platforms exist is also low. I didn’t know about the SB53 portal until Deb’s talk, even though this is kind of my whole thing right now!

Start with an explicitly scoped community in mind, and go talk to them directly.

All three notions of ‘public’ described above involve a very convenient slippage between “public feedback” and “data collection”: they require configuring “the public” as a generic, anonymous mass from which data points can be sampled, and for which summary statistics can be computed. I don’t think this is always a bad thing, but it was striking to me that some of the most compelling case studies for effectively capturing “public feedback” started with an explicitly scoped community, and gathered information by having conversations with individuals within that community.

HCI researcher Samantha Dalal spoke about her work with the Workers’ Algorithm Observatory, which builds tools with gig workers to help identify, measure, and contest algorithmic systems. They bootstrapped worker outreach by collaborating with unions — existing legal entities with corresponding social networks. However, it was still necessary to have face-to-face conversations with workers to build trust and understand their needs.

Humphrey Obuobi shared his work with BLOOM, a nonprofit that is actively facilitating community dialogues in three counties in central Oregon. In this case, while BLOOM develops its own deliberative platform, I see the substance of their work as downstream of the specific, situated collaborations with local civic organizations which — like the unions in Samantha’s work — naturally scope participants to a more specific community.

Finally, Jasmine’s datacenter reporting was focused on understanding the popular backlash; this meant traveling to locations where backlash was visible or brewing. Of course, she talked to local elected officials, who were broadly responsible for decisions at the level of contracts and permitting, as well as activists who were leading more organized opposition. But she also spent a lot of time hanging out at county fairs and beer gardens, talking to people for whom datacenters may not have been the absolutely most salient issue in their lives, but who, nevertheless, turned out to have opinions when asked.

But wait, what about “unknown unknowns”?

I’ve often claimed that the major practical benefit of “public feedback” is the ability to support harm discovery — i.e., to surface unknown unknowns that a centralized decisionmaker (policymaker, company, etc.) might otherwise never think to look for. This seems, to some extent, at odds with the provocation above: doesn’t starting with a specific community in mind inevitably constrain the content of “public feedback” to known unknowns?

One reasonable response is that unknown unknowns exist even within well-scoped communities; prespecifying what groups of people one might be interested in doesn’t necessarily also require committing to particular outcomes or topics, especially if individuals are given the space to express their perspectives in their own voices (e.g., via semi-structured interviews instead of, or in addition to, providing yes/no answers or numerical ratings).

On the other hand, bottom-up community formation can and does already happen. Social media platforms are a natural and well-studied example: networks of like-minded people find each other through a mixture of organic encounters, algorithmic nudges, and moderation choices. For instance, after the initial release of GPT-5, a decentralized campaign to “bring back 4o” emerged on social media (mostly Reddit and Twitter); The Human Line is a research and advocacy group for survivors of AI spirals that emerged from initial encounters on Reddit.

Therefore, if working within communities (rather than a generic abstraction of “users” or “recruited participants”) can be uniquely effective, then perhaps we should try to build systems that explicitly help guide the development of communities based on common concerns. One of the projects covered in Wesley Deng’s talk was an end-user auditing tool called WeAudit, which intentionally included a deliberative and semi-social element that helped individuals contextualize their observations with other user-auditors. A similar motivation is behind an early-stage study shared by Eve Fleisig, where participants see other people’s experiences and can explicitly compare them to their own. In the universe of these projects, the community itself might emerge as an unknown unknown.

The other thing about “community” is that it’s likely of some intrinsic value to a participant — a very different source of motivation from the $10/h paid by Prolific. I’ll write more on the question of “what someone might get out of participating” in the next post.

1. If you disagree, we should talk, but the goal of this post is not to convince you otherwise.