I am sure that what happened in those conversations is the most interesting intellectual work going on right now when it comes to AI (at least to me and growing handful of others). That said, the value of your blogging for those who weren't there is their coarseness, which leaves a lot of room to think, as I am now, about your description of what Chumley said and why examples of good bureaucracy are key. The videos won't capture in any important sense the scene, nor is it likely that any peer-reviewed papers will do so. Informal writing of the talks and about the talks is perhaps best positioned to convey something of the moment, and of the movement of the ideas in play.
In terms of good bureaucracy in the "constrains that deconstrain sense": doing PhD research. It could be better, but students are generally shielded from a lot of nonsense and given the opportunity to be creative under some constrained time to graduate.
Would also be very interested to hear about any readings folks think are missing from this list…
Our first series of topics/readings will likely be:
• Epistemic virtues: Daston, Lorraine and Galison, Peter. "Objectivity" Ch. 1 "Epistemologies of the Eye", 3 "Mechanical Objectivity", 6 "Trained Judgement" (2010)
• Intro to STS: Rouse, Joseph. “What Are Cultural Studies of Scientific Knowledge?” (1992) AND Eglash, Ron. “Multiple Objectivity: An Anti-Relativist Approach to Situated Knowledge” (2011) AND Law, John. “Material Semiotics” (2019)
• Objectivity: Douglas, Heather. "The Irreducible Complexity of Objectivity" (2004) AND Daston, Lorraine and Galison, Peter. "Objectivity" Ch. 4 "The Scientific Self", 7 "Representation to Presentation" (2010)
• Bureaucratic quantification: Porter, Theodore M. “Trust in Numbers: The Pursuit of Objectivity in Science and Public Life” (2020) AND Igo, Sarah E. "The Averaged American: Surveys, Citizens, and the Making of a Mass Public" (2008)
• Privacy policy: Mulligan, Deirdre K. and Koopman, Colin and DotyIgo, Nick. "Privacy is an Essentially Contested Concept: A Multi-Dimensional Analytic for Mapping Privacy" (2016) AND Sarah E. “The Known Citizen: A History of Privacy in Modern America” (2020)
• Standards: Star, Susan Leigh and Lapland, Martha. “Standards and Their Stories: How Quantifying, Classifying, and Formalizing Practices Shape Everyday Life” and especially the introduction “Reckoning with Standards” (2008)
• Trust & mistrust: MacKenzie, Donald. "Mechanizing Proof: Computing, Risk, and Trust" (2001) AND Carey, Matthew. “Mistrust: An Ethnographic Theory” (2017)
• Cryptographic digitization: Blanchette, Jean-François. “Burdens of Proof: Cryptographic Culture and Evidence Law in the Age of Electronic Documents” (2012)
I don't have any suggestions, but you're totally right that I find this interesting. Thank you for posting it!
(Since you've already got a couple of things from Susan Leigh Star on here, this might be redundant, but her and Geoffrey Bowker's SORTING THINGS OUT also came to mind.)
Always excited to share the STS wealth with whoever is interested! 😃
And you’re right that I should add Sorting Things Out! I think that Star’s other pieces I added are more directly relevant to TCS, but it’s such a classic that it should be on there for completeness.
On your first footnote, I don't know which physicists you've been talking to because I think a lot of physicists don't think it's all cut and dried! Not everything is effective field theories and the renormalisation group!
"Through the lens of linguistic anthropology, Chumley described how language models obscure contractual relationships underlying enterprise software. The primary interaction with language models is through the chat box. When we squeeze our demands into prompts and skill files that use the institutional language of management, we are mimicking the casual nature of Irving Goffman’s “open-state of talk” with a computer. The interaction feels personal rather than transactional. However, your interactions with all of the work software are contingent on inscrutable vendor contracts with complex webs of accountabilities, restrictions, and obligations. The employee is left with only a chat interface that has been RLHFed into a servile caricature of a 1950s secretary. This erases the heavily surveilled, legally bound, hyper monetized relationships between corporate behemoths."
This is true, but also it's not a property specific to LLMs. You could say exactly the same about the UX of the Bloomberg terminal (just think about all the data sources, each coming with its own license and ToS), or of a complex desktop photo processing application, with its plethora of licensed dependencies.
If you integrate a number of technologies developed by different vendors, you MUST obscure the relationships between them if you want your product to be interacted with as a coherent whole.
I don't have specific examples off the top of my head, but on the topic of good bureaucracy I have often thought that bureaucracy is process you *don't* like
Did anyone approach this from the legal perspective? Law also must operationalize at multiple levels. First, the conflicting intents of legislators are reduced to prose. But then police, judges, and juries further operationalize the terms. A beautiful aspect of common law is that there is room for every defendant to explain why their case is exceptional and should lead to a change in the interpretation. Is anyone studying this terminological/interpretive flexibility from a CS perspective?
In practice, coarse graining is about measurement. Coarse graining is not just about governance though. It is bureaucracy. And bureaucracy is how we organise information and reason about our values and organise change and.... etc. If we figure out our measurement process we in many ways figure out our bureaucratic processes (and the areas other than measurement besides). This level of clarity would be in itself coarse graining, constraints that deconstrain, but meta give us a very useful clarity across many domains.
I should say, not to collapse the distinction between different forms of bureacracy. Was just excited to get out the idea, meta bureaucracy of this form seems very useful and interesting
I am sure that what happened in those conversations is the most interesting intellectual work going on right now when it comes to AI (at least to me and growing handful of others). That said, the value of your blogging for those who weren't there is their coarseness, which leaves a lot of room to think, as I am now, about your description of what Chumley said and why examples of good bureaucracy are key. The videos won't capture in any important sense the scene, nor is it likely that any peer-reviewed papers will do so. Informal writing of the talks and about the talks is perhaps best positioned to convey something of the moment, and of the movement of the ideas in play.
Really enjoyed these conference blogs. I think we're getting dangerously close to a ben recht blog with the phrase "always already" in it
Perhaps! But Max Raginsky has Heidegger covered: https://realizable.substack.com/p/artificial-intelligence-interactive
In terms of good bureaucracy in the "constrains that deconstrain sense": doing PhD research. It could be better, but students are generally shielded from a lot of nonsense and given the opportunity to be creative under some constrained time to graduate.
On a somewhat related note to this post, I’m starting up a reading group on Theoretical Computer Science (TCS) + Science, Technology, & Society studies (STS): https://sites.google.com/berkeley.edu/zoebell/tcs-sts-reading-group
You/the readers of this blog may find our to-read list interesting: https://docs.google.com/document/d/18ysSsceuQYcWqsXGnOPU5OxfROnZ85mIK5HjcbQQIws/edit?usp=sharing
Would also be very interested to hear about any readings folks think are missing from this list…
Our first series of topics/readings will likely be:
• Epistemic virtues: Daston, Lorraine and Galison, Peter. "Objectivity" Ch. 1 "Epistemologies of the Eye", 3 "Mechanical Objectivity", 6 "Trained Judgement" (2010)
• Intro to STS: Rouse, Joseph. “What Are Cultural Studies of Scientific Knowledge?” (1992) AND Eglash, Ron. “Multiple Objectivity: An Anti-Relativist Approach to Situated Knowledge” (2011) AND Law, John. “Material Semiotics” (2019)
• Objectivity: Douglas, Heather. "The Irreducible Complexity of Objectivity" (2004) AND Daston, Lorraine and Galison, Peter. "Objectivity" Ch. 4 "The Scientific Self", 7 "Representation to Presentation" (2010)
• Bureaucratic quantification: Porter, Theodore M. “Trust in Numbers: The Pursuit of Objectivity in Science and Public Life” (2020) AND Igo, Sarah E. "The Averaged American: Surveys, Citizens, and the Making of a Mass Public" (2008)
• Privacy policy: Mulligan, Deirdre K. and Koopman, Colin and DotyIgo, Nick. "Privacy is an Essentially Contested Concept: A Multi-Dimensional Analytic for Mapping Privacy" (2016) AND Sarah E. “The Known Citizen: A History of Privacy in Modern America” (2020)
• Standards: Star, Susan Leigh and Lapland, Martha. “Standards and Their Stories: How Quantifying, Classifying, and Formalizing Practices Shape Everyday Life” and especially the introduction “Reckoning with Standards” (2008)
• Trust & mistrust: MacKenzie, Donald. "Mechanizing Proof: Computing, Risk, and Trust" (2001) AND Carey, Matthew. “Mistrust: An Ethnographic Theory” (2017)
• Cryptographic digitization: Blanchette, Jean-François. “Burdens of Proof: Cryptographic Culture and Evidence Law in the Age of Electronic Documents” (2012)
I don't have any suggestions, but you're totally right that I find this interesting. Thank you for posting it!
(Since you've already got a couple of things from Susan Leigh Star on here, this might be redundant, but her and Geoffrey Bowker's SORTING THINGS OUT also came to mind.)
Always excited to share the STS wealth with whoever is interested! 😃
And you’re right that I should add Sorting Things Out! I think that Star’s other pieces I added are more directly relevant to TCS, but it’s such a classic that it should be on there for completeness.
On your first footnote, I don't know which physicists you've been talking to because I think a lot of physicists don't think it's all cut and dried! Not everything is effective field theories and the renormalisation group!
"Through the lens of linguistic anthropology, Chumley described how language models obscure contractual relationships underlying enterprise software. The primary interaction with language models is through the chat box. When we squeeze our demands into prompts and skill files that use the institutional language of management, we are mimicking the casual nature of Irving Goffman’s “open-state of talk” with a computer. The interaction feels personal rather than transactional. However, your interactions with all of the work software are contingent on inscrutable vendor contracts with complex webs of accountabilities, restrictions, and obligations. The employee is left with only a chat interface that has been RLHFed into a servile caricature of a 1950s secretary. This erases the heavily surveilled, legally bound, hyper monetized relationships between corporate behemoths."
This is true, but also it's not a property specific to LLMs. You could say exactly the same about the UX of the Bloomberg terminal (just think about all the data sources, each coming with its own license and ToS), or of a complex desktop photo processing application, with its plethora of licensed dependencies.
If you integrate a number of technologies developed by different vendors, you MUST obscure the relationships between them if you want your product to be interacted with as a coherent whole.
I don't have specific examples off the top of my head, but on the topic of good bureaucracy I have often thought that bureaucracy is process you *don't* like
Did anyone approach this from the legal perspective? Law also must operationalize at multiple levels. First, the conflicting intents of legislators are reduced to prose. But then police, judges, and juries further operationalize the terms. A beautiful aspect of common law is that there is room for every defendant to explain why their case is exceptional and should lead to a change in the interpretation. Is anyone studying this terminological/interpretive flexibility from a CS perspective?
In practice, coarse graining is about measurement. Coarse graining is not just about governance though. It is bureaucracy. And bureaucracy is how we organise information and reason about our values and organise change and.... etc. If we figure out our measurement process we in many ways figure out our bureaucratic processes (and the areas other than measurement besides). This level of clarity would be in itself coarse graining, constraints that deconstrain, but meta give us a very useful clarity across many domains.
Insightful piece.
I should say, not to collapse the distinction between different forms of bureacracy. Was just excited to get out the idea, meta bureaucracy of this form seems very useful and interesting