The Robot of Pure Reason

Jaynes, subjective probability, and the thinking machine.

One of the quirkier parts of Jaynes’ development of probability is his use of a metaphorical robot as the ideal processor of information.

“How could we build a machine which would carry out useful plausible reasoning, following clearly defined principles expressing an idealized common sense?”

“In order to direct attention to constructive things and away from controversial irrelevancies, we shall invent an imaginary being. Its brain is to be designed by us, so that it reasons ​​according to certain definite rules. These rules will be deduced from simple desiderata which, it appears to us, would be desirable in human brains; i.e. we think that a rational person, on discovering that they were violating one of these desiderata, would wish to revise their thinking.”

Jaynes’ robot would be his tool for developing his formal rules of Bayesian logic. He subscribed to John von Neumann’s 1948 dictum that “If you will tell me precisely what it is that a machine cannot do, then I can always make a machine which will do just that!” von Neumann made this argument in 1948 and soon went on to try to build machines to literally control the weather. As computers became a reality in the 1950s, and his weather domination proved impossible, von Neumann’s beliefs about the possibilities softened somewhat. But Jaynes, writing in the 1990s, pushed onward.

Jaynes’ development is an unabashedly normative specification of how humans and machines should reason. He asserts that his robot has to have some qualitative congruence with the way people think. And so he builds up a framework based on logical syllogisms and “common sense” composition of such logic. He’s certainly vague about what is common sense and what is not. More often than not, common sense means whatever is required to get Cox’s theorem to apply. But Jaynes is also clear that whatever he’s doing, it cannot be too alien. He aims to design an AI to reason in a way recognizable to people while having all of the illogical parts of human reasoning ablated. Who wouldn’t want this?

“The robot always takes into account all of the evidence it has relevant to a question. It does not arbitrarily ignore some of the information, basing its conclusions only on what remains. In other words, the robot is completely nonideological.”

Ah, the completely nonideological being of pure reason. Jaynes aspires to design a cold, calculating deductive machine. A machine written as code and free from bodily desires and foolish allegiances. The robot would be a pure Cartesian mind that approached its data skeptically and extracted truth through cold calculus. Following his derivations, the pure rational agent is a Bayesian utility monster. Hold on, is this our ideal? 

First expounded upon by the Greeks, rationalism embraced a vision that the world could be described by mathematical order. The universe was a giant, intricate machine whose motions were dictated by logic and mathematics. Millennia later, Descartes would argue that these motions could be derived from pure mental reasoning.  

Post World War II, the “rational” agent has been the cold, calculating AI of science fiction. That agent is nonideological but acutely selfish. It maximizes utility, rigorously reasoning about uncertainty and preference structures. It is mercurial and unpredictable, as Game Theory proves that rational utility maximizers necessarily act randomly.

I’m fascinated by this phase change. 300 years after Descartes, we somehow decided that the pure reasoning being would be selfish and free from the tethers of ethics. Philosophers, mathematicians, engineers, and economists decided that “rationality,” which used to be about mechanism, determinism, and unity, was instead about chance, deceit, and selfishness. “Bayesianism” is the mathematization of this paradigm shift. 

Let me follow up on this tomorrow, describing the ideas that gave rise to rational thinkers being cunning gamblers. The history is fascinating and can’t be decoupled from Cold War arrogance.