I’ve been mulling over two quotes about the future and trying to figure out how one was 100% accurate and the other was 100% wrong. One thing that’s nice about the old-school blog format is you can post block quotes as text rather than images so people might actually read them. I’m curious to get some insights into why there is such a discrepancy in accuracy between these two predictions.
The first quote is from Gordon Moore’s article in Electronics magazine, “Cramming more components onto integrated circuits.” This is the paper where he poses Moore’s Law. But the paper gets so much more right about the integrated circuit industry and its impact. Here are the opening three paragraphs:
The future of integrated electronics is the future of electronics itself. The advantages of integration will bring about a proliferation of electronics, pushing this science into many new areas.
Integrated circuits will lead to such wonders as home computers—or at least terminals connected to a central computer—automatic controls for automobiles, and personal portable communications equipment. The electronic wristwatch needs only a display to be feasible today.
But the biggest potential lies in the production of large systems. In telephone communications, integrated circuits in digital filters will separate channels on multiplex equipment. Integrated circuits will also switch telephone circuits and perform data processing.
Accurate on all accounts! The paper goes on to discuss detailed steps to scale up the manufacturing of integrated circuits and even lists some potential hurdles that might lie along the way.
For comparison, consider a quote from Claude Shannon, one of my academic heroes, discussing the future of artificial intelligence. I’ve been watching a bunch of old documentaries about MIT, and they’re mostly embarrassing with their overblown predictions and naive technoutopianism. Among the more cringy is the 1961 CBS series Tomorrow. CBS produced the series to commemorate the 100th anniversary of the founding of MIT. You can find the episodes on YouTube. Shannon is featured at the end of the episode “The Thinking Machine” predicting the near-term future of artificial intelligence:
In discussing the problem of simulating the human brain on a computing machine, we must carefully distinguish between the accomplishments of the past and what we hope to do in the future. Certainly the accomplishments for he past have been most impressive. We have machines that will translate to some extent from one language to another. Machines that will prove mathematical theorems. Machines that will play chess or checkers sometimes even better than the men who designed them. These however are in the line of special-purpose computers aimed at particular specific problems.
What we would like in the future is a more general computing system capable of learning by experience and forming inductive and deductive thoughts. This would probably consist of three main parts. In the first place that would be sense organs, akin to the human eye or ear, whereby the machine can take cognizance of events in its environment. In the second place there would be a large, general-purpose, flexible computer programmed to learn from experience, to form concepts, and capable of doing logic. In the third place, there will be output devices. Devices in the nature of the human hand capable of allowing a machine to make use of the thoughts that has had of the cognitive processes in order to actually affect the environment. Work is going on in all of these fronts simultaneously and rapid progress is being made.
I confidently expect that within 10 or 15 years we will find emerging from the laboratories something not too far from the robotic science fiction fame. In any case, whatever the result, this is certainly one of the most challenging and exciting areas of modern scientific work.
Welp, 10-15 years was way off. Even 60 years later, we’re not close at all to the robots of science fiction. What’s also amazing is how he is referring to ideas where they currently had capabilities in 1960: translation, gameplay, theorem proving. Not only did prototypes of these technologies already exist, we are not doing anything that dramatically different from the initial prototypes. We’ve made a lot of progress, but the main engineering approaches are the same. Even our Large Language Models are building upon an idea of Shannon’s from 1950.
So why was Moore so right and Shannon so wrong? Tell me in the comments!
[A long time reader of your argmin blog here, and also a twitter follower from behind the 🔒. So I'm taking the opportunity to interact here, because I thought about sharing the same thing replying to your Twitter thread on that subject].
A very interesting read.
Here's an attempt to "save" some of Shannon's prediction abilities by using a different source [while keeping in mind the caveat that if you make enough predictions, maybe something eventually hits...].
Earlier this year I've had a look at the following book, which similar to that CBS show was also part of the 100 anniversary celebration of MIT:
https://mitpress.mit.edu/9780262570022/computers-and-the-world-of-the-future/
It is basically the transcription of the lectures + discussion of a conference they had where they invited all the "who and who" to talk about their wild predictions for the future of computing. Obviously some predictions are completely absurd but some are quite interesting (and particularly how they imagined future tech would look like given their current tech; there's an interesting chapter about online libraries for example).
Other than the lectures themselves and the Q&A, there were also longer invited "comments" for each lecture. One of them is a comment by Claude Shannon on the lecture of John Pierce (said Pierce was some Bell Labs bigshot who later on would be responsible for the gov invited report on Machine Translation that is said to be putting that field into its own "Winter"). In that comment, Shannon raises some pretty good predictions, *and* a much more nuanced view on the relation between biological brains and computers. I'm copying from the Twitter thread in which I shared some of this (which is the reason for all the [..] skipped parts):
[QUOTE]
[...] I believe that, in fact, there is very little similarity between the methods of operation of the computers and the brain [..]. [First] the wiring circuitry of the computers are extremely precise and methodical. A single incorrect connection will generally cause errors and malfunction. The connections in the brain appear, at least locally, to be rather random, and even large number of malfunctioning parts do not cause complete breakdown of the system. [Second] computers work on a generally serial basis, doing one small operation at a time. The nervous system, otoh, appears to be more of a parallel-type computer with a large fraction of the neurons active at any given time. [Third] it may be pointed out that most computers are either digital or analog. The nervous system seems to have a complex mixture of both representations of data.
[..]
These and other arguments suggest that efficient machines for such problems as pattern recognition, language translation, and so on, may require different type of computer than any we have today. It is my feeling that this computer will be so organized that single components do not carry out simple, easily described functions. One cannot say that this transistor is used for this purpose, but rather that this group of components together performs such and such function. If this is true, the design of such a computer may lead us into something very difficult for humans to invent [..].
Most machines are invented by breaking the overall problem down into a series of simple and perhaps previously solved problems [..] In a machine of the type I'm suggesting, it would be impractical to describe the purpose of any single component. I know of very few devices in existence which exhibit this property of diffusion of function over many components. [...] Can we design [..] a computer whose natural operation is in terms of patterns, concepts, and vague similarities rather than the sequential operations on 10 digit numbers?
[END QUOTE]
It's a hard question, I wish I knew ... But I think the key is to make precise and restricted predictions. I suppose Moore was very knowledgeable about large electronics. He saw they could get smaller. He saw that people liked to have products (in 65 they were big products, like cars, TVs, washing machines). They could get smaller, with more circuits. Whatever computers they had at the lab Moore was leading, he saw they could eventually fit in an object the size of a TV. There was a clear(ish) path from how things were like in 65 to how things could be if the prediction turned out right. In Shannon’s prediction, I guess the path was less clear. In some sense, eyes existed at the time: video and photo cameras. These pictures or videos could be featurized. But how exactly would they use these very large matrices of inputs with “flexible computer programmed to learn from experience”. Shannon was very knowledgable too, of course. But it just sounds less precise, all-encompassing ... nebulous.
All this is easy to say in retrospect. Perhaps I’m all wrong. I have no idea! From another angle, I think the first problem is mostly about engineering, while the second is about “magic”. Magic in the sense that we don’t exactly know what we want, but we know it will be awesome. It’s going to be awe-dropping. I’ve seen this pattern a few times in the social sciences with the use of machine learning. Not exactly clear what we want, but it will be a “game changer”, will make a “big splash”, will lead to a “breakthrough in the field”.
p.s. I’ve started to follow you recently on Twitter (and read a few of your blogs). Great online presence. Really appreciate the writings. You and a few others seem just more clear eyed than most on what people (even big names) do with statistics these days.