Limits to Simulated Growth

Jay Forrester, Systems Dynamics, and the allure of the simulated society.

Hi there, argmin readers! Today’s post is a live blog of Class 17 of my graduate seminar “Forecasting: A Critical Retrospective.” In this class, we’re going to discuss simulation as a means of prediction. The syllabus and list of past posts are here.

---

In a surprisingly dark survey paper from 1971, “Counterintuitive Behavior of Social Systems,” electrical engineer turned social scientist Jay Forrester made a case for informing public policy with computer simulations. Forrester argued that policymakers made all decisions with models, but his computer models were more explicit and concrete than people’s mental models. Computer models laid all of the assumptions on the table and let stakeholders adjust the parameters until they conformed to reality. They also let people create a variety of prefactual worlds, demonstrating plausibility and illustrating the consequences of various decision-making paths. Most importantly for Forrester, the computer models captured the surprising and unintuitive nature of feedback effects, showing pathways that mental models might not have predicted.

Forrester’s survey gives three examples, each with more ambitious aims than the last. He describes his time as a management consultant and how computer simulations revealed that decisions meant to fix things led to worse outcomes. Good intentions could lead to a “downward spiral,” and this would often convince managers to double down on their initial wrong decision. Forrester argues that a simulation would have predicted these troubles and informed management of a better path.

He then turns these ideas toward urban planning. He claims that his simulations can explain and fix the “depressed nature of central cities.” Indeed, his simulations argue that four well-intentioned liberal programs are causing urban blight, not fixing it.

“One program was creation of jobs by busing the unemployed to suburban jobs or through governmental jobs as employer of last resort. Second was a training program to increase skills of the lowest-income group. Third was financial aid to depressed cities from federal subsidies. Fourth was construction of low-cost housing.”

Forrester goes on to explain the vicious cycle here, where a surplus of cheap housing draws in more new residents than the job market can support. This leads to overcrowding and a less attractive urban environment. This, in turn, repels new affluent residents. Forrester says his simulations “both confirm and explain much of what has been happening over the last several decades in cities.”

A dispassionate observer might say that his simulation matched his politics. But Forrester saw modeling power, not confirmation bias. He went on to extend his models to a global scale.

Forrester goes on to describe work he had done that predicted the end of the world by 2071. Current trends could not support population growth, and pollution would run amok, and we’d run out of natural resources. The only solution was to impose policies that swiftly dampened population growth and austerely conserved resources. Applying Forrester’s simulator, a team led by Dennis Meadows assembled the dire predictions into the 1972 book The Limits to Growth, which sold 12 million copies worldwide.

Forrester’s report is filled with feedback diagrams and simulation traces. The ability to predict so many different possibilities under different scenarios is intoxicating. Historian Kevin Baker argues that this sort of interactive experience was core to the allure of the simulation model. By adding knobs and switches, people could play out different worlds, making the simulation feel more trustworthy and real. This would eventually lead to enthralling games with bizarre hidden politics, notably the world bestseller SimCity. As Baker puts it, “The capacity of simulation models to captivate and enthrall is taken for granted as a natural property of simulation. But these abilities were often deliberately engineered into software for historically-specific purposes.” Forrester loved his simulators because they quantified and externalized mental models. He underemphasized how they created new mental models, letting policymakers play video games in imaginary worlds that let them feel as if they were creating the future.

One of the funnier things about statistical prediction is that it forecasts a static world. Your best guess is to mimic past rates. By contrast, simulation can imagine different worlds, mechanistically mapping trajectories into the future based on our assumptions about the present.

However, simulations are not reality, and more sophistication is not good in and of itself. In fact, if small changes lead to unexpected consequences, a single poor modeling choice can too. The doom of The Limits to Growth did not play out. Nor did Forrester’s libertarian predictions about urban blight.

In the last class session, we grappled with two competing sales pitches for “silicon sampling.” Most people felt comfortable using silicon samples as part of some prediction loop. For example, you could create virtual training data for some recommendation system by asking LLMs to simulate answers to questions based on demographics. You might have it label examples while “pretending to be a plumber” and then “pretending to be a schoolteacher,” using those labels as synthetic training data. You can then validate the resulting system against standard KPIs with A/B tests or benchmarks. If the prediction system passes evaluation metrics, it doesn’t matter if it accurately models people.

On the other hand, it’s easy to believe that because LLMs perform well on these prediction tasks, they create realistic simulations of people. You can interactively inspect the answers LLMs give and gain trust in their fidelity. And then you can convince yourself that you can use these simulations to understand human behavior without talking to people. This is the same seductive simulation loop that Baker calls out. We’re more skeptical of simulation now than in 1970, likely because we’re so familiar with video games and how they aren’t real. But the prospect of simulating society is still enticing, and new well-funded research programs and startups are playing the same old tricks of immersive gameplay.