Walk the Marble Malls

Identifying the elements of a theory of engineering architecture.

John Doyle, he of robust control infamy, has been a close friend and mentor of mine for over two decades. And for the entirety of those two decades, he’s been yelling about the need for a unified theory of engineering architecture. If you know John, you’ve likely heard the same rants and seen his psychedelic PowerPoint slides with bowties, hourglasses, and bonobos. He’ll show you a picture of the internet stack and a picture of the organization of bacterial metabolism, and expect you to see that they are the same thing.

Now, I spent a good chunk of the last week trying to figure out how to pack John’s grand unified theory into a single lecture. Just like with simulation, I failed. However, I came out much more convinced that John is onto something than I went in. Let me do my best to explain why without sounding like a complex-systems crazy person.

In this class, and in control research more generally, we keep getting trapped by optimization. It’s always easiest to frame problems in terms of optimization problems, and then worry about the particulars of the solution method. However, optimal control is far too limited and brittle for any practical application. It is more often than not a simulation steering problem: we build a simulator at some abstraction level, and then shape a policy by designing an appropriate set of costs and constraints. The framework forces us to operate at a particular abstraction layer, which means we end up at a specific point on the action-impact trade-off curve. We can’t design for hidden states, and we’re sensitive to modeling errors. It forces us to model unknowns as average-case or worst-case disturbances. Optimization is specific and rigid, whereas control systems need to be diverse and flexible. What’s the right way to think about diversity and flexibility?

Fortunately, we have a lot of existence proofs to learn from when trying to answer that question. The world is run on complex, engineered feedback systems with astounding robustness, diversity, and flexibility. We transmit thousands of trillions of bits to each other every second on the internet. We maintain electricity for billions of people. We can have any product we can imagine delivered to our doorstep. We can get from our house to almost any point on earth in a couple of days. We carry around supercomputers in our pockets so we can watch vertical videos whenever we’re bored. We live in a world of engineering miracles that are more robust than any LQR system. So how in the world do they work?

John Doyle is not alone in arguing that the answer is architecture, a set of organizing principles for engineering design. Architecture is the rules and protocols for assembling components to enable diversity in system execution. You want systems that can accommodate a diversity of objectives: balancing speed, accuracy, and impact. You’d like to be able to solve a diversity of tasks. You’d like to accommodate a diversity of end users. Diversity is a particular kind of robustness. It’s robustness to intent. And you have to design for it.

Looking at the last hundred years of engineering, you definitely see repeated patterns in architectural design. First, the need for hierarchy to handle complexity isn’t surprising. Herb Simon was already on this in the sixties. But graph structure and emergulence are not enough.

Feedback is essential. As we’ve seen throughout the class, you can take two systems with dramatically different behaviors and put together a system with “best of both worlds” through feedback. A powerful amplifier and a precise attenuator combine in feedback to make a powerful, precise amplifier. A language machine that can recapitulate all software in feedback, using a simple agent harness with iterative exception handling, lets you build and manage complex software packages. Architecture lets you scale this feedback design principle.

The key to scalable architectures is protocols. Computer science — the engineering discipline of scaling logical systems — is obsessed with architectural protocols. To build a complex system like the internet or the modern computer, you build a hierarchy of abstraction boundaries. You design interfaces to talk across these boundaries with clean, well-specified protocols. The protocols let each system operate with a particular set of agreements about what the other will do. When you stack these protocols together, you get ridiculously impressive diversity. From this design strategy, you can build out the internet, the integrated circuit, the cell, and the control system. The internet serves arbitrary applications on arbitrary physical layers, all funneled through a set of contracts with IP in the middle. We can design a diversity of complex computer chips from standard cells and data flow models. In each of these cases, each layer only speaks to its neighbors in a structured hierarchy.

Finally, there is a central concept that seems to drive architectural design: constraints that deconstrain. They are restrictions on what we can do at one point of the hierarchy that end up enabling diversity at another. Constraints that deconstrain were proposed by Marc W. Kirschner and John Gerhart as a facilitator of evolution. Systems engineers like Alberto Sangiovanni-Vincentelli and Edward Lee emphasize how they provide a Devo-esque “Freedom of choice,” removing the paradox of choice and enabling more efficient design cycles. I’ll connect this to similar architectural theories of computer scientists and electrical engineers.

Engineering architecture is far too much to cram into a single lecture, but I’ll give a brief introduction to the ideas this week on the blog. I’ve come to the conclusion that there’s a whole semester’s class to be taught here. How better to end a class than by describing what the next class looks like?