As with every new argmin series, I love this series on homeostasis!
One rather tangential comment about this point:
"However, the mathematics of adaptation and homeostasis looks quantitative but aims to capture qualitative phenomena. I feel like I should be able to describe qualitative phenomena without mathematical notation "
I think this is a very important point, and I came to think about it quite a bit in the context of behavioral / psychological sciences. Why do we need or want quantitative models of learning and behavior? Things like rescorla-wagner, Q-learning, etc. Or all sort of Bayesian accounts of perceptual inference. Or Drift-Diffusion models, and so on and so on.
It is often times that, at the end of the day, the "intuition", or qualitative, explanation for how the model works could have been (and/or actually had been) easily described by psychologists decades before the mathematical models. On the other hands, all the quantitative details are clearly (again most of the times) only serve as rough approximations / modeling choices. So, why do we bother with all the math if what we get in the end is a qualitative explanation that doesn't need the math at all?
My answer (I should say, this was one of the topics which I had endless discussions on with my phd supervisor) is that the mathematical/quantitative model building offers a mechanistic "concretization". This helps in identifying components and implications of the qualitative solution. We cannot use vague words that are too much open for interpretation if the model has to be translated into code. The 'mechanistic' aspect is also important because with some luck, it might help us from a neuroscience perspective of looking for a mechanistic/implementation (a similar argument will hold from an 'engineering' perspective I guess, like in your example: if you actually want to program/build systems that can do this form of regulation, you will have very hard time doing this without the math, _even though_ the principle itself stays qualitative).
ok that's probably long enough for a comment that is a digression from the main point! Looking forward for the next posts.
I really like your perspective here, and it's been something I've been thinking a lot about myself. It's clear that mathematical abstractions can be helpful to codify design principles. But it's also clear that they are often overkill and often too metaphorical to be useful. Unfortunately, it's almost a theorem that there can't be a theorem telling you when math becomes metaphor.
As with every new argmin series, I love this series on homeostasis!
One rather tangential comment about this point:
"However, the mathematics of adaptation and homeostasis looks quantitative but aims to capture qualitative phenomena. I feel like I should be able to describe qualitative phenomena without mathematical notation "
I think this is a very important point, and I came to think about it quite a bit in the context of behavioral / psychological sciences. Why do we need or want quantitative models of learning and behavior? Things like rescorla-wagner, Q-learning, etc. Or all sort of Bayesian accounts of perceptual inference. Or Drift-Diffusion models, and so on and so on.
It is often times that, at the end of the day, the "intuition", or qualitative, explanation for how the model works could have been (and/or actually had been) easily described by psychologists decades before the mathematical models. On the other hands, all the quantitative details are clearly (again most of the times) only serve as rough approximations / modeling choices. So, why do we bother with all the math if what we get in the end is a qualitative explanation that doesn't need the math at all?
My answer (I should say, this was one of the topics which I had endless discussions on with my phd supervisor) is that the mathematical/quantitative model building offers a mechanistic "concretization". This helps in identifying components and implications of the qualitative solution. We cannot use vague words that are too much open for interpretation if the model has to be translated into code. The 'mechanistic' aspect is also important because with some luck, it might help us from a neuroscience perspective of looking for a mechanistic/implementation (a similar argument will hold from an 'engineering' perspective I guess, like in your example: if you actually want to program/build systems that can do this form of regulation, you will have very hard time doing this without the math, _even though_ the principle itself stays qualitative).
ok that's probably long enough for a comment that is a digression from the main point! Looking forward for the next posts.
I really like your perspective here, and it's been something I've been thinking a lot about myself. It's clear that mathematical abstractions can be helpful to codify design principles. But it's also clear that they are often overkill and often too metaphorical to be useful. Unfortunately, it's almost a theorem that there can't be a theorem telling you when math becomes metaphor.
This "gradient descent as integrator" view reminds me of a cute derivation I came across recently. Solving Ax=b using least-squares gradient descent can be written as a harmonic sum which converges to b\A . So different variations of gd are different integrators converge to b\A at different rates https://www.dropbox.com/scl/fi/u1c7dy6uh858x6tsc93hc/Screenshot-2024-12-22-at-11.16.24-AM.png?rlkey=kkt5w9eh4042gi3wf1r1tg4az&e=1&dl=0