This course sounds great, and I'm also very much looking forward to the syllabus and other info! I'm also a huge fan of L4DC; thanks for your work on that.
One thing I'm maybe not understanding is the distinction you're making between optimal control and feedback (possibly just to do with definitions). E.g. LQR is an optimal controller for an LTI sytem, and outputs a linear feedback controller. If you consider a PID controller as feedback, its behavior is tied to a local Lypaunov / LaSalle function, just as an optimal controller's behavior is tied to a value function. MPC will locally estimate (and re-estimate online) the value function, and would also be considered to be a "feedback" controller. Trajectory optimization (I agree) has no feedback, but typically in robotics this trajectory will still be stabilized using a feedback controller generated using MPC or LQR. Similarly, for a learned policy generating actions from observations, domain randomization during training necessitates some amount stochastic robustness in the feedback control, which appears to be key for all the robotics behaviors being developed using RL in simulation these days.
I hope you will also be willing to share your lecture notes later during the semester. I’m from systems biology and there’s a small group of people that I talked to interested in applying control theory to cellular systems, and since biology is so complicated generally we are finding success with ML approaches as well. Maybe there will be some cross pollination of ideas based on what you would like to teach!
Thank you, I am so looking forward to this! Some ten years or so again I went into machine learning after I thought "hey, this looks like a bit of applied control theory" (I have a PhD in control, so I am biased) and since then I could not shake the feeling that most of it is some glorified optimisation reinventing the wheel and giving fancy new names to long-established ideas.
My hunch is, that there is a common, less de-trivialised lingo which leads to a more intuitive understanding and de-mystifies a lot of the mystical technical terms. E.g. once we start formulating the RL-policies as (slightly) different types of feedback controllers trying to drive some target value, we might have an easier time thinking and talking about them and that could lead to a deeper understanding with a wider audience.
Cybernetics strikes again! The principles of feedback control really seem to be fundamental in some way to complex systems, the way conservation laws underpin mechanics and thermodynamics.
I look forward to your syllabus and future discussions.
This course sounds great, and I'm also very much looking forward to the syllabus and other info! I'm also a huge fan of L4DC; thanks for your work on that.
One thing I'm maybe not understanding is the distinction you're making between optimal control and feedback (possibly just to do with definitions). E.g. LQR is an optimal controller for an LTI sytem, and outputs a linear feedback controller. If you consider a PID controller as feedback, its behavior is tied to a local Lypaunov / LaSalle function, just as an optimal controller's behavior is tied to a value function. MPC will locally estimate (and re-estimate online) the value function, and would also be considered to be a "feedback" controller. Trajectory optimization (I agree) has no feedback, but typically in robotics this trajectory will still be stabilized using a feedback controller generated using MPC or LQR. Similarly, for a learned policy generating actions from observations, domain randomization during training necessitates some amount stochastic robustness in the feedback control, which appears to be key for all the robotics behaviors being developed using RL in simulation these days.
Is there any way I can access the notes or recording or tune in live?
Doing my PhD in AI and human control processes and your article captures a lot of the concepts I'm looking into!
It’s a seminar, so there won’t be a video, but I’ll liveblog and post all the readings.
Thanks a lot!!
I hope you will also be willing to share your lecture notes later during the semester. I’m from systems biology and there’s a small group of people that I talked to interested in applying control theory to cellular systems, and since biology is so complicated generally we are finding success with ML approaches as well. Maybe there will be some cross pollination of ideas based on what you would like to teach!
Absolutely. I’ll post all of the materials here, starting next week.
Sounds awesome!!!
Thank you, I am so looking forward to this! Some ten years or so again I went into machine learning after I thought "hey, this looks like a bit of applied control theory" (I have a PhD in control, so I am biased) and since then I could not shake the feeling that most of it is some glorified optimisation reinventing the wheel and giving fancy new names to long-established ideas.
My hunch is, that there is a common, less de-trivialised lingo which leads to a more intuitive understanding and de-mystifies a lot of the mystical technical terms. E.g. once we start formulating the RL-policies as (slightly) different types of feedback controllers trying to drive some target value, we might have an easier time thinking and talking about them and that could lead to a deeper understanding with a wider audience.
I have been pursuing a possibly related line of inquiry —
Just by way of priming the pump, here is the initial vision statement —
Prospects for Inquiry Driven Systems
• https://oeis.org/wiki/User:Jon_Awbrey/Prospects_for_Inquiry_Driven_Systems
Cybernetics strikes again! The principles of feedback control really seem to be fundamental in some way to complex systems, the way conservation laws underpin mechanics and thermodynamics.
I look forward to your syllabus and future discussions.