Catching Signals That Sound in the Dark
This is the eleventh part of “An Outsider’s Tour of Reinforcement Learning.” Part 12 is here. Part 10 is here. Part 1 is here. The essence of reinforcement learning is using past data to enhance... Continue
The Best Things in Life Are Model Free
This is the tenth part of “An Outsider’s Tour of Reinforcement Learning.” Part 11 is here. Part 9 is here. Part 1 is here. Though I’ve spent the last few posts casting shade at model-free... Continue
The Ethics of Reward Shaping
I read three great articles over the weekend by Renee DiResta, Chris Wiggins, and Janelle Shane that touched on a topic that’s been troubling me: In machine learning, we take our cost functions for granted,... Continue
Benchmarking Machine Learning with Performance Profiles
A common sticking point in contemporary reinforcement learning is how to evaluate performance on benchmarks. For a general purpose method, we’d like to demonstrate aptitude on a wide selection of test problems with minimal special... Continue
Clues for Which I Search and Choose
This is the ninth part of “An Outsider’s Tour of Reinforcement Learning.” Part 10 is here. Part 8 is here. Part 1 is here. Before we leave these model-free chronicles behind, let me turn to... Continue
Updates on Policy Gradients
This is the eighth part of “An Outsider’s Tour of Reinforcement Learning.” Part 9 is here. Part 7 is here. Part 1 is here. I’ve been swamped with a bit of a travel binge and... Continue
A Model, You Know What I Mean?
This is the seventh part of “An Outsider’s Tour of Reinforcement Learning.” Part 8 is here. Part 6 is here. Part 1 is here. The role of models in reinforcement learning remains hotly debated. Model-free... Continue
The Policy of Truth
This is the sixth part of “An Outsider’s Tour of Reinforcement Learning.” Part 7 is here. Part 5 is here. Part 1 is here. Our first generic candidate for solving reinforcement learning is Policy Gradient.... Continue
A Game of Chance to You to Him Is One of Real Skill
This is the fifth part of “An Outsider’s Tour of Reinforcement Learning.” Part 6 is here. Part 4 is here. Part 1 is here. The first two parts of this series highlighted two parallel aspirations... Continue
The Linear Quadratic Regulator
This is the fourth part of “An Outsider’s Tour of Reinforcement Learning.” Part 5 is here. Part 3 is here. Part 1 is here. What would be a dead simple baseline for understanding optimal control... Continue