12 Comments

Here's a spicy take: Sutton and Barto's book has completely ruined a generation of researchers. Among other things, they barely mention partially observed scenarios. I was shocked to discover that some of my colleagues who work on RL don't see a problem with using uncompressed histories of observations and actions when working with POMDPs, and the mention of belief state elicits blank stares.

Expand full comment
Nov 29, 2023Liked by Ben Recht

You're telling me that if I upload GPT-N to a robot's brain and start running PPO, it won't struggle to its feet moments later? And half an hour later it won't be running at 20 miles per hour?

Expand full comment
Nov 29, 2023Liked by Ben Recht

Thanks for the post!

What resource (book, course) would you recommend to unRL one's brain?

Expand full comment

A good way to understand **cooking** is to consider some of the examples and possible applications that have guided its development.

- A red pill that, if taken, reveals unpleasant truths for you.

- A druid recipe from ancient Gaulle that lets you prepare a drink so powerful, you will have the muscles of ten for the rest of the day!

- A magic potion that will turn the user into an invincible bear, immune to the arrows of all hunters of the realm combined.

- A medicine so strong, it cures cancer.

Expand full comment