This is a live blog of Lecture 5 of my graduate machine learning class “Patterns, Predictions, and Actions.” A Table of Contents for this series is here. There’s no faster way to suck out the feeling of a lecture than an unintuitive optimization convergence analysis. I’m sure this will similarly kill engagement on this post. But I’ve decided to go all in for a day, and there will be equations.
no where to hide: equations were, are and always be useful. long live GD. (albeit the pseudoinverse is nicer).
Yeah, those regret bounds can be a super drag sometimes.
Typo found: it should be v_t in the last term of the first equation
thanks, fixed!