Embracing the Random

Ed. Note: this post is again in my voice, but co-written with Kevin Jamieson. Kevin provided all of the awesome plots, and has a great tutorial for implementing the algorithm I’ll describe in this post... Continue

The News on Auto-tuning

Ed. Note: this post is in my voice, but it was co-written with Kevin Jamieson. Kevin provided the awesome plots too. It’s all the rage in machine learning these days to build complex, deep pipelines... Continue

The Role of Convergence Analysis

This year marks the retirement of Dimitri Bertsekas from MIT. Dimitri is an idol of mine, having literally written the book on every facet of optimization. His seminal works on distributed optimization, dynamic programming, and... Continue

Mechanics of Lagrangians

In my last post, I used a Lagrangian to compute derivatives of constrained optimization problems in neural nets and control. I took it for granted that the procedure was correct. But why is it correct?... Continue

Song of the week - Both Hands

David Bazan of Pedro the Lion fame just released a fantastic new record Blanco, and “Both Hands” is the leadoff track. I haven’t followed Bazan’s work since Pedro the Lion, but in the interim he’s... Continue

Mates of Costate

There have been four thousand new frameworks for deep learning thrown on the market the past year, and I bet you were wondering what you needed to jump into this hot marketplace. Essentially, there are... Continue

Song of the week - The Lurch

Ack! This regular blogging thing is hard. You let one week slip and then you’re in deficit. The semester has wrapped here, and I think I’ll have time to actually write again. If everything goes... Continue

Song of the week - B1

My new favorite music for hardcore focusing sessions is the intense minimalism of Nisennenmondai. This trio from Japan churns out epic 15 minute meditations at the boundary of post-rock and dub techno. They’ve been at... Continue

Large Margin Micro Blogging

While intricate deep models are all the rage in machine learning, my last post tried to make the case that we still need to care about model regularization. As it has been since the dawn... Continue