There’s more to data than distributions.

This is first guest post by Deb. More to come! In “The clinician and dataset shift in artificial intelligence,” published in the New England Journal of Medicine, a set of physician-scientists describe how a popular... Continue

Machine Learning has a validity problem.

One of the central tenets of machine learning warns the more times you run experiments with the same test set, the more you overfit to that test set. This conventional wisdom is mostly wrong and... Continue

Let us never speak of these values again.

A recent Twitter quiz asked “what is a powerful concept from your field that, if more people understood it, their lives would be better?” Unambiguously, the answer from my field is statistical significance. Significance testing... Continue

What were the effects of the Bangladesh mask intervention?

There’s been a bit of a social-media back-and-forth between us and Jason Abaluck about the design and statistical significance of the Bangladesh Mask RCT. To focus and hone the discussion on some crucial details, we... Continue

The cult of statistical significance and the Bangladesh Mask RCT.

In the last post, I argued that the effect size in the Bangladesh Mask RCT was too small to inform policy making. I deliberately avoided diving into statistical significance as arguments about p-values quickly devolve... Continue

Revisiting the Bangladesh Mask RCT.

In an earlier post, I raised a few issues with a large-scale RCT run in Bangladesh aimed at estimating the effectiveness of masks on reducing the spread of the coronavirus. In particular, I was a... Continue

The Perceptron as a prototype for machine learning theory.

Just as many of the algorithms and community practices of machine learning were invented in the late 1950s and early 1960s, the foundations of machine learning theory were also established during this time. Many of... Continue

The Saga of Highleyman's Data.

The first machine learning benchmark dates back to the late 1950s. Few used it and even fewer still remembered it by the time benchmarks became widely used in machine learning in the late 1980s. In... Continue

Machine learning is not nonparametric statistics.

Many times in my career, I’ve been told by respected statisticians that machine learning is nothing more than nonparametric statistics. The longer I work in this field, the more I think this view is both... Continue

Experiments as randomized algorithms

While every statistics course leads with how correlation does not imply causation, the methodological jump from observation to causal inference is small. Using the same algorithmic summarization and statistical analysis tools that we use to... Continue