Sunday Never Knows

When do you give up on your model and trust your gut?

I watched way too much football yesterday. In California, the NFL monopolizes Sunday TV from 6:30 AM to 8:30 PM. You can saturate your life with football. Actively or passively, I watched football while eating breakfast, while doing meal-prep for the week, while exercising in my garage gym, while eating lunch, while doing laundry, while cooking dinner. A day of chores accompanied by screens flickering between perpetual green and advertisements for schizophrenia drugs and AI.

I can now claim this subliminally tainted noise as research for my Monday decision-theory post. Yesterday’s noise did not disappoint! Because the outcomes were good, you might not have noticed, but there were two fun two-point conversion decisions that further complicated the value of analytics.

The Seattle Seahawks hosted the Tampa Bay Buccaneers for a dramatic shootout that went down to the wire. As he always does, Baker Mayfield drove the Bucs down to score a touchdown with 1:08 remaining, bringing the score to 34-35. Mayfield signalled that he wanted to go for 2. He was overruled by coach Todd Bowles, and the Bucs kicked their extra point to tie the game.

Analytics gurus Seth Walder and Aaron Schatz applauded this call. Walder said, “Too much time left. Conversion results in induced aggression (another way to put it: teams are usually sub-optimally conservative when tied -- and that goes away when down 1).” Schatz wrote, “With 1:08 left, a 1-point lead would influence the Seahawks to be more aggressive (compared to a tie score) and you still lose with a FG.”

I was confused. For unstated reasons, the numbers guys declared it was time to abandon the numbers and consider sports psychology in our decisions. They were conceding that the chances of making a high-risk play might depend on contextual features not modeled in raw statistical counts. So at that point, we’re supposed to go with our guts? To say this was breaking character was an understatement.

I’m a big fan of Schatz, who is not one of the earliest proponents of football analytics but also one of the least dogmatic. I asked him about the turn to psychology, and he replied that the gap in probability with the go-for-two-down-8 call is much larger than it was in the go-for-two-down-1 scenario.

Is it though? Let’s play by the analytics game. I’ll use Walder’s numbers. The probability of kicking the extra point is 94%. Assuming you then stop the Seahawks from scoring, the odds of winning in overtime are (apparently?) 50%. Putting this together gives you a 47% chance of winning if you kick the extra point. The analytics guys always love to tell you that the probability of making a two-point conversion is around 48%. That means that conversion gives a higher win probability! Shouldn’t you do it?

In the case of the go-for-two-down 8, Walder computed a 59% win percentage if making the two-point attempt and 47% if kicking the extra point. I mean, given the crudeness of these calculations, it’s a stretch to claim that you can tell the difference between 59% and 47% but not 48% and 47% is certainly an assertion. Walder later excortiatted the Patriots’ call to punt from the 50-yard line, showing his analytics provided a 1.8% edge for going for it over kicking. In this case, 1.8% is good enough, 1% is not. Is the threshold for psychology 1.5%? Where do you draw the line?

Though they’ll accuse me of outcome bias, I think we should look at what actually happened next. The Seahawks then received the ball with about a minute to get into field goal range. On the second play of the drive, Seahawks quarterback Sam Darnold threw into double coverage and was intercepted by linebacker Lavonte David. The pick sealed the game. The Bucs ran about 14 yards and kicked a winning field goal with no time remaining. By the way, a second field goal never entered as a potential outcome in the analytics calculation.

By contrast, in the Broncos-Eagles matchup, Sean Payton went for two, and the analytics guys were incensed. The game had been a defensive battle for three quarters, with neither team making much offensive progress. The Broncos offense had been particularly stymied by the Eagles all day, held to three points in three quarters. But on their first drive of the fourth quarter, they finally marched down the field and almost effortlessly scored a touchdown. Down 8 with 13 minutes left in the game, they kicked the extra point. Their defense forced the Eagles into a three and out, and the Broncos again drove 72 yards, now down 1 with 7:36 remaining. Broncos Coach Sean Payton knew the Eagles’ defense was gassed and wanted to go for a kill shot. He called for a two-point conversion, and the team succeeded, putting the Broncos up by 1.

This was the wrong analytics play. Schatz posted, “No, no, Sean Payton, you go for 2 after the FIRST touchdown, not the SECOND touchdown.” Walder exclaimed, “Sean Payton failed the down 8 test.”

The Eagles were now going to aggressively go for the win, right? They couldn’t passively play for overtime. Philadelphia made an excellent play on the kickoff and started on their own 43. They only needed about twenty yards to get into field goal range. They proceeded to commit two costly offensive penalties and went three and out. The Broncos got the ball back and put together a game-sealing, four-minute drive that ended in a field goal. Again, the analytics calculation doesn’t model the possibility of this extra field goal.

Analytics advocates love the two-point conversion and 4th-down attempt analyses because they are so discrete. They use these microcosms to argue for a broader numerification of football. But these plays don’t occur in isolation. We can say Payton made the wrong decision and Bowles made the right one. But what about all of the other decisions inside the game? How many wrong decisions can coaches make and still win? If coaches keep making mistakes and winning, are they swayed by outcome bias? Or is the analytics department blinded by confirmation bias?