Matt, a Patriots fan and friend of mine, sent me an email after the AFC Championship Game, asking me to analyze New England’s decision to go for it in the following situation:

At the Denver 16-yard line, 4th-and-1, 6:03 remaining in game, trailing by 8 points

Play: Tom Brady pass complete short left to Julian Edelman for -1 yards (tackle by Chris Harris and Aqib Talib)

In real time, I thought this was a no-brainer. You have to go for it. That’s because, in general, 4th-and-1 is a “go for it” down. But after some deep review, I’m not so sure.

Down by 8 points seems to cut in favor of needing a touchdown, but  not quite as much as you might think. That’s because announcers — and maybe coaches — seem to treat being down 8 points as not much different from being down 6 or 7 points. But being down 8 is not the same thing as being down 7, as the Patriots painfully discovered later in the game. And being down 7 is a far cry from being down 6, where a touchdown wins the game. Down by 6, I think it’s clear that you go for it. ^[1]The irony is not lost on me that some coaches would view this as a reason to kick, because you can make it a one-field goal game. But down by 8, it’s easy to play for one score, which relies on you both making a 2-point conversion (50/50) and winning in overtime (50/50). Taking the field goal cuts the lead to 5, and then you can win the game in regulation with a touchdown.

Let’s look at some numbers.

• Down by 6: According to Brian Burke’s win probability model, if the Patriots were down by 6 points, the decision is clear: New England’s win probability would be 47% with a conversion and 17% with a miss; given the 74% expected conversion rate (more on that in a minute), the Patriots would have a 39% chance of winning if they chose to go for it, and 22% if they kicked the field goal.

• Down by 7: If the Patriots were down by 7, the decision is still in favor of going for it: the expected conversion rate doesn’t change, of course, but the win probability drops from 29% with a successful conversion, and 9% with a failure, for a 24% weighted average.  By kicking a field goal, New England’s win probability would be 22% ^[2]This is outside the scope of this article, but it’s interesting that the win probabilities are considered the same here. That’s because down by 4 is actually better than being down by 3, … Continue reading.

• Down by 8: Now, the probabilities drop a bit, to 21% in the event of a conversion and 7% in the event of a miss, for a weighted-average win probability of 17%. ^[3]That seven percent might seem low given what happened, but recall that before the Patriots last offensive snap of the season — facing 4th-and-goal at the Denver 4-yard line — New … Continue reading   A field goal attempt? That has a win probability of 18%, courtesy of a 20% WP if successful and 7% with a miss.  As a result, down by 8 makes for a big difference, and here, swings the odds.

Now, a difference of one percent is well within the margin of error, so let’s look into this a little more closely.  First, the field goal odds, which assumes an 86% conversion rate on the field goal attempt.  That is way too low.  Why? During the 2015 regular season, kickers made 166 of 173 (96%) kicks from between the 13-19 yard line.  Given that Gostkowski is one of the top three field goal kickers in the league, and that Denver is the easiest place to kick, it seems like his odds of converting were no less than 96%. ^[4]I’ll note that since field goal rates have been consistently rising, if you look at numbers over the last 5 years, you need to consider those a very conservative floor. Sure, he missed an extra point earlier in the game, but that doesn’t change the analysis. Frankly, he probably hits from 33 yards away here more than 96 times out of 100.

If Gostkowski hits, he likely follows that up with a touchback. Based on Brian’s win probability calculator, Denver would have a 79% chance of winning if the Broncos had the ball at their 20, up by 5, with 5:55 remaining.  ^[5]You may notice that the “successful” conversion yields a 20% WP; that’s because the model assumes the ball at the 22; that makes sense for most games, but not one in Mile … Continue reading  Factor in the 4% chance of a miss, and we can estimate that the Patriots had a roughly 20% of winning if they elected to kick.

Now, what about the 18%? That assumes a 74% conversion rate.  On 3rd-and-1 or 4th-and-1 from the opponent’s 30 to the opponent’s 10 yard line, teams converted 71% of the time this year. ^[6]Why the 10 and the 30? The closer you get to the goal line, the harder the conversion gets, so stopping at the 10 makes sense.  And to increase my sample size, I went out to the 30, rather than … Continue reading  From 2011 to 2014, that rate was 66%.

But here’s something to consider: rushing plays are more successful than passing ones in short yardage.  Rushing plays converted 73% of the time this year, while passing plays converted 64% of the time. From 2011 to 2014, rushing plays converted 69% of the time, while passing plays converted 55% of the time.  I am not sure if the higher numbers in 2015 are representative of a smaller sample size (and therefore sample error) or a trend, given how pass-friendly the 2015 season was.

New England being New England, you have to figure the Patriots odds were better than average.  On the other hand, the Broncos pass defense was awesome in general, and really awesome yesterday in particular.  Estimating New England’s odds of converting on 4th-and-1 knowing that they chose a pass play is difficult, but I do feel pretty confident in saying it was less than 74%.  I think a 2-out-of-3 likelihood makes sense here. ^[7]Believe it or not, no team threw a pass on 3rd-and-1 or 4th-and-1 against Denver during the entire regular season.  New England, meanwhile, had just one such play.  That came against the … Continue reading

Given that New England wasn’t going to run (a decision I agree with) ^[8]Another thing to consider: running in that situation is more likely to convert at home than on the road., let’s say the Patriots had a 67% chance of converting.  Using Burke’s numbers, that means a weighted average winning percentage of 16%.  So we’re talking about a 4% swing (rather than a 1% swing using the earlier assumptions). That’s not insignificant (thought of another way, the decision dropped New England’s odds of winning by 20%), but it’s probably within the margin of error.

On the other hand… there may be some errors with Burke’s model. Let’s look at his Win Probability calculator (rather than the 4th down calculator): Denver would have a 93% chance of winning if the Broncos had the ball, up by 8, at their own 17, with about six minutes to go.  This is the New England miss option, which jives with the 7% number.

On the other hand, if you have 1st-and-10, down by 8, with 5:30 to go, at the opponent’s 10-yard line, the win probability is 29%.  At the 11, it’s 29%. At the 12, it’s 29%. At the 13, it’s 29%. At the 14, it’ 29%. But at the 15, it’s 14%!  That’s obviously a bug in the model, but it may be a big one.  If we say the Patriots has a 29% win probability with a make, and a 7% with a miss, that puts the Patriots right at 21% with a 67% conversion rate. Drop the conversion rate to 63%, and the win probability drops to 20%.  In other words, this one is really, really close.

The NYT 4th Down bot thought it was a good call, putting New England’s win probability at 23% if the Patriots went for it and 21% if they kicked the field goal.  The bot assumes a 68% conversion rate, so a 23% blended average could be the result of say, 29% with a success and 7% with a miss.

So this one is well within any margin of error.  But even that result is interesting to me, since I thought it was clear that the Patriots should have kept the offense on the field. I think the notion that New England should have gone for it because they were so close to the end zone and had struggled all day is misguided; this is the Patriots, and teams routinely have their best offensive drive of the day when trailing at the end of games.  I thought they should go for it for the normal reasons: it was only one yard, and a miss would back Denver up. But I don’t think I quite processed the gravity of how much being down by 8, rather than 7, was a factor here.  I think I’m generally much more aware of that issue than most, but even this one slipped by me because of the default idea of “go for it on 4th-and-1.”  If nothing else, this analysis made me think.