Even for Football Perspective, this is a very math-heavy post. I’ve explained all the dirty work and fine details behind this system, but if you want to skip to the results section, I’ll understand. Heck, it might even make more sense to start there and then work your way back to the top.

Background

In 2012, Neil Paine wrote a fascinating article on championship leverage in the NBA, building on Tom Tango’s work on the same topic in Major League Baseball. Championship Leverage was borne out of the desire to quantify the relative importance of any particular playoff game. Truth be told, this philosophy has more practical application in sports where each playoff round consists of a series of games. But Neil applied this system to the NFL playoffs and crunched all the data for every playoff game since 1965. Then he was kind enough to send it my way, and I thought this data would make for a good post.

The best way to explain Championship Leverage is through an example. For purposes of this exercise, we assume that every game is a 50/50 proposition. At the start of the playoffs, the four teams playing on Wild Card weekend all have a 1-in-16 chance of winning the Super Bowl (assuming a 50% chance of winning each of four games). This means after the regular season ended, the Colts had a 6.25% chance of winning the Super Bowl. After beating Kansas City, Indianapolis’ win probability doubled to 12.5%. Win or lose, the Colts’ Super Bowl probability was going to move by 6.25%, a number known as the Expected Delta.

New England, by virtue of a first round bye, began the playoffs with a 12.5% chance of winning the Lombardi. With a win over Indianapolis, the Patriots’ probability of winning the Super Bowl jumped 12.5% to 25%; had New England lost, the odds would have moved from 12.5% to zero. Therefore, the Expected Delta in a Division round game is 12.5%.

By beating Indianapolis, New England set up a crucial showdown with Denver. A win would again double the Patriots’ Super Bowl odds, this time from 25% to 50%, while a loss would drop it to zero. In the conference championship round, the Expected Delta is always 25%.

Denver also started the playoffs with a 12.5% chance of winning the Super Bowl, since the Broncos don’t get any extra credit for rostering Peyton Manning. Those odds jumped to 25% after beating San Diego and up to 50% after beating the Patriots. The Super Bowl, of course, has an Expected Delta of 50%. After the game, Denver’s odds of being Super Bowl champs will have moved 50% to either 100% or 0%.

There are 11 games (or 22 if you look at each game as one game for each team) in every NFL postseason, at least since 1990. The final game has an Expected Delta of 50%; the previous two each have an Expected Delta of 25%; the four before that each have an Expected Delta of 12.5%; and the first four each have an Expected Delta of 6.25%. This means, on average, each playoff game in the NFL has an average Expected Delta of 15.91%.

This means that the Super Bowl — with an Expected Delta of .50 — is 3.14 times as “important” as the average playoff game. That importance is what we call the Leverage Index, and at least since 1990, each Super Bowl has had a Leverage Index of 3.14. Peyton Manning’s performance against Seattle comes with a Leverage four times as great as Manning’s game against San Diego, because the stakes are four times as high. I’m reticent to ever type the word clutch, but using this method, we can at least quantify the stakes for each game.

Calculating a Quarterback’s Leverage-Adjusted Postseason Value

We can use this metric to grade each individual postseason by a quarterback after accounting for Leverage. For each game in the Super Bowl era, Neil calculated the Adjusted Net Yards per Attempt average allowed by each defense during the regular season, and then gave each quarterback credit (or blame) for his ANY/A average relative to that particular defense. By using this method, we have both an era and SOS adjustment all in one. Then, each quarterback’s production is adjusted for leverage. Let’s work through an example.

In the first round of the 2008 playoffs, Kurt Warner faced the Atlanta Falcons, who allowed an average of 5.96 ANY/A during the regular season. Warner produced a 19/32-271-2-1 (0-0) sack line, meaning he completed 19 of 32 passes for 271 yards, threw two touchdowns and one interception, and was not sacked. That gave Warner an ANY/A average of 8.31, 2.35 ANY/A better than we would expect (based on the Atlanta defense). Of course, this was not a particularly significant game: the leverage was only 0.39.

The following week, Warner went 21/32-220-2-1 (1-5) against the Panthers. Carolina allowed 5.43 ANY/A during the regular season, and Warner averaged 6.36 ANY/A in this game, giving him 0.93 ANY/A over expectation. The leverage was 0.79.

In the conference championship game, Warner faced an Eagles defense that allowed only 4.57 ANY/A during the regular season, but he went 21/28-279-4-0 (2-12); that works out to an 11.57 ANY/A average, giving him an incredible 7.00 ANY/A better than average in a game with a leverage of 1.57.

Then, in the Super Bowl, Warner was again outstanding. The Steelers defense allowed only 3.17 ANY/A to opposing quarterbacks during the regular season but Warner (31-43-377-3 (2-3), an 8.64 ANY/A average) was 5.47 ANY/A better than that against Pittsburgh. The Super Bowl, of course, has a leverage of 3.14.

All told, Warner had 140 attempts (including sacks). On average, each pass attempt came in a game with a Leverage of 1.62 (Warner’s four games, by definition, had Leverages of 0.39, 0.79, 1.57, and 3.14; that would give him a simple average of 1.47, but since he threw 12 more passes in the Super Bowl than in any other game, his weighted average leverage is a bit higher.) If you multiply his attempts by his ANY/A over expectation by the leverage for each of the four games, and then divided that total by 1.62 (the average leverage for each attempt), you get 714, the amount of (leverage-adjusted) adjusted net yards over expectation Warner produced. That’s the most by any quarterback in a single post-season.

Here’s another way to think about it. Based on the defenses Warner faced, he would have been expected to produce a weighted average ANY/A (weighted for both SOS and Leverage) of 3.87; in reality, he produced a weighted ANY/A (again, weighted for Leverage) of 8.97. Therefore, Warner exceeded expectations by 5.10 ANY/A. Since he had 140 dropbacks, that gives him 714 adjusted net yards of value over average. The table below shows the top 100 postseasons by a quarterback (looking at only passing numbers) using this method:

The Best Single-Postseason Passing Performances From 1965 to 2012

Seeing Warner’s 2008 as the top postseason performance isn’t too surprising. What about Jim Plunkett’s 1980 season? Over four games, he had a 96.2 passer rating and averaged 9.1 yards per attempt while throwing for 7 touchdowns. Remember, this was 1980, not 2013: that year, Brian Sipe won the passer rating crown with a 91.4 rating. More importantly, Plunkett gets extra credit for being at his best in the biggest games. In the AFC Championship Game, he was 14/18 for 261 yards and 2 touchdowns. In the Super Bowl, he threw for 261 yards and 3 touchdowns on 21 passes. If the goal is to reward quarterbacks for being at their best in the most critical games, then Plunkett’s position at number two is legitimate.

Joe Montana’s magical 1989 is number three, and the only reason he’s behind Plunkett is because he had fewer attempts. Montana’s leverage-adjusted 7.76 ANY/A over expectation is the best performance in a postseason since 1965. Jake Delhomme’s presence at #4 might be surprising, but that’s a function of his production in a high-leverage situation against a great opponent. Forget the name, and consider that a quarterback ten years ago faced the #1 pass defense in the league (by ANY/A) and threw for 323 yards and 3 touchdowns with no interceptions on 33 pass attempts in the Super Bowl. Had Carolina won that game (and, perhaps, had Delhomme not subsequently imploded five years later), we might remember his 2003 playoffs the way we think of Joe Flacco’s 2012 postseason.

Working Through Another Example

Speaking of Flacco, I was a bit surprised he wasn’t in the top 3, but that’s essentially a function of the built-in era adjustment. Let’s go through this method using Flacco’s 2012 season, but using another method to get to the same result.

Flacco’s first game was against the Colts, who allowed 6.54 ANY/A during the regular season. If we multiply that number by the leverage (0.39) and his number of dropbacks (24), we get 61.66. In Denver, Flacco faced a team that allowed 4.87 ANY/A during the regular season. Multiply that number by the leverage (0.79) and dropbacks (35), and you get 133.85. In the AFC Championship Game, Flacco had 38 dropbacks against a team that allowed 6.31 ANY/A during the regular season; multiply those two numbers by the leverage (1.57) and you get 376.83. Finally, he faced the 49ers in the Super Bowl, and San Francisco allowed 4.88 ANY/A during the regular season. Flacco had 35 dropbacks in a game with massive leverage (3.14); the product of those three numbers is 536.91.

If you add up those four numbers – 61.66, 133.85, 376.83, and 536.91 — you get 1,109.26. Next, we divide that number by Flacco’s total number of dropbacks (132) to get 8.40. The last step is to divide that number by the average leverage of each pass Flacco attempted. Do the math, and that number is 1.57. Once you divide 8.40 by 1.57, you get the Expected (leverage-adjusted) ANY/A for Flacco during the post-season, which is 5.37. That’s a really high number, at least historically speaking, which simply reflects the fact that putting up good passing numbers was a lot easier in 2012 than it was in 1972. Flacco is essentially getting dinged for an era adjustment, but that’s appropriate.

How did Flacco actually do? He had an ANY/A of 12.88 against Indianapolis, 10.97 against Denver, 7.76 in New England, and 9.54 in the Super Bowl. Adjust for his number of attempts and leverage, and Flacco produced a leverage-adjusted ANY/A of 9.37. That’s also a hair lower than his non-leverage adjusted ANY/A of 10.02, which makes sense: his best game was against Indianapolis, the lowest-stakes game in which he played. Flacco was still great, but the leverage and the era combine to put him at “only” 4.00 ANY/A better than expectation.

You might be surprised that Tom Brady doesn’t fare all that well in this metric. In fact, he only has three top-100 seasons, and none in the top 40. Well, Brady’s never had one dominant postseason. His best year was in 2004, but even then, his numbers there look merely “very good” as opposed to historically great. Is his playoff reputation overrated? Perhaps. In Part II, I’ll show you the career playoff ratings.

I won’t pretend that the math involved isn’t overly complicated. But hey, Neil and I already did all the work for you. So what do you think of the list?