Eight years ago — almost to the day — our old PFR colleague Doug Drinen wrote a Sabernomics post about “The Manning Index”, a metric designed to roughly gauge the clutchness (or chokeitude) of a given quarterback by looking at how he did relative to expectations (he revived this concept in version two, six years ago). In a nutshell, Doug used the location of the game and the win differential of the two teams involved to establish an expected winning percentage for each quarterback in a given matchup. He then added those up across all of a quarterback’s playoff starts, and compared to the number of wins he actually had. Therefore, quarterbacks who frequently exceeded expectations in playoff games could be considered “clutch” while those who often fell short (like the Index’s namesake, Peyton Manning) might just be inveterate chokers.

Doug ran that study in the midst of the 2004-05 playoffs, so it shouldn’t be surprising that Tom Brady (who was at the time 8-0 as a playoff starter and would run it to 10-0 before ever suffering a loss) came out on top, winning 3.5 more games than you’d expect from the particulars of the games he started. Fast-forward eight years, though, and you get this list of quarterbacks who debuted after 1977:

Flacco, whom Chase wrote about yesterday, predictably finishes near the top of the list. As usual, the table is fully sortable (the table includes 130 quarterbacks, and you can change the number of quarterbacks displayed or use the search function to find other quarterbacks), and you can click on the “Index” column to bring the “biggest chokers” to the top. A few other notes: I generated the expected wins for this Manning Index not from Doug’s formula, but from the pregame win probabilities we can derive from the Vegas lines. Also, since I don’t have a list of all QB starts since 1978, I’m considering a player to be the QB of record for his team if he led them in pass attempts in a game (with yards as the tiebreaker). ^[1][Chase Note: Scanning the old comments to Doug’s post, Jason Lisk chimed in with this still-applicable comment: Steve Young is unfairly penalized for throwing more than 10 pass attempts in the … Continue reading

By now, you’ve probably noticed that the Manning Index has reached its full potential, with a Manning brother at the top and at the bottom! But I also want to focus on the strange career arc of Brady, and how much his Manning Index has changed since the end of the 2004 season.

Here are Brady’s career playoff stats as the Patriots’ primary QB in a game, year-by-year:

Year	Age	Gm	tmPts	opPts	Cmp	Att	Yds	TD	Int	Sk	SkYd	Rush	rYds	rTD	Won	Lost	Exp W	Index
2001	24	2	36	30	48	79	457	1	1	4	26	6	19	1	2	0	0.7	+1.3
2003	26	3	73	57	75	126	792	5	2	0	0	12	18	0	3	0	2.0	+1.0
2004	27	3	85	51	55	81	587	5	0	7	57	7	3	1	3	0	1.8	+1.2
2005	28	2	41	30	35	63	542	4	2	4	12	3	8	0	1	1	1.1	-0.1
2006	29	3	95	75	70	119	724	5	4	4	22	8	18	0	2	1	1.5	+0.5
2007	30	3	66	49	77	109	737	6	3	8	52	4	-1	0	2	1	2.5	-0.5
2009	32	1	14	33	23	42	154	2	3	3	22	0	0	0	0	1	0.6	-0.6
2010	33	1	21	28	29	45	299	2	1	5	40	2	2	0	0	1	0.8	-0.8
2011	34	3	85	51	75	111	878	8	4	3	15	9	10	1	2	1	2.1	-0.1
2012	35	2	54	56	54	94	664	4	2	1	9	3	4	0	1	1	1.5	-0.5

The big thing that jumps out is the difference between his Manning Index early (through age 27 it’s +3.5; through age 29 it’s +3.8) and late in his career (starting in 2007, it’s -2.5). As an aside, two years ago, Jason Lisk wrote about Brady’s career in reverse (and Bill Barnwell did the same earlier this week), showing how easy it is to flip the narrative about him depending on which direction you view his playoff record. It’s almost as though perceived early-career clutchness has no predictive value for the latter half of a QB’s career… How shocking!

But it’s worth noting that Brady isn’t alone — here are the QBs who had the most playoff games before and after age 27:

Player	Gm Thru 27	W Thru 27	eW Thru 27	Index	Gm After 27	W After 27	eW After 27	Index
Brett Favre	10	7	5.0	2.0	14	6	8.2	-2.2
Tom Brady	8	8	4.5	3.5	15	8	10.1	-2.1
Troy Aikman	7	6	4.5	1.5	9	5	5.9	-0.9
John Elway	7	4	3.9	0.1	14	10	7.8	2.2
Donovan McNabb	9	5	4.6	0.4	7	4	3.7	0.3
Peyton Manning	6	2	2.9	-0.9	14	7	9.0	-2.0
Dan Marino	6	3	4.0	-1.0	12	5	5.0	0.0
Steve McNair	5	3	2.3	0.7	5	2	2.4	-0.4
Joe Montana	5	4	2.6	1.4	16	12	10.5	1.5
Mark Brunell	4	2	1.0	1.0	5	2	2.4	-0.4
Dave Krieg	4	3	1.7	1.3	5	0	2.1	-2.1
Eli Manning	7	4	2.7	1.3	4	4	1.7	2.3
Ben Roethlisberger	10	8	5.5	2.5	4	2	2.3	-0.3

There are some exceptions, of course, but the majority of QBs who had playoff success (as measured by the Manning Index) through age 27 saw their late-career performance fall off a cliff. In fact, if we run a linear regression attempting to predict a QB’s Manning Index from age 28 onward using his Manning Index through age 27, we get a statistically significant negative coefficient:

Post27ManningIndex ~ -0.3997 * Thru27ManningIndex

In other words, the better you do relative to expectations (i.e., the “clutcher” you are) early in your career, the more you can expect to choke later in your career. The same trend also holds up, albeit with less significance, if we split quarterbacks’ Manning Indices at age 29 (through which Brady was +3.8) instead of 27. It’s unclear what causes this “Brady Effect” — whether it be Vegas and/or the betting public becoming overconfident in a QB who had success early in his career, the effect of opponents having more tape on a QB as he ages, or just a matter of QBs peaking as playoff performers at an early age — but it’s a real historical phenomenon.

While you could argue that quarterbacks lose their clutchness over time, maybe this just means clutch quarterbacking doesn’t exist — or at least isn’t detectable to the point that it could be used to predict future outcomes. Here’s a final bit of evidence for the latter camp: the correlation between a quarterback’s Manning Index in even-numbered years and his Manning Index in odd-numbered ones is just 0.05, meaning there’s really no relationship there at all.

If a quarterback’s Manning Index/clutchness was a persistent skill, we would expect to be able to predict it in one random subset of a player’s career from another random subset. But we see here that it’s impossible to predict the “even” half of a player’s playoff career from the “odd” half… and if we try to predict future performance from a quarterback’s Manning Index at an early age, we end up with a negative correlation! In other words, this basically means we should avoid forming any opinion of a quarterback’s clutchness from his playoff W-L record. Period.