Adam Steele is back with the crazy notion that we should stop using stochastic, binary events in ratio form as the basis for judging quarterbacks. Fancy that. We thank Adam for his ideas and analysis.

The most commonly cited quarterback stats in mainstream analysis are touchdown passes and interceptions, usually presented as TD/INT ratio. This essentially functions as shorthand to compare the quantity of a player’s great plays against his terrible plays. But this is quite unfortunate since both stats are very noisy and situation dependent. TD/INT ratio not only lacks important information but it can be downright misleading at times.

Luckily for us the good folks at Pro Football Focus have come up with a much better alternative: Big Time Throws (BTT) and Turnover Worthy Plays (TWP). These stats are tabulated by watching film so they capture far more signal than the process-blind box score numbers. Passers get credited with a BTT when they make a throw that goes well beyond what’s expected on a given play, and this includes passes which are dropped or wiped out by penalty. Meanwhile a TWP is charged when a throw is made that has a good chance of being intercepted (whether it’s actually picked or not), or when the QB gets careless with the ball during his dropback and fumbles when such an error could’ve been avoided.

We can further improve both of these stat pairs by ditching ratios and using differences instead. 40/4 and 10/1 are the same ratio but +36 is much more impressive than +9. Volume matters, which difference accounts for.

In this study I looked at all 2021 quarterbacks and compared their TD-INT to their BTT-TWP. The difference between these two, uh, differences should give us a good idea of who was underrated and overrated by the traditional box score numbers. For now I’m calling this ‘Luck’ even though it’s not purely luck.

It’s worth noting that TD-INT overrates the entire league compared to BTT-TWP, as quarterbacks tend to get away with more unpunished mistakes than suffer inadequate reward from great throws. The league posted a TD-INT of 836-439 but a BTT-TWP of 822-746. Keep this in mind as nearly every QB will score as being lucky, it’s just a matter of how lucky compared to their peers.

According to this method, the luckiest QB’s by TD-INT last season were Patrick Mahomes and Ben Roethlisberger. But they were lucky for different reasons. In Mahomes’ case, his TD total was highly inflated compared to his relatively low BTT rate. While Mahomes is obviously a transcendent talent, his playing under an offensive genius like Andy Reid affords him more opportunities for schemed-up TD passes than most passers are bestowed. I watched replays of his 37 TD’s and many of them were wide open, short, and/or screen passes. That’s not necessarily a diss on Mahomes as it is an acknowledgment that he doesn’t always have to work as hard to score as other signal callers do.

Roethlisberger, meanwhile, was fairly represented by his TD total but got away with murder in the INT column. As age has eroded away his skills, Big Ben constantly put the ball in harm’s way last year, both through the air and with butterfingers in the pocket. He had more than triple the number of Turnover Worthy Plays as he did interceptions; that’s extremely fortunate and really papered over just how awful his final season was.

Interestingly, the next three luckiest QB’s are often maligned for being deficient in some fundamental way, but critics can’t necessarily put their finger on why that is. Well, here’s some stats to back those feelings up. Garoppolo, Tagovailoa, and Goff each had very low Big Time Throw totals which suggests that they aren’t doing much heavy lifting within their offenses. Jimmy G is often ridiculed for being a pawn in Kyle Shanahan’s brainiac scheme, while Tua and Goff are taken to task for having weak arms and not being able to make the tough throws that great QB’s do with relative ease.

At the other end of the luck spectrum we find Kyler Murray. I didn’t watch many Cardinals games last year, but according to PFF Murray was dropping dimes all over the field without seeing the reward show up in his box score numbers. There have been persistent whispers Kliff Kingsbury‘s scheme isn’t well suited for the NFL, so maybe Murray’s superior PFF numbers bear out the theory that he’s carrying a broken offense on his back.

Justin Fields had a terrible rookie season by conventional stats but looked halfway decent according to PFF’s charting. This isn’t a surprise given his overwhelmed head coach (Matt Nagy) and a dumpster fire of a supporting cast. He did some nice things on the field but his inferior teammates and coach couldn’t put his talent to use. Hopefully that changes with an overhauled coaching staff in Chicago.

Not many people would guess Derek Carr being tied for third in BTT, but he finally broke out of his shell and started throwing downfield in 2021. Like Alex Smith before him, Carr apparently throws a nice deep ball but didn’t bother to utilize it until he had a burner WR to connect with. Unfortunately said WR happened to be the disgraced and imprisoned Henry Ruggs; it’s no coincidence that Carr’s BTT rate steeply declined once Ruggs was cut from the team.

In the near future, I plan on doing this same analysis on every season from the PFF era (2006-present) to see what patterns might emerge from a larger sample. ^[1]I’d like to give a big shoutout to Pro Football Focus for giving me the green light to publish proprietary statistics at a free website. Analytical discourse can only improve if more people … Continue reading