An interesting discussion broke out in the comments to Friday’s post, as sn0mm1s pointed out a bizarre result he’d come across. Before the NFL adopted the 2-point conversion option following touchdowns, teams that trailed by 8 points entering the 4th quarter had a lower winning percentage than teams that trailed by 9 points.

My initial assumption was that this observation is tangentially related to a couple of posts Doug and I had at the old PFR blog where we saw that teams that scored 16 points have had a better winning percentage than teams that scored 21 points and about why teams that scored 13 points had fared better than teams with 14 points. In those cases, scoring 13 or 16 points wasn’t causing teams to win (relative to scoring 14 or 21 points) but rather was an effect of game situations. Teams that scored 13 or 16 points were occasionally kicking field goals to take the lead, while teams that scored only touchdowns were doing so because they were trailing big and eschewing field goal attempts.

In this case, the effect is less clear. Let’s start with the facts. From 1970 to 1993, take a look at the success of teams entering the 4th quarter trailing by 7, 8, or 9 points. The fifth and sixth columns there show how many points the teams had scored and allowed just before the start of the 4th quarter, while the last two columns show the final points scored and allowed data.

Just so we’re clear on how that table reads, there were 594 games where a team trailed by 7 entering the final frame, and they won 21.3% of those games (including ties). On average, at the start of the 4th quarter, they had scored 9.3 points and allowed 16.3 points, and, on average, the game ended with them scoring 15.6 points and allowing 22 points.

It shouldn’t be surprising that trailing by 8 entering the 4th quarter is worse than trailing by 7 in any environment, but especially so in one that does not have the two-point option. The surprising part is that 20% of teams won games when trailing by exactly 9 points at the start of the 4th quarter. That’s a particularly high number. It is possible that there is nothing to this effect; this distinction would fail standard significance tests, ^[1]If we include every game in NFL history, the results would only be significant at the p=0.13 level; over the period form ’70 to ’93, it would only be significant at the p=0.25 level. but we also know that we don’t always want to apply such methods. I’ll get into this more, but suffice it to say I do think there is something going on here, even if there is almost certainly not a causal relationship.

A look at two types of scores in each group is instructive. Trailing 14-6 or 21-13 entering the 4th quarter, teams won just 10 of 63 games (16%). On the other hand, when trailing 16-7 or 23-14, teams have pulled off the upset 13 of 52 times (25%).

I’ll note that there are some weird things going on with the data, which does make me wonder if this is due to randomness. For example, 32 of the 52 teams (62%) trailing 16-7 or 23-14 were at home compared to just 26 of 63 teams (41%) trailing 14-6 or 21-13. Overall, teams trailing by 9 points were significantly more likely to be at home (57% of the time) than teams trailing by 8 points (43%). So that’s… odd.

Although that doesn’t tell the full story, ^[2]i.e., this is not an example of Simpson’s Paradox as 9-point teams still did better than 8-point teams regardless of location. In 70 home games, 9-point teams won 15 games, for a 21.4% rate. In 52 road games, 9-point teams won 19.2% of the time (10 games). In 50 home games, teams trailing by 8 when entering the 4th quarter won 20% of the time, but on the road, they won just 7 of 66 such games (10.6%). Road comebacks when trailing by 8 points were extremely rare, while road teams trailing by 9 won nearly twice as often.

I have point spread data from 1978 to 1993, ^[3]Ignoring games played with replacement players in 1987 so let’s re-work the data around that time frame. Not much changes — 50 of the 88 teams (57%) to enter the 4th quarter trailing by 9 were the home team while just 38 out of 86 teams (44%) that trailed by 8 at the start of the final quarter were the hosts. Those 88 teams trailing by 9 points were, on average, 1.1-point underdogs entering the game. Of the 86 teams trailing by 8 points, they entered the game as … 1.2 point underdogs, on average.

I also looked at the scoring logs in the 4th quarter. We have 174 games to look at, with a nearly 50/50 split between 8- and 9-point games. There were 44 games where the trailing team did not score in the 4th quarter. 22 were in 8-point games, 22 were in 9-point games. So nothing there.

On the other hand, that makes the rest of the results more pronounced. This means that in 20 of the 66 games where a team trailing by 9 in the 4th quarter scored, they won, vs. 13 of 68 games when trailing by 8. Can we get any more detail?

Unfortunately, nothing else really interesting comes out of this. The teams trailing by 9 were more likely to kick field goals and the teams winning by 9 were less likely to pass for touchdowns. It is certainly possible that teams leading by 9 became more conservative; it doesn’t make rational sense in an era before the 2-point conversion for teams to treat an 8-point lead differently than a 9-point lead, but it’s possible that coaches did.

At this point, I pretty much ran out of ideas. Except for this: I have play by play logs since 2000. This is obviously during the 2-point conversion era, but maybe something useful would come of it?

From 2000 to 2011, 12 of the 59 teams to enter the 4th quarter trailing by 9 went on to win, or 20.3% of all teams. Over that same span, only 12 of the 85 teams trailing by 8 entering the 4th quarter — or 14% — ultimately won. ^[4]It’s probably more appropriate to note that 9 of the 85 teams went on to force overtime — and all three won. If we counted all OT games as tied, the winning percentage would drop to 12%, … Continue reading

So again we’re seeing significant differences and unexpected results. This makes me think it is likely that something actually is going on here, as a different set of data produced the same results despite it being less likely to have done so because of the two-point option.

On average, the 9-point games were 18.0 to 9.0 after three quarters, while the 8-point games were more high-scoring with an average of 20.2-12.2. Unlike with the prior data set, just 24 of the 59 nine-point games (40.1%) saw the trailing team at home; in addition, 33 of the 85 eight-point games (38.8%) were at home. This is more in line with what you would expect, as home teams are less likely to be trailing late in games. My guess is the numbers in the prior set on home/road splits were just random results.

Here’s another area where the data diverges. On average, the 8-point trailing teams were 2.8-point underdogs, while the 9-point trailing teams were 0.1-point favorites. Thirteen times since 2000 a team was favored to win by at least 7 points but then trailed by 9 entering the 4th quarter; four of those teams still went on to win (30.1%). Similarly, 3 of the 10 teams to trail by 8 after being a touchdown or more favorites went on to win. So in this case, it’s at least possible that truly better teams end up trailing by 9 points rather than 8.

I looked at the numbers through 3 quarters, and well, it wasn’t all that helpful.

Margin	G	PFs4Q	PAs4Q	Line	Yards	Rsh	Pass	RYD	PYD	TO	OPPTO	NY/A	YPC
-8	85	12.2	20.2	2.8	235.8	19.8	26.4	87	148.8	1.2	0.8	5.6	4.4
-9	59	9	18	-0.1	198	18.2	23.6	73.8	124.2	1.5	0.7	5.3	4.1

The teams trailing by 9 points seem to have played worse on offense, gaining significantly fewer yards through 3 quarters, averaging fewer yards per pass and per carry, and scoring 3.2 fewer points. That doesn’t exactly scream “more likely to score more points in the 4th quarter” but perhaps it also plays into the theory that the opponents are letting up.

Generally, teams trailing by 9 points have scored touchdowns, as a 9-point differential comes from the opponent getting 3 more FGs. 16-7, 19-10, 23-10, and so on. On the other hand, trailing by 8 usually happens when you score 2 FGs and your opponent scores 2 touchdowns. So you’re down 14-6, or 17-9, or 21-10.

Maybe this “trailing by 9” thing is random variation. Maybe it’s something I haven’t considered. The best I can come up with right now is that teams up by 9 really are letting the pedal up on the gas. Often times they will have scored on 3 more drives than their opponent, whereas the 8-point games often feature teams that have scored on the same number of drives. In addition, the overall offensive stats are closer in the 8-point game. If just a few teams up by 9 gave up too soon based on how the game unfolded, that would be enough to skew the numbers. But for now, that’s just a guess.