You know the name Pete Gogolak, don’t you? The former Buffalo Bill placekicker is a famous figure in football history for two reasons. First, he played a key role in the merger between the AFL and NFL in the 1960s. ^[1]Gogolak was the first AFL player stolen by an NFL team. In 1965, Bob Timberlake succeeded on just one of his fifteen field goal attempts for the Giants. That prompted a desperate Wellington Mara to … Continue reading He’s also remembered for what he did on the field: Gogolak is widely credited with being the first soccer-style kicker in pro football history.

But Gogolak’s impact wasn’t limited to identifying the optimal technique for kicking a football: he also helped usher in an era of specialists. In the early days of the NFL, there was no room for a specialist as rosters were tiny and players played on offense, defense and special teams. Unlimited free substitution wasn’t permanently instituted until 1950, and as recently as 1963, teams were limited to just 37-man rosters.

Once teams were allowed to roster more players, and a certain unique brand of kicking was proven to be superior, a more specialized NFL emerged. In 1949, nobody would have signed a soccer-style kicker, or any person who could only kick a football. We joke now that kickers aren’t real football players, because back in 1949, a kicker would also need to play tight end or free safety. The idea that 5’11, 182-pound, 42-year-old Jason Hanson could be a contributing member of an NFL team is as noncontroversial in 2012 as it would have been laughable in 1952. It’s not going to take 60 years before an advanced statistical analyst — perhaps the front office version of a kicker — becomes a contributing member of an NFL organization.

This weekend, I sat down with Brian Burke, the founder of Advanced NFL Stats, a fantastic website on football, statistics and game theory. Burke’s win probability calculator has been one of the most exciting innovations in our industry. In Part II of this series, I’ll be interviewing Football Outsiders’ Aaron Schatz. Neither person is a threat to Ron Rivera’s job security anymore than Jason Hanson is a threat to steal Calvin Johnson’s job. Specialization is the way of the world, and hiring someone trained in the art of decision-making isn’t any different than choosing to hire a lawyer or doctor. We can’t expert anyone to be an expert in everything.

There are some things NFL coaches do wonderfully well. They know how to train and develop players, how to teach proper technique, and how to line up against different coverages. Coaches know how to design a play and how to craft a defense to stop such a play. They’re experts at designing an offense and sprinkling in the right constraint plays. Coaches aren’t better at this than the average person because coaches are geniuses; they’re better at it because that’s what they do. They’ve been trained to do these jobs for decades, and they’re very good at them.

But what does knowing how far to shade a defensive tackle have to do with knowing whether to go for it on 4th and 1 from the 50? In my view, it is about as similar as the skills involved in sprinting 30 yards down the field and catching a pass and the skills needed to kick a 45-yard field goal. Why should we expect there to be overlap in these areas? Why do we expect Marvin Lewis to know the inner workings of a win probability model? We shouldn’t, at least not anymore than we’d expect Calvin Johnson to be able to kick a field goal. That’s where guys like Schatz and Burke come in. Both have consulted with NFL teams. Below is my interview with Brian.

Q: Brian, how have teams reached out to you? Has your work with the New York Times and Washington Post been helpful in increasing your exposure?

Teams initially contacted me through email. In most cases, they set up a phone call or asked to meet with me in person before we went any further. In other cases, it’s strictly been an online relationship.

The New York Times and Washington Post exposure definitely helps. It lends credibility to the work, and it spreads the word. I feel a great debt of gratitude to the Times in particular. It was my analysis there of the 2009 Patriots-Colts 4th down play that catapulted Advanced NFL Stats into its 15 minutes of fame. But I’m learning now that my exposure to the league has also come thanks to early readers of my website. Young front office guys and analysts who buy into the analytics movement were readers of ANS in its early days when they were in grad school or even younger. They grew up speaking the language and using the tools ANS developed, so even many of the analysts for teams I’ve had no direct contact with ‘speak ANS.’ The big breakthrough was probably in 2008/09 when I was able to start doing analysis that could truly alter the way teams approached the game. Until then, advanced football stats had primarily been team ranking and player ranking stuff, which is great fodder for fans but is of little real value to teams.

Q: NFL teams are notoriously secretive: do teams ask you if you’re working with other teams? Have any teams asked you to sign an exclusivity agreement with them?

Whenever I do formal work for a team I’ll sign a non-disclosure agreement. That way the teams can be assured I’ll compartmentalize their own ideas and data. It works both ways; the stuff I do stays within their organization. No one has asked for a total exclusivity agreement, just exclusivity within the their division. Most of my portfolio is available for free online to everyone, but some of the projects I’ve worked on recently are proprietary to one team or another. That’s one reason I haven’t been posting as often as I had in the past.

Q: What type of work do you do for NFL teams? Are they mostly looking for work related to your Win Probability Model?

I can’t comment specifically on what I do for them. Teams are secretive about these things for obvious reasons. But it’s no secret that the win probability model has a lot of direct applications that teams would be interested in. Different teams are interested in different things. Some are interested in decision analysis (4th downs, onside kicks, and clock management, for example). Others are interested in player analysis or salary analysis. I’ve become aware of a couple teams that have been using my Expected Points model for their own purposes even though they had never contacted me. (The full model is freely available.)

Q: As someone who has been an outsider to the NFL, what has been the most surprising part of working with NFL teams?

The first thing that surprised me was how smart coaches really are. They didn’t get to where they are by being dull. They’re thinking about considerations that never dawned on guys like us. We’ll criticize them for sub-optimum decisions, but they have a hundred different things to juggle on any different play–clock, play, personnel package, opponent personnel, injuries, etc. It’s unfair for us to expect them to be game theorists and microeconomists on top of everything else they do.

The second surprising thing is that even teams that have analytic staff don’t often listen to them. The analytic staff is usually separate from the coaching staff on the org chart, and they don’t always have the coaches’ attention. It may be another half-generation before analytic-friendly coaches are in positions to take full advantage of what we’ve learned.

But the biggest surprise, in all honesty, was that some teams were taking advanced stats seriously. I first noticed hits at the site from various team facilities. (My old hit counter service made that very easy.) I thought they must be stray clicks simply because my site comes up on the first page when you Google ‘nfl stats’. My thinking was that we were doing all this analysis in a vacuum just for consumption by like-minded statheads, but it turns out ANS had a few secret admirers.

Q: Why do you think coaches aren’t listening to their analytic teams?

Right now the league is in a sub-optimum equilibrium. Nobody is playing the ‘optimum’ way. The thing is, if every coach plays outdated strategies, there’s little inducement to change. Coaches see the Super Bowl teams playing the same risk averse way that they do, so they mistakenly think that’s the optimal strategy.

Q: Are there any particular stats teams like from ANS, such as Air Yards?

Expected Points is very useful, but I would say Air Yards is one of those things for fans rather than teams. QB performance metrics like that aren’t that useful to teams because there is tape on every quarterback in the league. They can just watch the tape on a guy and learn a lot more than any stat can tell you. Stats are great for when you don’t have time to watch dozens of hours of film. They’re also useful for calibrating your expectations and checking your biases, but with only 16 games in a year, film can is more valuable to an expert than any statistic.

Q: You’ve written a lot about how the running game is overvalued and paying running backs in particular is a losing proposition. How would you sell this to an NFL team?

It should be an easy sell. All you have to do is ask, “Remember all those top running backs who led their teams to Super Bowl wins? Yeah, neither do I.” But I have a feeling that the view of running backs will be tough to overcome because there is so much emotion wrapped up around ‘franchise’ running backs. They are typically very popular players in their home city, so letting a top free agent walk would be very tough for an owner, even if the front office bought into what the stats say.

Q: Thanks again for your time, Brian. Last question: Do you think 20 years from now, every team will use a Win Probability Model to help them make decisions?

I think win probability models will have an impact on in-game decisions, but not the way you might think. NFL rules prohibit the use of computers during games, so the WP model will serve to recalibrate coaches’ intuitive judgments. Maybe we’ll be able to create some rules of thumb. Certainly the EPA-based rules are easy enough to use during games though. But I have no doubt that things like WP models will be used for many things. I wouldn’t say I’m only scratching the surface with its applications, but there is much more to be done.

I think there will be a tipping point in a few years. There will be a head coach that adopts the analytics mindset, and when other teams see his success they’ll quickly follow. The advantage is there for the first adopters. Just like with ‘Moneyball,’ once a critical mass of teams are all using the same analytical tools, the advantage is neutralized. Now is the time for teams to capitalize, before that tipping point occurs.