Last offseason, Adam Steele helped administer a Wisdom of Crowds experiment on running backs and quarterbacks. Today, an update from Adam, along with some news. Below are Adam’s words:
Thanks to the opportunities Chase has given me at FP to publish my research and writing, I’ve decided to branch off and start my own website, quarterbacks.com. Ultimately, my mission for this site is to build the most complete database of NFL quarterbacks on the internet, a resource for statistics, history, opinion pieces, and FP-esque engagement among the readership. However, I can’t do this alone, so consider this an open invitation to the FP faithful to collaborate with me for this admittedly ambitious project. I welcome all types of submissions, including custom stats you’d like to publish and op-eds about anything related to NFL quarterbacks. If you think a certain QB is overrated or underrated and want to make a case for him, send it to me! At this juncture, the site is still under construction, and it will be a month or two before anything is published, so consider this the foundation building stage. For any aspiring writers out there, I’d like to help give you the same opportunity Chase gave me. Please email all inquiries and submissions to quarterbacks1031@gmail.com.
Wisdom of the Crowds: Ideas
Last offseason, Football Perspective ran crowdsourcing experiments to determine the greatest quarterbacks and running backs of all time. Given the amount of interest the community showed in WotC, I will be running more crowdsourcing projects this offseason! Before any votes are cast, I want your feedback on what you’d like to see in this year’s iterations. I definitely want to run a WotC for wide receivers (didn’t happen last year) and quarterbacks again (draws by far the most interest), but I’m certainly open to doing more if the readership desires. What other positions or units would you like to see crowdsourced?
Last year there were three main problems that I’d like to address and fix before the next go-around:
1) Lack of precision from ordinal rankings. An ordered list may be the simplest method to evaluate players, but it’s not the most accurate. Ordinal rankings don’t allow the voter to show the magnitude of difference between players. For example, if you think two players are head and shoulders ahead of the pack, that won’t be reflected in the linear gap between #2 and #3. My proposed solution is to switch from rankings to ratings, most likely on a 1-10 scale.
2) Difficulty comparing players across eras. It’s hard to compare a modern player with someone from the 50’s, and a number of participants last year voiced their struggle in dealing with this. I think the best solution is to separate players into groups based on their era, then rate all the players from each era together. This would help voters put players in proper context, knowing that we’re evaluating them only in relation to their direct peers. I would then take the winner from each era and put him in a pool for the overall GOAT title, which would involve a re-vote.
3) Voters accidentally leaving players off their ballot. Even for a football historian, it’s a daunting task to pick out X number of players from everyone in history who’s ever played the position. With an open ballot, it’s easy to forget a few players by accident, which several participants lamented in last year’s edition. This year, I’d strongly prefer to use a ballot with a predetermined pool of players for each participant to rate. I’m thinking maybe 15-20 players per era depending on the position. This solves the issue of forgetting players, forces voters to think about players they might not otherwise have, and provides statistical symmetry since every player will receive the same number of ratings.
Now I’ll open the forum to the FP readership. What do you think of my proposed changes? For those of you who participated last year, what did you like and dislike about it? I welcome any suggestions to make Wisdom of the Crowds a better experience for all!
Oh, and one note from Chase: does anyone have any recommendations on how to automate this process? That would obviously save us lots of time on the back end.