What Happened To Football's Next Great Stat?

ESPN's QBR passer rating was supposed to be the rating that quantified the most complicated position in sports. Instead, it more or less disappeared.
Share |

It’s football season, and so men are shouting. The merits of their respective arguments are measured in proportion to the decibel level at which those arguments are offered. Business as usual at ESPN.

Specifically, five men are sitting around an oblong table, yowling hymns to powerful notions they are unable or unwilling to define: intangibles, willpower, heart, effort, desire. It could be any NFL pregame or halftime or postgame show, but this is ESPN's Monday Night Countdown, and it is helping viewers get ready for some football. Those five men shouting are not enough, and so there are three other men at the site of Monday's game; they sit around another oblong table, voices also raised. Of these eight men, six are former players or coaches, and the other two are ESPN Personalities Chris Berman and Stuart Scott. Across the bottom of the screen, one of the most complex, involved and fascinating statistics about football ever created—the result of years of work and multitudes of dollars that only a profit-machine like ESPN could afford to spend on a statistic—is scrolling across the screen. No one is shouting about Total QBR, though. No one is even really whispering about it.

Total QBR (or more accurately, QBR, since the 'total' is a marketing prefix) was fighting an uphill battle from the start. Jon Gruden, former-coach-turned-ESPN-shouter, welcomed it into the world by cautioning, “I’m worried about it because it sounds like a lot of stuff and it’s another quarterback statistical rating formula.” And this was Gruden, a quarterback-obsessive who, in our high-volume/low-wattage national football converation, qualifies as an intellectual. The rest of the football discourse—all limp n' loud machismo and soft-focus Tebow slashfic and sentimental pomp—is backwards and anti-thought enough to make Tony Siragusa look like Slavoj Zizek. This was the environment in which QBR was expected to grow.

Last year, after years of expensive and time-consuming research, ESPN unveiled QBR with great fanfare as a replacement for the conventional (and widely unloved) passer rating. It was supposed to change the way quarterbacks were measured and compared, and since it was coming from the Worldwide Leader in Sports, it was sure not be left in academic obscurity. And that would be, or would have been, a good thing: passer rating is strange, from its maximum possible score of 158.3 on down, and both over-precise and vague; QBR, on the other hand, was supposed to be easily accessible to the average fan and complex enough to encapsulate both the evaluative concepts that stat-nerds love and the dizzying range of facets that comprise quarterbacking. It was supposed to be a tough thread of reason running through the noise-field emanating from those former players and coaches around those oblong tables. The shouters might even have recognized it, in time, and used it to support or rebut those endless arguments about the Most Clutch Quarterback, or help make clearer just what it is that a QB has when he has that indescribable and undeniable "it." QBR was supposed to be all these things, and so of course it is none of them.

"It’s got a lot of really cool stuff in it," Aaron Schatz of Football Outsiders told me about QBR. "I wish they used it more." Instead, QBR has become a literal sidebar to your regularly scheduled NFL programming—another number vanishing under the crush of all that noise.

***

Until recently, ESPN wanted no part of advanced statistics. Websites like Advanced NFL Stats, Football Outsiders, and their equivalents in every major sport evolved and thrived in the vacuum created by ESPN's proud lack of interest. These online intellectual communities developed in response to the dearth of any such analysis in the mainstream, and created their own statistics to better understand what we saw. But the online stathead scene has always been just a community, never a mass movement. Advanced stats were unlikely to reach the type of person who watches Around The Horn. QBR was, at least in part, an attempt to bridge that gap.

At its core, QBR is an amalgamation of several other stats. Dean Oliver (that's him at left), who's widely regarded as a stats genius and is ESPN’s director of production analytics, consulted the work that Brian Burke and others had done at Advanced NFL Stats and made it the foundation for QBR. The resulting hybrid stat incorporates the concepts of Wins Probability Added and Expected Points Added, both used by Burke. But QBR does several unique things Football Outsiders and Advanced NFL Stats don’t do in their ratings: it combines passing and rushing values, doesn’t adjust for the strength of the opponent, and incorporates ESPN’s game-charting data to assign responsibility to the quarterback depending on each play's outcome. The game-charting is a labor-intensive task that only ESPN could do, Schatz says, and it allows QBR to account for dropped passes, value sacks differently based on the situation, add new wrinkles to air yards, or count scrambles and planned rushes differently, among other things.

Those are, obviously and inevitably, a lot of "other things." But that's quarterbacking, and taking the position's many facets all into account helps make QBR such a useful metric. "The goal of QBR is to come up with the best possible measure of how well a quarterback played," Schatz says, and he emphasized it was pretty good at this. (It's worth mentioning that Schatz, aside from his longstanding role at Football Outsiders, is a regular on ESPN programming. He noted several times to me that he understands how his praise might be perceived, but stressed he truly does value QBR as a good metric.)

As with everything having to do with ESPN, QBR is not without its controversies. Most notably, QBR is proprietary—as are Football Outsiders stats like DYAR and DVOA—meaning that the statistical models behind the stat are still secret. This is an old conflict: the challenge is popularizing a product without losing the ability to commodify it. It's understandable, but it has brought QBR in for some criticism. Schatz defended it—so long as the components of the metric are revealed, which ESPN has done with QBR. "It’s a black box, but I’ll tell you what’s in the box, I just don’t let you see it," Schatz says. "I tell you it’s a ham sandwich. I don’t tell you how many slices of ham it is, or how thick it is, but you know it’s a ham sandwich. It’s the same way with QBR."

Without a doubt, the most controversial aspect of QBR—and the one that would surely please ESPN's bellowing knights of the oblong table—is the Clutch Index, which weights every play according to the situation's import. In football as in baseball, mentioning the word "clutch" is the advanced stats equivalent of yelling "fire" in a crowded theater; people get panicky, incredulous and angry at once, start shouting, and ultimately try and run away as fast as possible. Stuff gets trampled. It's not a good look.

It's also not necessarily a defensible concept. If Quarterback A plays in a blowout, and Quarterback B puts up identical stats as Quarterback A but does so in a close victory that features a fourth quarter comeback, Quarterback B will have the better QBR. (Schatz has written that offensive performance in the first quarter correlates better with winning than performance in any other quarter.) As Schatz clarified, in QBR, "quarterbacks are not rewarded for being in clutch situations, but rather for performing well in them."

But nobody knows how ESPN defines a clutch situation or how it’s weighted, which only adds to the community’s nervousness over the concept. Still, most people outside the comment sections of Football Outsiders do believe in the concept of clutch, and ESPN built QBR for everyone.

Which is a big part of how and why ESPN has not succeeded just yet in crossing-over QBR—the stat is for everyone, which alienates the purists in the stathead community, and yet it took 600 words for me to fully explain the damn thing and so is clearly not for the “average” fan as ESPN conceives that fan to exist. Those who need something simpler than the traditional passer rating because of its infamous 158.3 scale have, in QBR, a metric normalized on a 0-100 scale. But it's not simpler, really, because it couldn't be simpler and even begin to do its job. And it could, arguably, not be simple enough to resonate with brainiacs like Woody Paige while still deserving to be called an idea. The market for QBR is, inevitably, strained.

***

After ESPN's early season QBR-propaganda blitz in 2011, it gradually stopped being talked about on TV. Jaws and Gruden mentioned it less and less on the Monday Night Football broadcast; by my unscientific count, it hasn’t been mentioned once this season. QBR stopped being featured on Monday Night Countdown, and its appearances on SportsCenter tapered off. It is prominent on ESPN.com’s sub-blogs and displayed on Football Outsiders side-by-side with their own stats, but seldom discussed out loud. On Monday Night Football broadcasts, ESPN uses the old passer rating. Nobody is shouting about QBR anymore.

Which makes sense, since this is not a stat built to be shouted about, and because shouting is a lousy way to have a conversation. QBR requires the kind of discourse Burke, Schatz, and their sites have created over years of conversation; it's a metric built on several thousand lines of code, years of investment, and several complex concepts, all of them reduced to one number. It's a great idea, in a discourse broadly resistant to ideas.

Brian Burke isn’t surprised to see ESPN giving QBR the sink-or-swim treatment. "You cannot possibly watch and accurately process all 16 games in one weekend, no less all the games of a season, including the playoffs and Super Bowl," he says. "The purpose of a statistic is to be a first estimate towards the truth, because your brain can’t process all that information. So it’s going to be flawed, but you have to accept that." What ESPN wanted—and what ESPN's way of talking about the NFL demands—from QBR was an argument-ender, a metric that doubled as truth-with-capital-T. But stats don't do that. They can't.

Instead, the complications inherent in advanced statistics only increase the number of things to argue about. "Every time you’re factoring something in, you’re bringing in a possible source of error," Burke says. "Inherent in all statistics is the Central Limit Theorem, which is that a lot of these immeasurables are uncorrelated with what you’re trying to get at. So sometimes it will be the case that a quarterback did make an awesome, exceptionally accurate throw, and sometimes it will be the receiver [who made a big play happen], but on average it’s going to wash out in the long run. That’s why you need the long run."

QBR is maybe best understood as a metaphorical representation of ESPN’s own Central Limit Theorem. In the short run, it seems like a failure because no one is talking about it. But it hasn’t yet been given the long run in which to show its worth. It’s stuck scrolling below the Monday Night Football/Countdown crowd, who are doubtless as skeptical as Gruden about the stat ("I'm concerned about it," Gruden said after it launched. "I’m excited about it. But I’m skeptical.") and almost certainly less excited. If football's designated talkers can’t coherently discuss their feelings about QBR—or if they're so dedicated to the ambiguities of clutch and intangible that they find the very idea of quantification offensive—how can they be expected to discuss QBR itself?

QBR deserved better than this. It’s too useful, too intriguing, and too rich to be dismissed, and not too complicated to be discussed. You don’t have to agree with the clutch index in order to see its potential. You don’t have to embrace its lack of adjusting for defenses to see the value in measuring quarterback performances. But as it stands, QBR is stuck. ESPN programming can’t talk about it coherently, and as Schatz succinctly stated, "This is not meant to be a metric for Brian Burke." So if the ESPN crowd is concerned-excited-skeptical-worried about QBR, but QBR is not sufficiently hardcore for the stat nerds, what is QBR's audience?

There is a chance that QBR is itself the statistical noise that will wash out in the long run per the Central Limit Theorem Burke described; in that case, ESPN programming will continue to be ESPN programming, and the idea of QBR will wither, gather dust, disintegrate back into the internet. Or QBR could be the start of a little renaissance, an idea good enough to last and maybe earn spots for Aaron Schatz, Brian Burke, Dean Oliver and others at those oblong tables. It's still too soon to tell QBR's fate. But this is an easy idea to cheer for, and not just because some quiet reason would bring the volume down in Bristol.

NOTE: For more on the "Clutch Index," and an explanation of QBR from architect Dean Oliver, check out this update.


Share |

Comments

This site is excellent Read Full Article and so is how the subject matter was explained. I also like some of the comments too. Waiting for next post.

Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites! Sam Ivy

App Creator It looks like I am raking in money now, but the truth is that it was not always like that! Just 6 months ago I was completely struggling. I had no programming skills, no team and no money. The only thing I had was a great desire to create the game of my dreams. http://03dd4gqsm6tr0yebio1bnhhsxg.hop.clickbank.net/?tid=PHXDAN ...

www.rebelmouse.com/ Ultimate Herpes Protocol: Melanie Addington Reveals Her Science-Backed, All Natural Herpes Home Remedy That Boosts Your Immune System, Stops The Virus From Replicating, Eliminates Outbreaks, And Heals Blisters Without Medications And With No Side Effects... Written for consumers by a consumer, this fan site includes a fun and informative Ultimate Herpes Protocol review, ebook FAQ, and PDF download guide that provides an insider's look at the program and answers frequently asked questions about Melanie Addington's unique and highly popular herpes simplex type 1 and type 2 treatment guide. Please visit website: https://www.rebelmouse.com/ultimateherpesprotocolreviews/ ...

www.facebook.com/ Get an insider's look into the Text Your Ex Back program by author and relationship coach Michael Fiore. http://textyourexbackmichaelfiore.com/ ...

https://rebelmouse.com/boostyourbustjennybolton Boost Your Bust: Get The Sexy Curves That Drive Men Wild By Using The "Magic Formula" To Natural Breast Enlargement. This site is a leading resource for information about the Jenny Bolton Boost Your Bust PDF book and breast enhancement system that teaches you how to get bigger breasts naturally right from home. Includes a collection of Jenny Bolton Boost Your Bust reviews, PDF ebook download information, and tips for increasing your cup size from around the web. Please click here webpage: https://www.rebelmouse.com/boostyourbustjennybolton/ ...

rebelmouse.com/thedesiresystemreview Desire System: Dr. David T. Reveals The Holy Grail Of Seduction Techniques That Uses Emotional Contagion To Turn A Woman On And Compels Her To Chase You And Want Sex With You This site includes a Desire System review and buyers guide that provides an insider's look at the program and answers frequently asked questions about Dr. David's unique and controversial seduction system. Learn everything you need to know about the program so you can decide if the Desire System is the right choice for you before you buy it. Please visit https://www.rebelmouse.com/thedesiresystemreview/ ...

I am a new user of this site so here i saw multiple articles and posts posted by this site,I curious more interest in some of them hope you will give more information on this topics in your next articles.desenho web porto alegre rs

This post is very simple to read and appreciate without leaving any details out. Great work! You completed certain reliable points there. I did a search on the subject and found nearly all persons will agree with your blog.fazer backlinks

A good blog always comes-up with new and exciting information and while reading I have feel that this blog is really have all those quality that qualify a blog to be a one.I wanted to leave a little comment to support you and wish you a good continuation. Wishing you the best of luck for all your blogging efforts.desenho web curitiba pr

It’s very informative and you are obviously very knowledgeable in this area. You have opened my eyes to varying views on this topic with interesting and solid content. Savannah wedding planners

Thanks for the nice blog. It was very useful for me. I'm happy I found this blog. Thank you for sharing with us,I too always learn something new from your post.css layout tutorial

Regular visits listed here are the easiest method to appreciate your energy, which is why why I am going to the website everyday, searching for new, interesting info. Many, thank you!phuket property

This is also a very good post which I really enjoyed reading. It is not every day that I have the possibility to see something like this.. CPC cost

Trimming Down "I love the Venus Factor workouts and my body has transformed into something I thought I would never have!" Terry C. Warrington, Pennsylvania ! Terry’s Vital Stats: Age 53, weight lost: 33 lbs, inches lost: 8” on waist, 7” on hips. Due to time constraints I typically do half of a workout in the early morning and either finish it that night or the next morning. I do not spend a lot of time in the gym working out. This program is so doable even for someone who works full-time and is very involved in my church, son’s school and has a very active social life. I love the VF workouts and my body has transformed into something I thought I would never have! At my age I cannot believe the shape that I am finally getting, and I owe it all to the Venus Factor program. My 54th birthday is next month and I want to thank John for creating a workout and an environment that has helped me have the shape I have always dreamed of, I feel awesome! And other information, visit my webpage url: http://02c56hwst6ps8paffcxfrctrbr.hop.clickbank.net/?tid=PHXDAN ...

I like viewing web sites which comprehend the price of delivering the excellent useful resource free of charge. I truly adored reading your posting. Thank you! barcode scanner for Android

Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. bong blog

It has fully emerged to crown Singapore's southern shores and undoubtedly placed her on the global map of residential landmarks. I still scored the more points than I ever have in a season for GS. I think you would be hard pressed to find somebody with the same consistency I have had over the years so I am happy with that.Free heroes of the storm cheats

Thank you for your post, I look for such article along time, today i find it finally. this post give me lots of advise it is very useful for me. marvel contest of champions game cheats

It is perfect time to make some plans for the future and it is time to be happy. I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it!facebook hacker

Glad to chat your blog, I seem to be forward to more reliable articles and I think we all wish to thank so many good articles, blog to share with us. I will really appreciate the writer's choice for choosing this excellent article appropriate to my matter.Here is deep description about the article matter which helped me more.delhi agra jaipur

Hi! This is my first visit to your blog! We are a team of volunteers and new initiatives in the same niche. Blog gave us useful information to work. You have done an amazing job!www.yellowtaxicincinnati.com

It is perfect time to make some plans for the future and it is time to be happy. I’ve read this post and if I could I desire to suggest you few interesting things or tips. Perhaps you could write next articles referring to this article. I want to read more things about it!paleo diet recipes

Truly, this article is really one of the very best in the history of articles. I am a antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work! Clash of Clans Hack

I’m excited to uncover this page. I need to to thank you for ones time for this particularly fantastic read !! I definitely really liked every part of it and i also have you saved to fav to look at new information in your site.Free marvel champions hack

You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.http://theofficialwebsite.co/christian-h-girlfriend-activation-system-review/

Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing. home dna kit

I am looking for and I love to post a comment that "The content of your post is awesome" Great work! Wool Rug Cleaning New York

Thanks for the nice blog. It was very useful for me. I'm happy I found this blog. Thank you for sharing with us,I too always learn something new from your post.Organize events

I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.http://kwikspeak.com/

It's really nice and meanful. it's really cool blog. Linking is very useful thing.you have really helped lots of people who visit blog and provide them usefull information.search engine optimization vancouver

I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. In fact your creative writing abilities has inspired me to start my own BlogEngine blog now. Really the blogging is spreading its wings rapidly. Your write up is a fine example of it.area rug cleaning

Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I've been searching for. You have a clever yet attractive way of writing.UK Coverage

This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea.date

First You got a great blog .I will be interested in more similar topics. i see you got really very useful topics, i will be always checking your blog thanks. Lake Geneva WI Hotels

Your blogs further more each else volume is so entertaining further serviceable It appoints me befall retreat encore. I will instantly grab your rss feed to stay informed of any updates.rv dealers in colorado

Thank you for your post, I look for such article along time, today i find it finally. this post give me lots of advise it is very useful for me. http://gunbloodcheats.rocks/

Truly, this article is really one of the very best in the history of articles. I am a antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work! voyance finance

You might comment on the order system of the blog. You should chat it's splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.voyance

I am a new user of this site so here i saw multiple articles and posts posted by this site,I curious more interest in some of them hope you will give more information on this topics in your next articles. Glad to chat your blog, I seem to be forward to more reliable articles and I think we all wish to thank so many good articles, blog to share with us. Rossie Boling

Glad to chat your blog, I seem to be forward to more reliable articles and I think we all wish to thank so many good articles, blog to share with us. I will really appreciate the writer's choice for choosing this excellent article appropriate to my matter.Here is deep description about the article matter which helped me more.Portia Villareal

You actually make it look so easy with your performance but I find this matter to be actually something which I think I would never comprehend. It seems too complicated and extremely broad for me. I'm looking forward for your next post, I’ll try to get the hang of it!Shelley Lopez

I have express a few of the articles on your website now, and I really like your style of blogging. I added it to my favorite’s blog site list and will be checking back soon…Moses Walker

The next time I read a blog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to read, but I actually thought youd have something interesting to say. All I hear is a bunch of whining about something that you could fix if you werent too busy looking for attention.Brett Graham

I have bookmarked your website because this site contains valuable information in it. I am really happy with articles quality and presentation. Thanks a lot for keeping great stuff. I am very much thankful for this site.Theresa Simpson

This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea.Jacqueline Welch

Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing. crossfit shoes

Writing with style and getting good compliments on the article is quite hard, to be honest.But you've done it so calmly and with so cool feeling and you've nailed the job. This article is possessed with style and I am giving good compliment. Best!Shelley Lopez

Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future also.Rossie Boling

It is perfect time to make some plans for the future and it is time to be happy. I’ve read this post and if I could I desire to suggest you few interesting things or tips. Perhaps you could write next articles referring to this article. I want to read more things about it!Theresa Simpson