What Happened To Football's Next Great Stat?

ESPN's QBR passer rating was supposed to be the rating that quantified the most complicated position in sports. Instead, it more or less disappeared.
Share |

It’s football season, and so men are shouting. The merits of their respective arguments are measured in proportion to the decibel level at which those arguments are offered. Business as usual at ESPN.

Specifically, five men are sitting around an oblong table, yowling hymns to powerful notions they are unable or unwilling to define: intangibles, willpower, heart, effort, desire. It could be any NFL pregame or halftime or postgame show, but this is ESPN's Monday Night Countdown, and it is helping viewers get ready for some football. Those five men shouting are not enough, and so there are three other men at the site of Monday's game; they sit around another oblong table, voices also raised. Of these eight men, six are former players or coaches, and the other two are ESPN Personalities Chris Berman and Stuart Scott. Across the bottom of the screen, one of the most complex, involved and fascinating statistics about football ever created—the result of years of work and multitudes of dollars that only a profit-machine like ESPN could afford to spend on a statistic—is scrolling across the screen. No one is shouting about Total QBR, though. No one is even really whispering about it.

Total QBR (or more accurately, QBR, since the 'total' is a marketing prefix) was fighting an uphill battle from the start. Jon Gruden, former-coach-turned-ESPN-shouter, welcomed it into the world by cautioning, “I’m worried about it because it sounds like a lot of stuff and it’s another quarterback statistical rating formula.” And this was Gruden, a quarterback-obsessive who, in our high-volume/low-wattage national football converation, qualifies as an intellectual. The rest of the football discourse—all limp n' loud machismo and soft-focus Tebow slashfic and sentimental pomp—is backwards and anti-thought enough to make Tony Siragusa look like Slavoj Zizek. This was the environment in which QBR was expected to grow.

Last year, after years of expensive and time-consuming research, ESPN unveiled QBR with great fanfare as a replacement for the conventional (and widely unloved) passer rating. It was supposed to change the way quarterbacks were measured and compared, and since it was coming from the Worldwide Leader in Sports, it was sure not be left in academic obscurity. And that would be, or would have been, a good thing: passer rating is strange, from its maximum possible score of 158.3 on down, and both over-precise and vague; QBR, on the other hand, was supposed to be easily accessible to the average fan and complex enough to encapsulate both the evaluative concepts that stat-nerds love and the dizzying range of facets that comprise quarterbacking. It was supposed to be a tough thread of reason running through the noise-field emanating from those former players and coaches around those oblong tables. The shouters might even have recognized it, in time, and used it to support or rebut those endless arguments about the Most Clutch Quarterback, or help make clearer just what it is that a QB has when he has that indescribable and undeniable "it." QBR was supposed to be all these things, and so of course it is none of them.

"It’s got a lot of really cool stuff in it," Aaron Schatz of Football Outsiders told me about QBR. "I wish they used it more." Instead, QBR has become a literal sidebar to your regularly scheduled NFL programming—another number vanishing under the crush of all that noise.


Until recently, ESPN wanted no part of advanced statistics. Websites like Advanced NFL Stats, Football Outsiders, and their equivalents in every major sport evolved and thrived in the vacuum created by ESPN's proud lack of interest. These online intellectual communities developed in response to the dearth of any such analysis in the mainstream, and created their own statistics to better understand what we saw. But the online stathead scene has always been just a community, never a mass movement. Advanced stats were unlikely to reach the type of person who watches Around The Horn. QBR was, at least in part, an attempt to bridge that gap.

At its core, QBR is an amalgamation of several other stats. Dean Oliver (that's him at left), who's widely regarded as a stats genius and is ESPN’s director of production analytics, consulted the work that Brian Burke and others had done at Advanced NFL Stats and made it the foundation for QBR. The resulting hybrid stat incorporates the concepts of Wins Probability Added and Expected Points Added, both used by Burke. But QBR does several unique things Football Outsiders and Advanced NFL Stats don’t do in their ratings: it combines passing and rushing values, doesn’t adjust for the strength of the opponent, and incorporates ESPN’s game-charting data to assign responsibility to the quarterback depending on each play's outcome. The game-charting is a labor-intensive task that only ESPN could do, Schatz says, and it allows QBR to account for dropped passes, value sacks differently based on the situation, add new wrinkles to air yards, or count scrambles and planned rushes differently, among other things.

Those are, obviously and inevitably, a lot of "other things." But that's quarterbacking, and taking the position's many facets all into account helps make QBR such a useful metric. "The goal of QBR is to come up with the best possible measure of how well a quarterback played," Schatz says, and he emphasized it was pretty good at this. (It's worth mentioning that Schatz, aside from his longstanding role at Football Outsiders, is a regular on ESPN programming. He noted several times to me that he understands how his praise might be perceived, but stressed he truly does value QBR as a good metric.)

As with everything having to do with ESPN, QBR is not without its controversies. Most notably, QBR is proprietary—as are Football Outsiders stats like DYAR and DVOA—meaning that the statistical models behind the stat are still secret. This is an old conflict: the challenge is popularizing a product without losing the ability to commodify it. It's understandable, but it has brought QBR in for some criticism. Schatz defended it—so long as the components of the metric are revealed, which ESPN has done with QBR. "It’s a black box, but I’ll tell you what’s in the box, I just don’t let you see it," Schatz says. "I tell you it’s a ham sandwich. I don’t tell you how many slices of ham it is, or how thick it is, but you know it’s a ham sandwich. It’s the same way with QBR."

Without a doubt, the most controversial aspect of QBR—and the one that would surely please ESPN's bellowing knights of the oblong table—is the Clutch Index, which weights every play according to the situation's import. In football as in baseball, mentioning the word "clutch" is the advanced stats equivalent of yelling "fire" in a crowded theater; people get panicky, incredulous and angry at once, start shouting, and ultimately try and run away as fast as possible. Stuff gets trampled. It's not a good look.

It's also not necessarily a defensible concept. If Quarterback A plays in a blowout, and Quarterback B puts up identical stats as Quarterback A but does so in a close victory that features a fourth quarter comeback, Quarterback B will have the better QBR. (Schatz has written that offensive performance in the first quarter correlates better with winning than performance in any other quarter.) As Schatz clarified, in QBR, "quarterbacks are not rewarded for being in clutch situations, but rather for performing well in them."

But nobody knows how ESPN defines a clutch situation or how it’s weighted, which only adds to the community’s nervousness over the concept. Still, most people outside the comment sections of Football Outsiders do believe in the concept of clutch, and ESPN built QBR for everyone.

Which is a big part of how and why ESPN has not succeeded just yet in crossing-over QBR—the stat is for everyone, which alienates the purists in the stathead community, and yet it took 600 words for me to fully explain the damn thing and so is clearly not for the “average” fan as ESPN conceives that fan to exist. Those who need something simpler than the traditional passer rating because of its infamous 158.3 scale have, in QBR, a metric normalized on a 0-100 scale. But it's not simpler, really, because it couldn't be simpler and even begin to do its job. And it could, arguably, not be simple enough to resonate with brainiacs like Woody Paige while still deserving to be called an idea. The market for QBR is, inevitably, strained.


After ESPN's early season QBR-propaganda blitz in 2011, it gradually stopped being talked about on TV. Jaws and Gruden mentioned it less and less on the Monday Night Football broadcast; by my unscientific count, it hasn’t been mentioned once this season. QBR stopped being featured on Monday Night Countdown, and its appearances on SportsCenter tapered off. It is prominent on ESPN.com’s sub-blogs and displayed on Football Outsiders side-by-side with their own stats, but seldom discussed out loud. On Monday Night Football broadcasts, ESPN uses the old passer rating. Nobody is shouting about QBR anymore.

Which makes sense, since this is not a stat built to be shouted about, and because shouting is a lousy way to have a conversation. QBR requires the kind of discourse Burke, Schatz, and their sites have created over years of conversation; it's a metric built on several thousand lines of code, years of investment, and several complex concepts, all of them reduced to one number. It's a great idea, in a discourse broadly resistant to ideas.

Brian Burke isn’t surprised to see ESPN giving QBR the sink-or-swim treatment. "You cannot possibly watch and accurately process all 16 games in one weekend, no less all the games of a season, including the playoffs and Super Bowl," he says. "The purpose of a statistic is to be a first estimate towards the truth, because your brain can’t process all that information. So it’s going to be flawed, but you have to accept that." What ESPN wanted—and what ESPN's way of talking about the NFL demands—from QBR was an argument-ender, a metric that doubled as truth-with-capital-T. But stats don't do that. They can't.

Instead, the complications inherent in advanced statistics only increase the number of things to argue about. "Every time you’re factoring something in, you’re bringing in a possible source of error," Burke says. "Inherent in all statistics is the Central Limit Theorem, which is that a lot of these immeasurables are uncorrelated with what you’re trying to get at. So sometimes it will be the case that a quarterback did make an awesome, exceptionally accurate throw, and sometimes it will be the receiver [who made a big play happen], but on average it’s going to wash out in the long run. That’s why you need the long run."

QBR is maybe best understood as a metaphorical representation of ESPN’s own Central Limit Theorem. In the short run, it seems like a failure because no one is talking about it. But it hasn’t yet been given the long run in which to show its worth. It’s stuck scrolling below the Monday Night Football/Countdown crowd, who are doubtless as skeptical as Gruden about the stat ("I'm concerned about it," Gruden said after it launched. "I’m excited about it. But I’m skeptical.") and almost certainly less excited. If football's designated talkers can’t coherently discuss their feelings about QBR—or if they're so dedicated to the ambiguities of clutch and intangible that they find the very idea of quantification offensive—how can they be expected to discuss QBR itself?

QBR deserved better than this. It’s too useful, too intriguing, and too rich to be dismissed, and not too complicated to be discussed. You don’t have to agree with the clutch index in order to see its potential. You don’t have to embrace its lack of adjusting for defenses to see the value in measuring quarterback performances. But as it stands, QBR is stuck. ESPN programming can’t talk about it coherently, and as Schatz succinctly stated, "This is not meant to be a metric for Brian Burke." So if the ESPN crowd is concerned-excited-skeptical-worried about QBR, but QBR is not sufficiently hardcore for the stat nerds, what is QBR's audience?

There is a chance that QBR is itself the statistical noise that will wash out in the long run per the Central Limit Theorem Burke described; in that case, ESPN programming will continue to be ESPN programming, and the idea of QBR will wither, gather dust, disintegrate back into the internet. Or QBR could be the start of a little renaissance, an idea good enough to last and maybe earn spots for Aaron Schatz, Brian Burke, Dean Oliver and others at those oblong tables. It's still too soon to tell QBR's fate. But this is an easy idea to cheer for, and not just because some quiet reason would bring the volume down in Bristol.

NOTE: For more on the "Clutch Index," and an explanation of QBR from architect Dean Oliver, check out this update.

Share |


I can’t believe focusing long enough to research; much less write this kind of article. You’ve outdone yourself with this material without a doubt. It is one of the greatest contents.
habilitação quente

Very good points you wrote here..Great stuff...I think you've made some truly interesting points.Keep up the good work.

Great job here on _______ I read a lot of blog posts, but I never heard a topic like this. I Love this topic you made about the blogger's bucket list. Very resourceful.
rencontrer adulte

Great job here on _______ I read a lot of blog posts, but I never heard a topic like this. I Love this topic you made about the blogger's bucket list. Very resourceful.
rencontrer adulte

i was looking at a number of your posts in this website and i conceive this net web site is in imply of fact advisory! hold setting taking vicinity.. Tauchen In Hurghada

thanks for sharing this twist towards. I in truth moreover your weblog publicize no consider lots. you've got in truth shared a informative and appealing weblog publish following human beings.. 스포츠토토

thanks for taking the time to discuss that, I man or woman strongly just roughly this and in view of that during reality gone getting to know extra on the order of this best of ground. Do you thoughts updating your weblog post bearing in mind supplementary shrewdness? It need to be basically beneficial for anybody. Showbox APK Latest Version

positive website online, where did u arrive taking region at the back of the advice regarding this posting? i'm appreciative i discovered it even though, knocked out the weather be checking abet soon to find out what courting posts you tote happening. Shopify

Hello, this weekend is good for me, since this time i am reading this enormous informative article here at my home. Oral Pathology Los Angeles

this is such a pleasant useful resource which you are offering and you find the cash for it away for understandable. i really like seeing blog that appointment the fee of supplying a vibes useful resource for forgive. cheap college papers

This blog website is pretty cool! How was it made ! https://www.kplokusa.com/manifold-valves

This article was written by a real thinking writer. I agree many of the with the solid points made by the writer. I’ll be back. judi sbobet

Your table are too sword like to feature and relaxed to see. seguidores instagram

Today, I was just browsing along and came upon your blog. Just wanted to say good blog and this article helped me a lot, due to which I have found exactly I was looking. https://sites.google.com/site/duancanhovincityquan9/

Pretty nice post. I just stumbled upon your weblog and wanted to say that I have really enjoyed browsing your blog posts. After all I’ll be subscribing to your feed and I hope you write again soon! https://canhovincityquan9site.wordpress.com/

A debt of gratitude is in order for the blog entry pal! Keep them coming... kitchen remodeling walnut creek

Keep up the good work; I read few posts on this website, including I consider that your blog is fascinating and has sets of the fantastic piece of information. Thanks for your valuable efforts. Zwembad in tuin

Attractive, post. I just stumbled upon your weblog and wanted to say that I have liked browsing your blog posts. After all, I will surely subscribe to your feed, and I hope you will write again soon! http://canho-vincity-quan9.blogspot.com/2018/04/du-long-hau-tt-co-nhung-tien-ich-vuot-troi-so-voi-du-an-khac-cung-khu-vuc.html

Your work is truly appreciated round the clock and the globe. It is incredibly a comprehensive and helpful blog. Custom Web Design

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include. happy fasting wish

Pretty nice post. I just stumbled upon your weblog and wanted to say that I have really enjoyed browsing your blog posts. After all I’ll be subscribing to your feed and I hope you write again soon! situs judi bola terbesar

Your work is truly appreciated round the clock and the globe. It is incredibly a comprehensive and helpful blog. Test Bank for Living in the Environment 19th Edition by Miller

Wow, What an Outstanding post. I found this too much informatics. It is what I was seeking for. I would like to recommend you that please keep sharing such type of info.If possible, Thanks. mechanics of materials 3rd edition

This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again. nursing test banks

I am interested in such topics so I will address page where it is cool described. essay writing services

Here you will learn what is important, it gives you a link to an interesting web page: best essay writing service

I recommend only good and reliable information, so see it: buy facebook live viewers

find the good from the bad sometimes. but I think you’ve nailed it! would you mind updating your blog with more information جستجوی وکیل

That happened to me a few times. I was in an elevator going to a meeting or something, and a guy was like,, “I know you write for that site.” And I was like, visitar pagina

Among one of the finest pieces on the internet, freshening in its specificity.holm law group san diego

Thanks so much for sharing this awesome info! I am looking forward to see more postsby you! self assessment tax return crawley

Wow, cool post. I’d like to write like this too – taking time and real hard work to make a great article… but I put things off too much and never seem to get started. Thanks though. Haywards Heath Accountants

I am very much pleased with the contents you have mentioned. I wanted to thank you for this great article. Horley Accountants

Pretty nice post. I just stumbled upon your weblog and wanted to say that I have really enjoyed browsing your blog posts. After all I’ll be subscribing to your feed and I hope you write again soon! Horsham Accountants

I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. baltic sightseeing

I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. baltic sightseeing

I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. baltic sightseeing

I wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. baltic sightseeing

Superior post, keep up with this exceptional work. It's nice to know that this topic is being also covered on this web site so cheers for taking the time to discuss this! Thanks again and again! property tax accountants

Nice post mate, keep up the great work, just shared this with my friendz haywards heath accountants

The data you have posted is exceptionally helpful. The destinations you have alluded was great. A debt of gratitude is in order for sharing.. pool heater repair

This is really a nice and informative, containing all information and also has a great impact on the new technology. Thanks for sharing it, Crawley Accountants

Thanks for your insight for your fantastic posting. I’m glad I have taken the time to see this. nugenix ultimate testosterone reviews

Masseur naturiste paris domicilej'ai trouvé un super masseur naturiste paris super qualité vraiment formé aux USA c'est une grande différence en qualité masseur naturiste paris paris masseur naturiste.Magnifiques sensations masseur naturiste paris Paix repos tranquilité J'ai plus d'energie moins de fatigue masseur naturiste paris Qu'est-ce qui t'empecherai de prendre rdv maintenant? Rdv aller sur paris masseur naturiste.

This is such a great resource that you are providing and you give it away for free. eye masks

Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work! Hampton bay ceiling fan

Decent post. I was checking continually this online journal and I'm inspired! To a great degree helpful data extraordinarily the last part I administer to such data a considerable measure. I was looking for this sure data for quite a while. Much obliged to you and good fortunes. assisted living in potomac

If you don't mind proceed with this extraordinary work and I anticipate a greater amount of your amazing blog entries. custom architecture design for home

Somebody Sometimes with visits your online journal routinely and prescribed it as far as I can tell to peruse also. The method for composing is brilliant furthermore the substance is first rate. A debt of gratitude is in order for that knowledge you give the perusers! strawberry

CPH Services offers a professional carpet steam cleaning service on the Gold Coast. Our carpet cleaning prices are affordable without jeopardising professionalism. ?At CPH Services we only use eco-friendly products which are safe around young children and pets.