I would say they are trying to mitigate biases, but I don't think they can be gotten rid of.

 

There are basically two major ways of doing what they are doing: You get a bunch of experts to provide a score for each player, then you average out the scores (a consensus score of experts would mitigate against individual biases or outliers) or you create very comprehensive and rigid parameters for how to score each position (this means there are fewer "judgement" calls, but it's influenced by both who is applying the parameters and who devised them, which is where the human factor comes in).

 

It's not clear to me which they are doing (it sort of sounds like a mix of both), but the point is their final number is unique to PFF and a different group might come up with a different number altogether. I'd wager a guess that if you ranked quarterbacks according to QBR and to PFF grades each week they would be different.

 

Good points Habib. I am thinking about signing up for their service.

 

Also, from an article I found, part of this is funny:

 

Collinsworth used to be a customer until he decided he liked the company so much, he bought it. It began with a phone conversation early in 2012 with Neil Hornsby, who founded the site in 2007 and whose work was a key tool for the Giants in their Super Bowl XLVI run. "I typed into the 'contact us' section, 'Are you guys coaches? Who are you?' " Collinsworth recalled. "Three minutes later, I get a call from this Brit, Neil Hornsby. As soon as he started talking, all I could think of is, I can't believe I got hustled out of $26.99 by some con man from England."

 

But soon enough, Hornsby was answering technical questions about inside linebackers and guards. "I just shut up and started listening," Collinsworth said. "He knew more about it than me . . . It's a pretty amazing story."

 

Cris Collinsworth liked using Pro Football Focus, so he bought it | Newsday

Great stuff like always Habib.

 

To the objectivity, I partly agree. However, I think the idea is to do what ProFootballFocus.com does it, for example PFF:

 

1. Analyst grades every single player and every single play.

2. Second analyst reviews the first one.

3. Third analyst reviews the second one.

4. Pro Coach Network verifies the three analysts grades.

5. Advanced Normalization: The raw grades (as seen in our Premium Stats) are normalized to better account for the situation; this ranges from where the player lined up to the drop-back depth of the quarterback, to everything in between.

6. We set the grade. Convert to 1-100 scale.

 

And finally:

 

GOING BEYOND THE STATS - OUR GRADING SYSTEM IS CONCERNED WITH HOW A PLAYER PERFORMED ON A PARTICULAR PLAY, NOT THE OUTCOME OF THAT PLAY.

 

For example, if a quarterback hits a wide receiver perfectly in stride on a post route in between two defenders, and the receiver drops the ball, it goes down in the box score as an incompletion. But in our system, that quarterback receives a positive grade for making a great throw. His statistics should not be punished based on the fact that his receiver dropped a pass.

 

Our detailed analysis starts by adding to the base statistics with critical information, including the depth of the drop by the quarterback, the intended and actual point of attack on a run play, and the hang-time of punts and kickoffs on special teams.

 

We then add to this with a level of detail that surpasses any other data source: the grading of each player on a play-by-play basis to judge how each player has executed his assignment on a given play.

 

This in-depth grading allows us to judge player performance in every facet of the game before compiling individual plays into an overall grade for each player by game, by season, and beyond. As a result, we can show things like how two edge defenders may have the same overall grade, but one is a significantly better run defender while the other is significantly more disruptive as a pass-rusher.

 

https://www.profootballfocus.com/about/how-we-grade/

 

====================

 

In other words, like you I don't see Moneyball revolution but I definitely see a lot of objectivity in the process.

 

I really think PFF is the best I have read or seen. Granted I have not seen everything. Their problem and the real stumbling block is did they know the play and what every player is to do. Probably easier inside the organization than an outsider looking in.

 

Because of this, the Browns may have something far more than others have gone thus far. It is the Browns, but I can't believe they hired this guy without knowing and determining what can be analyzed and how will it impact the game.

 

I believe the Steelers are doing some sort of analytics on a higher scale than others. My reasoning is their going for two point conversions far more than others.

I really think PFF is the best I have read or seen. Granted I have not seen everything. Their problem and the real stumbling block is did they know the play and what every player is to do. Probably easier inside the organization than an outsider looking in.

 

Because of this, the Browns may have something far more than others have gone thus far. It is the Browns, but I can't believe they hired this guy without knowing and determining what can be analyzed and how will it impact the game.

 

I believe the Steelers are doing some sort of analytics on a higher scale than others. My reasoning is their going for two point conversions far more than others.

 

I am sure this would sound redundant to Cris and the ProFootballFocus people but they should probably invest in some analytics of their analytics for good measure. Maybe one with a common sense approach and one with a mathematical approach.

I know the Jags have used them for their past 2 draft classes. They actually have been leaning on it a bit more than other teams I know of the past 2 seasons.

I know the Jags have used them for their past 2 draft classes. They actually have been leaning on it a bit more than other teams I know of the past 2 seasons.

 

All teams use them to a point for draft selection and it probably widely varies how.. Not sure how many use them in other avenues for game planning, etc.

I just do not believe all plays in an NFL game are equal and would be interested in how a play in a 7-3 game in the second quarter is evaluated versus how a play in a 37-13 game in the fourth quarter is evaluated.

 

If my team's defense gives up 9 yards on the 3rd and 8 in the 7-3 game from the 42 yard line with under 2 minutes in the half that is different than if my team gives up 9 yards on a 3rd and 8 in a 37-13 game from my own 22 yard line with under 2 minutes left and the clock keeps running.

I just do not believe all plays in an NFL game are equal and would be interested in how a play in a 7-3 game in the second quarter is evaluated versus how a play in a 37-13 game in the fourth quarter is evaluated.

 

If my team's defense gives up 9 yards on the 3rd and 8 in the 7-3 game from the 42 yard line with under 2 minutes in the half that is different than if my team gives up 9 yards on a 3rd and 8 in a 37-13 game from my own 22 yard line with under 2 minutes left and the clock keeps running.

 

I am guessing that #5 tries to address that:

 

5. Advanced Normalization: The raw grades (as seen in our Premium Stats) are normalized to better account for the situation; this ranges from where the player lined up to the drop-back depth of the quarterback, to everything in between.

I know the Jags have used them for their past 2 draft classes. They actually have been leaning on it a bit more than other teams I know of the past 2 seasons.

 

I really think PFF is the best I have read or seen. Granted I have not seen everything. Their problem and the real stumbling block is did they know the play and what every player is to do. Probably easier inside the organization than an outsider looking in.

 

Because of this, the Browns may have something far more than others have gone thus far. It is the Browns, but I can't believe they hired this guy without knowing and determining what can be analyzed and how will it impact the game.

 

I believe the Steelers are doing some sort of analytics on a higher scale than others. My reasoning is their going for two point conversions far more than others.

 

The irony here is that when Joe Banner was the chief football guy at the Browns he sponsored a $100,000 study of the QBs available in the 2014 draft. Teddy Bridgewater was ranked #1, Derek Carr and Blake Bortles were also ranked highly. The Browns fired Banner and ignored his expensive study when they drafted Johnny Manziel with pick #22 ahead of Bridgewater (#32) and Carr (#36). Only Bortles was taken ahead of Johnny at #3 by the Jags.

 

Just think of the headaches and embarrassment that could have been avoided in Cleveland had they listened to the analytics.

The irony here is that when Joe Banner was the chief football guy at the Browns he sponsored a $100,000 study of the QBs available in the 2014 draft. Teddy Bridgewater was ranked #1, Derek Carr and Blake Bortles were also ranked highly. The Browns fired Banner and ignored his expensive study when they drafted Johnny Manziel with pick #22 ahead of Bridgewater (#32) and Carr (#36). Only Bortles was taken ahead of Johnny at #3 by the Jags.

 

Just think of the headaches and embarrassment that could have been avoided in Cleveland had they listened to the analytics.

 

Yes but if they did that, they wouldn't be Cleveland.

I haven't even read it all yet @Habib but I immediately thought of you.

 

Can a Moneyball approach work in the NFL?

 

That’s the question everyone is asking this week in response to the Browns hiring Paul DePodesta, the analytics maven who was prominently featured in the book Moneyballand played by Jonah Hill (in a largely fictionalized role) in the movie. DePodesta has had success applying the analytical approach commonly referred to as Moneyball during his tenure with the Cleveland Indians, Oakland Athletics and New York Mets, but many in the NFL world are skeptical that the same approach can work in the NFL.

 

Count Super Bowl-winning former Ravens coach Brian Billick as one of the skeptics. Billick said today on Mike & Mike that Moneyball simply isn’t going to work in the NFL.

 

“One of the most common questions I get is, Can you do Moneyball, for lack of a better term, in the NFL? And the answer is, No, you can’t,” Billick said. “You can’t quantify the game of football the way you do baseball. It’s not a statistical game. The parameters of the game, the number of bodies and what they’re doing in conjunction with one another.”

 

Billick is clearly correct that statistical analysis in football is not as advanced as it is in baseball. Most football analytics people would freely admit that. Baseball is a fundamentally different sport, in which one player’s performance is less reliant on his teammates’ performances, and therefore the statistics of a baseball player are a much better reflection of his own value than are the statistics of a football player.

 

But DePodesta is presumably smart enough to know that. The question, then, is not whether football statistics are as beneficial as baseball statistics, but whether DePodesta’s analytical approach to football statistics can find advantages over teams that don’t use the same approach. That’s a question that we haven’t seen answered yet because no NFL team has gone all-in on analytics the way teams like the Oakland A’s have in baseball. But DePodesta’s hiring is a big moment in the NFL: If he finds strategic advantages that help the Browns become winners, other teams will seek to emulate his approach. If DePodesta fails, analytics in football will take a hit.

 

Even more important than the fundamental differences in the sports of football and baseball are the fundamental differences in the business practices of Major League Baseball and the National Football League. For starters, the whole idea of the “money” in Moneyball was that the Oakland A’s had a lot less money to spend on players than the New York Yankees, Boston Red Sox and Anaheim Angels, which meant that the A’s had to find a way to compete on the cheap. But in the NFL, with revenue sharing and a salary cap, that’s really not an issue. The Browns have the same amount of money as a big-market rival like the New York Jets, and the same amount of money as a small-market rival like the Buffalo Bills.

 

So where DePodesta thrived at finding ways to help small-market teams compete by identifying valuable but inexpensive players (and then helped the big-market Mets win the pennant by identifying valuable players for a team that could spend whatever it took to acquire them), in Cleveland he has a different job. With the Browns, DePodesta isn’t looking for better values as much as he’s looking more broadly for better strategic practices. His success or lack thereof will go a long way toward determining whether the rest of the league agrees or disagrees with Billick’s premise that Moneyball can’t work in the NFL.

 

Brian Billick: You can?t do Moneyball in the NFL | ProFootballTalk

^ Interesting stuff, and a good point too about another difference between the NFL and MLB re: the "Moneyball" approach in that the whole idea was to find quality players who flew under the radar and could be acquired cheaply. Basically, everyone knows who the studs are, but if you can find someone who doesn't have crazy topline numbers and can still impact your team then it's a huge leg up.

 

The salary situation in the NFL changes those dynamics a little bit, but players also aren't moved around the NFL nearly as much. So, really your best leg up is finding out what sort of characteristics in college players translate to NFL success so you can draft better.

 

With all that said I'd also really like to see an article or something about how predictive analytics (or advanced statistical measures, otherwise) are used in the NFL. I've seen a few references to things being used already but I'm curious what the examples are.

I really like what the NFL did with the rookie salary cap a few years back. I am not exactly sure how that fits in this mix other than the allocation of resources.

 

Moving on, I think the analytics are great for teams to use with their own players and potential free agents. With their own players, I think it is a great metric for teams/coaches to say "here are the facts" you tell me how you can be a better player/teammate.

 

Where I think you make the strongest point, and where the analytics will really help, is drafting college players beyond the second round. If there is a secret analytical formula for late round draft picks that is where a team can turn the tide.

Stats get better and better all the time. They're no more perfect than they are in MLB, but every year somebody comes up with a better way to adjust them for situations, account for contributions from other players, and increase standardization.

 

No group of stats can accurately predict outcomes, but I'd bet they'd tend produce better outcomes over time. They certainly have in several other sports.

I am guessing that #5 tries to address that:

 

5. Advanced Normalization: The raw grades (as seen in our Premium Stats) are normalized to better account for the situation; this ranges from where the player lined up to the drop-back depth of the quarterback, to everything in between.

 

That doesn't sound like they have down to an exact science. EVERY play could be a candidate to "normalized".

 

Sounds like Bikini Stats to me......shows some stuff but can hide the really important stuff.

Since the Browns are doing it, it won't work.