Rating General Managers in the NFL Draft/NFP

  • To unlock all of features of Rams On Demand please take a brief moment to register. Registering is not only quick and easy, it also allows you access to additional features such as live chat, private messaging, and a host of other apps exclusive to Rams On Demand.

RamBill

Legend
Joined
Jul 31, 2010
Messages
8,874
Rating General Managers in the NFL Draft
Posted March 12, 2015 ·

http://www.nationalfootballpost.com/rating-general-managers-in-the-nfl-draft/

An NFL General Manager has a full plate of responsibilities ranging from signing or retaining free agents to overseeing the draft process. All these duties must be carried out in the context of the Salary Cap, adding another layer of complexity. The ultimate judgment regarding how well a General Manager does his job is, of course, his team’s won-lost record.

It should be noted that not everyone in our analysis carries the General Manager title. Nick Caserio of the Patriots, for example, is the Director of Player Personnel but is the closest person to a General Manager in their front office.

This article focuses on only one aspect of the job – – managing the draft process. We have typically used some measure of being a five-year starter in evaluating performance. That would not be useful in this analysis, though, because it effectively excludes the last six years in a 10-year analysis.

In order to facilitate a more current look at results, our rating were based on a comparison of actual starts and projected starts for all players drafted between 2005 and 2014. Let us take Frank Gore to illustrate the concept. Gore was drafted by San Francisco as the #65 selection in the 2005 draft. This means that he could have started a maximum of 160 (10 years times 16 games/year) games. Historically, players drafted at that point of the draft started about 36% of the maximum or about 58 games for a player drafted in 2005. The actual number of games started by Gore is 134, so he exceeded expectations by 76 games. This 76 game “surplus” is credited to Scot McCloughan, the General Manager at the time, because he was good enough or lucky enough to select Gore.

The percentage used is calculated for each of the Draft Ranges, as defined in earlier articles. This means there is no inherent advantage in our analysis from having an early first round choice (i.e., the top pick in the draft) compared to a later choice (e.g., pick #32). That is taken into consideration in calculating the expected number of starts.

This calculation is repeated for each player drafted between 2005 and 2014. A summation of the relevant individual scores is then made for each General Manager. The resulting total surplus or deficit for each General Manager is divided by the number of years in his tenure between 2005 and 2014, resulting in an average annual rate. McCloughan was, for example, employed as a General Manager for five years and ended up with a total surplus of 295 games, resulting in an average annual surplus of 59. The conversion to an annual rate is done to provide comparability among General Managers with different employment tenures.

An average annual surplus of zero indicates a General Manager that performs at exactly the league average. A high surplus is good. A deficit is bad.

The table that follows ranks all current General Managers by their average annual surplus or deficit, with the largest surplus indicating that the General Manager was the most effective at his job on draft day. Please note that the drafting General Manager receives credit for a player regardless of whether the player remains with the team. If Gore had left the 49ers after three seasons and played for the Chargers, for example, McCloughan would still receive credit for all of Gore’s starts.

The table requires some explanation:

The first two columns are self-explanatory.
The third column cannot be greater than 10 (representing the 10 years studied) and represents the total number of seasons spent as a General Manager from 2005 through 2014.

The next column cannot be greater than the preceding one and represents the number of years spent as General Manager of the current team during the 10-year time period.
The “Average Annual Surplus (Deficit)” column was calculated as explained above and the table is sorted by those values
The final column represents the other teams for which the person served as General Manager during the 10-year period.

cc5635fae7.png

There is no perfect way to statistically analyze a General Manager’s draft performance. What is presented here is one way of doing it. A surplus can result from drafting more NFL starters, players with longer careers than typical or some combination of the two.

A few highlights from the above:

Six teams have had the same General Manager for at least the past 10 seasons
Almost half the teams (14) are within 10 games per season of the average
Scot McCloughan left the 49ers for “mutual reasons” and went on to serve as an advisor to John Schneider, the #2 rated General Manager, for four drafts
Mike Maccagnan, the Jets’ new General Manager, came from the Houston Texans where he was Director of College Scouting

It is also interesting to look at people who served as General Managers for at least three seasons over the past 10 years and who are no longer employed as a General Manager. It is no surprise to Lions fans that Matt Millen is at the bottom of the rankings but some of the names at the top might be surprising.

6631c0b55a.png
 

jrry32

Rams On Demand Sponsor
Rams On Demand Sponsor
Joined
Jan 14, 2013
Messages
29,790
Pointing out the obvious, this also rewards GMs of teams with little talent because you might be forced to start draft picks that wouldn't be on the field for good teams.

If it were me, I would have looked at average approximate value(PFR stat) or average rating(PFF stat) of the guys they drafted. However, that also would have been a flawed method that would have punished newer GMs that hadn't had many players fully develop yet.

Plus, both of those stats are majorly flawed.

That all said, it would be a more accurate representation of quality rather than simply quantity.
 

CGI_Ram

Hamburger Connoisseur
Moderator
Joined
Jun 28, 2010
Messages
48,082
Name
Burger man
If it were me, I would have looked at average approximate value(PFR stat) or average rating(PFF stat) of the guys they drafted. However, that also would have been a flawed method that would have punished newer GMs that hadn't had many players fully develop yet.

Unless you also have a curve for #players drafted.

But I can't speak to those metrics.
 

Fatbot

Pro Bowler
Joined
Jun 25, 2014
Messages
1,467
The layers upon layers of the lack of logic in this makes it totally useless. One fun example, S.Janikowski was taken as the #17 overall pick. He has played 237 games and counting, which is probably more than the historic number of games "players drafted at that point of the draft started". Once again proving Al Davis is the best GM of all time -- or that this methodology is stupid.
 

Jorgeh0605

You had me at meat tornado.
2023 ROD Fantasy Champion
Joined
Jun 18, 2014
Messages
1,782
Pointing out the obvious, this also rewards GMs of teams with little talent because you might be forced to start draft picks that wouldn't be on the field for good teams.

If it were me, I would have looked at average approximate value(PFR stat) or average rating(PFF stat) of the guys they drafted. However, that also would have been a flawed method that would have punished newer GMs that hadn't had many players fully develop yet.

Plus, both of those stats are majorly flawed.

That all said, it would be a more accurate representation of quality rather than simply quantity.
I agree the the methodology isn't perfect but I think over the long run it is a rather good way of looking at one aspect of a GM's job.

You could argue that most GM's start out with little talent when they get a new team because their has to be a reason the last GM was fired right? This boosts the score a bit, but overtime it is somewhat evenly distributed.

I also feel that quantity should have an effect on the evaluation of a draft. If you got 2 starters for the price of one you are doing a good job at acquiring talent. Snead has that situation thanks to the RG3 trade. So quantity adequately fits in the model because it has the possibility of hurting as much helping. Trading back and drafting two busts vs trading back and gaining strong starters are both possibilities. Trading away draft picks for players like Trent Richardson also come into play. All that matters in the evaluation of a GM's draft performance and is reflected in this model. In the same manner, selling the farm to move up for a player only hurts you if he doesn't pan out. It positively impacts your team if he turns out to be a the start you though.

As for the comment on Al Davis and Janikowski from @Fatbot , this model actually shows that he was a below average GM when it came to the draft. It also would never prove that Al Davis was the "best GM of all time". This is strictly referring to their draft performance. So I'm not sure where the lack of logic is. I would love to be enlightened though.
 

Fatbot

Pro Bowler
Joined
Jun 25, 2014
Messages
1,467
the comment on Al Davis and Janikowski
This methodology would use "games played" to value the Janikowski pick as one of the best picks in NFL history. Many have instead argued taking a kicker that early was among the worst picks of all time. To have that individual example so far out of whack from what is considered good GM performance is an easy way to show the ridiculousness of the whole thing. Guess I was wrong, I will move along.
 

Jorgeh0605

You had me at meat tornado.
2023 ROD Fantasy Champion
Joined
Jun 18, 2014
Messages
1,782
This methodology would use "games played" to value the Janikowski pick as one of the best picks in NFL history. Many have instead argued taking a kicker that early was among the worst picks of all time. To have that individual example so far out of whack from what is considered good GM performance is an easy way to show the ridiculousness of the whole thing. Guess I was wrong, I will move along.
In stats these types of data are known as outliers and consist of less than 5% of the whole model and are as such insignificant and don't effect the outcome much. It doesn't prove the ridiculousness of the whole thing, just 5% of the whole thing. Hence why AL Davis still grades below average. You can't pull a specific data set out of a distribution like that since these models go with the trend and account for these kind of weird things. It seems like the author even tried to show the standard deviation being 10. Which means he was definitively a below average GM.

My point is just that individual examples don't necessarily ruin the entire model. And one could also argue that selecting Janikowski was a better pick than JaMarcus Russell. That premise is generally based on Janikowski playing more games and contributing more than Russell. If I had to pick between the two players I'd take the kicker knowing what I know now.
 

Fatbot

Pro Bowler
Joined
Jun 25, 2014
Messages
1,467
I'm making an argument against the premise itself, not its math. I'm sure the math is dandy -- it just doesn't amount to any meaningful conclusion because using "games played" to judge a draft pick as successful or not is meaningless. He might as well make a chart of "regular versus diet Mt. Dews consumed during the draft" to rank GM ability.
 

Jorgeh0605

You had me at meat tornado.
2023 ROD Fantasy Champion
Joined
Jun 18, 2014
Messages
1,782
I'm making an argument against the premise itself, not its math. I'm sure the math is dandy -- it just doesn't amount to any meaningful conclusion because using "games played" to judge a draft pick as successful or not is meaningless. He might as well make a chart of "regular versus diet Mt. Dews consumed during the draft" to rank GM ability.
But it isn't games played. It is games started. A starter for a long period is generally a good player.