- Joined
- Feb 9, 2014
- Messages
- 20,922
- Name
- Peter
http://mmqb.si.com/2015/01/25/pro-football-focus-nfl-neil-hornsby-cris-collinsworth-analytics/
Football’s Focus Group
The ultimate outsider—an Englishman who never played the game—abandoned a profitable business to run an NFL advanced stats website. In just a few years he’s gone from 80 page views a week to overseeing an international staff of 80 that’s helping to transform the league
By Jenny Vrentas
CINCINNATI — In a mostly empty basement in suburban Ohio, save for two bookshelves filled with NFL record books dating back to 1942, an Englishman scribbles the names of pass rushers on a whiteboard. Cam Jordan. Jason Pierre-Paul. J.J. Watt. The guy holding the marker knows nearly everything about each snap these defensive ends played this season. Where they lined up. What the situation was. How well they did. Now he’s trying to answer the most incisive question: What does all this information mean?
Neil Hornsby isn’t doing this solely as a labor of love, though that’s precisely how his business, Pro Football Focus, began nine years ago in Luton, England, some 30 miles north of London. Living stateside since October, he now counts 13 NFL teams—40% of the league—as clients, including seven teams from this season’s playoff field. He also oversees a staff of roughly 80 full- and part-time employees who watch countless hours of game footage from their home offices in California, Northern Ireland and seemingly everywhere in between.
There’s a good chance you’ve heard of Pro Football Focus. Last summer, former NFL wideout Cris Collinsworth made a significant investment in the company, and he trusts the analysis so much that he uses its scouting information when commentating on NBC’s Sunday Night Football. PFF’s statistics are widely cited in NFL media reports about players, teams and trends, and Hornsby has appeared in several stories on The MMQB. But what the public sees on Pro Football Focus’s website is just a tidal pool compared to the ocean of information that NFL teams are paying very good money (PFF won’t disclose how much) to access.
It’s more than a data dump. Pro Football Focus will meet virtually any of its clients’ requests, such as measuring hang time to two decimal points for punts and kickoffs, and tracking which direction a center turns after the snap as a potential indicator of which offensive guard is the weaker link. PFF has also created a computer program that diagrammed each of the 32,779 regular-season plays from the line of scrimmage in 2014, depicting details like wide receiver splits, depth of players off the line, motions, and route combinations. That information is then linked to a team’s video system so coaches can put eyes on what the numbers are telling them.
On this Friday afternoon in early January, Hornsby isn’t focusing on the minutiae of game data. He’s looking at the bigger picture (think Moneyball) and trying to enhance the process by which Pro Football Focus grades players. For pass rushers, that means factoring in things such as down and distance, how many steps the quarterback dropped back, and what position they’re rushing from. He’s also trying to establish a player’s value over a replacement player (another idea borrowed from baseball’s sabermetrics).
Sacks weren’t an official NFL statistic until 1982, so analytics is one department in which football lags behind other sports. But a 51-year-old bloke from England—a guy who has never played a down of football in his life—is rapidly changing the game along with a collection of far-flung employees who have only ever met through Skype.
* * *
Based in London, Pro Football Focus’s Khaled Elsayed charts the Washington-New York Giants Thursday night game in September. (Jenny Vrentas/The MMQB)
LONDON — It’s 7 a.m. on a Friday morning in late September, and Khaled Elsayed has been up for a few hours. He awoke at 4:30 local time, right when the Washington-New York Giants Thursday Night Football tilt was ending at FedEx Field in Landover, Md. By sunrise in America, Pro Football Focus’s clients will have access to the preliminary set of information that Elsayed collects from the Giants’ 45-14 victory.
Elsayed, a full-time PFF analyst, works from his flat in the Stoke Newington neighborhood, above a Salvation Army and across the street from a furniture store. His fiancée asleep in the next room, he’s now breaking down plays well into the second quarter. The current play is a mere three-yard run by the Giants’ Andre Williams, but Elsayed’s stat sheet requires that he chart more than two dozen pieces of information.
On his first run-through, he fills in the blanks: Trick look? No. Trick play? No. Was a time out called? No. No-Huddle? No. Spot? L, for the left hash mark. Shotgun? Check. Run position? ML, for middle left of the offensive line. The runner doesn’t change course, but if he had, Elsayed would have recorded the intended point of attack and what initiated the change: F (forced by defense) or V (voluntary). Yards after contact? Two. Who made first contact? No. 31. Run concept? C, a designed cutback.
“One of the things that one NFL team wants us to collect is if a defender is blocked before he makes contact,” Elsayed says. “There are lots of little bits of information that might seem very useless. But NFL teams just love them, and think they’re indicative of performance.”
Where the ball is spotted is another piece of requested information so teams can track tendencies. There’s a different set of blanks to fill in on passing plays, including drop-back depth, time to throw, time to pressure, pass position, and turn of center—there are also checkboxes for screens, play-action, pump fakes, shotgun and pistol.
Elsayed then rewinds the play so he can grade the offensive linemen, giving “statistics” to a position group that is generally bereft of analytical measures. He notices right away that the left guard, rookie Weston Richburg, has done a nice job. Richburg seals off defensive end Jason Jenkins, who is trying to work inside toward the ‘A’ gap. He gets a positive grade. Two of his linemates don’t fare as well. Center J.D. Walton tries to attack a linebacker at the second level but whiffs, and right guard John Jerry allows a defender to squeeze his gap. Both get a negative grade on the play.
These grades sometimes draw the ire of fans, other analytics websites, NFL coaches, and even some of Pro Football Focus’s own clients. The criticism? “They take subjective perceptions and then they puree them into a number with a decimal point that looks very objective,” says Bleacher Report’s Mike Tanier, who has worked for Football Outsiders, another analytics service, since 2005. “There’s a scientific veneer there that doesn’t belong, which is worrisome.”
Pro Football Focus doles out grades for every snap in all of the NFL’s 267 games, assigning incremental marks from -2 to no grade to +2. (Punts are handled with a formula that former NFL punter Chris Kluwe helped devise.) To give some context, only one play in this game is graded at −2; it’s given to Washington quarterback Kirk Cousins for misreading obvious zone coverage and throwing an egregious third-quarter interception.
The idea is to provide a picture of how players are performing, especially when other statistics are either unavailable or misleading. An NFL fan since 1982, when the U.K.’s Channel 4 first showed American football highlights, Hornsby used to read Paul Zimmerman’s evaluations of players in Sports Illustrated that were based on film study. It inspired him to do it for every game. Hornsby’s staff works off a 106-page grading rubric; the guidelines include a 71-word definition for a “wham block” and three specific ways that an offensive lineman can receive a positive grade when double-teaming a defender.
When in doubt, Elsayed will flag plays for further review when the All-22 coaches’ film becomes available. “Am I being generous?” he writes after assigning a +1 grade to Eli Manning on a touchdown pass to Larry Donnell. It was a good throw, but the cornerback made the quarterback’s job easier by hesitating in coverage. On a 20-yard touchdown run by Alfred Morris, safety Antrel Rolle is only seen on the TV broadcast at the end of the play, trailing Morris. “One of the problems is, you can’t just go straight over the top—he’s got to defend the cutback,” Elsayed says. “But I need a different angle to see that. Another analyst might decide, when he looks at the All-22, that he was slow over the top. No grade for Rolle right now.”
Elsayed, 30, completes his analysis in less than six hours. He’s one of Pro Football Focus’ top analysts, having caught the football bug after catching a late-night Falcons-Saints game on TV in what he called “one drunken night at university.” Not long after, in 2007, he met Hornsby through an online forum for NFL fans in the U.K.; a small group of friends who met there now form the company’s core.
“It’s a bit weird,” Elsayed says. “These guys from England helped create a new industry.”
* * *
Two hours south of Green Bay, Nathan Jahnke memorizes shoe colors, hairstyles and even tattoos of players to quickly spot and record which 22 are on the field. (Jenny Vrentas/The MMQB)
MADISON, Wisc. — On an NFL Sunday (or a Thursday or a Monday night), Nathan Jahnke is in his apartment two hours south of Green Bay, filling a spreadsheet with letters and acronyms that serve as a road map for every play. He works mostly in real time, creating a player participation log as the game unfolds.
Jahnke, who has an actuarial science degree from the University of Wisconsin-Eau Claire, memorizes shoe colors, hairstyles and even tattoos of players to quickly spot and record which 22 are on the field. A player’s play count factors into his grade in the same way that the number of yards a running back gains is only meaningful if you know how many times he carried the ball. But Jahnke is taking down more than just personnel groups. He records a player’s position based on where he lines up on the field and in relation to others.
On offense, the positions of receivers and tight ends are specified with L (left), R (right), SL (slot left), SR (slot right), SLi (slot left inside), SLo (slot left outside), and so on. If they’re on the line of scrimmage, they’re tagged with “^”. The alignment of fullbacks is marked as “L” or “R,” and in rare cases when two fullbacks are used, they’re recorded, for instance, as FBOR (outside right) and FBIR (inside right).
On defense, strong safeties are within eight yards of the line of scrimmage; free safeties are beyond that threshold. Safeties and linebackers are marked L (left), R (right) or M (middle). Three-men and four-men lines are labeled differently, and there can be both a RE (right end) and a REO (right end outside), for a player outside the tackle.
There are also tags for what players do on each play. A tight end that stays in to pass block is marked with a “k,” and a tight end that lines up in a two-point stance rather than a three-point stance is marked with an “a.” Any defensive back or linebacker who pass rushes is tagged with a “z,” and a defensive lineman who drops into coverage is tagged with a “v.” A defensive back that initiates contact within a yard of the line of scrimmage is tagged with a “y” to represent press coverage.
All NFL coaches use participation reports as guides when grading players the day after a game. But the official NFL snap count summaries aren’t always ready in time, nor is the report on the following week’s opponent from the advance scout who was sent on the road. Pro Football Focus’s data is ready by 6 a.m.
Jahnke, who grew up on Badgers and Packers football, answered a job ad on Pro Football Focus’s site in 2010. He scored well on a practice game, and then earned bonuses for accuracy as a part-timer before becoming full-time. “I’m doing a fourth of the league every week, so I recognize who everyone is and their tendencies,” he says. Tendencies help form the backbone of all game plans, and the weekly PDF scouting reports that Pro Football Focus gives its clients have pie charts breaking down the number of plays run per personnel group, tables of success rates based on yardage gained, bar charts showing how players are deployed and receptions by route type.
This year, PFF took it a step further. Jahnke spends his Tuesdays tracking advanced player participation stats from the NFL coaches’ film, such as wide receiver splits, the beginning and end points for players in motion, and the technique employed by each defensive lineman (which is represented on a 0-44 scale). Another person works on an all-route analysis, recording each receiving route run by each player throughout the game.
A new computer program—the play analysis tool—processes this data into individual diagrams for every play in every game. It’s sortable by personnel group, or by down and distance, or by formation, and it’s all linked to video of the corresponding play.
On a recent morning elsewhere in the Midwest, one veteran NFL coach was sifting through the information.
* * *
The Lions’ research via Pro Football Focus let them know that Ndamukong Suh and Ezekiel Ansah could pin their ears back when Tony Romo was in shotgun during their wild-card showdown. (Sarah Glenn/Getty Images)
DETROIT — A day after the conference championship games, Gunther Cunningham is sitting at his desk at the Lions’ headquarters. It’s been two weeks since Detroit lost to the Cowboys in the wild-card round, but Cunningham, a senior coaching assistant, is still dissecting the game via Pro Football Focus.
Cunningham is speaking over the phone to The MMQB, but he sends a picture of his oversized computer screens, on which he’s using PFF’s play analysis tool to sort defensive plays based on the Cowboys’ offensive formations.
A self-described “oddball” when it comes to stats and analytics, Cunningham, 68, first got a computer in 1991 to help him explore on-field tendencies when he was the Los Angeles Raiders’ linebackers coach. He started using Pro Football Focus three years ago as the Lions’ defensive coordinator; he found out about it from his friend Jon Dykema, who is staff counsel for Detroit and works with the team president on player contracts.
Cunningham still records some of the same statistics on his own, including time to throw or defensive targets (mostly out of habit), but he and defensive quality control coach Matt Raich lean on Pro Football Focus to shave time off their research. Before the Cowboys game, for instance, it took them less than 30 minutes to pull up all the shotgun plays Dallas had used during its previous five games. They were able to discern the personnel groups, in part, by looking at the play diagrams and corresponding video footage.
“As soon as you get an idea what the team is doing, it helps with guys like Ndamukong Suh and Ezekiel Ansah, because you can say, when they’re in shotgun, pin your ears back and go get ’em,” Cunningham says. “It’s not guesswork.” If the Lions were playing New England, Cunningham adds, he could quickly sort the PFF database for all offensive plays with unbalanced lines to help prepare for the Patriots’ eligible/ineligible gambit.
In his current role, Cunningham prepares scouting reports for the coaching staff. (A 600-page report on the Seahawks’ offense, unused after the Lions were eliminated, sits on his desk). He also helps the general manager grade players, so Cunningham often finds himself in the crosshairs of the debate that plays out in many NFL team headquarters over Pro Football Focus.
Vikings coach Mike Zimmer has been one of the site’s most vocal critics, calling out PFF in the opening statement of a press conference last summer. “I guarantee they don’t know who is in our blitz package and what they are supposed to do,” he said. “I would just ask everybody to take that with a grain of salt, including our fans.” Patriots coach Bill Belichick responded to a question last week about defensive sub-packages with a seeming dig, saying, “I’m sure there’s a bunch of websites that [track] that, Pro Football Extra or whatever they are.”
Cunningham agrees with some of Zimmer’s criticism. Even after spending five seasons as the Lions’ defensive coordinator, Cunningham found it initially difficult to wrap his head around what everyone is supposed to be doing in Teryl Austin’s new system. So PFF analysts grading players without knowing the defensive call is one area, he says, “where they are a little bit short.”
But Cunningham has also seen the reverse: coaches grading favorite players more easily, or giving themselves too much credit for placing a player in the correct position when he makes a play. When one Lions player raised a stink about how Pro Football Focus graded him this season, Cunningham checked the PFF grades against his own analysis. “I wanted so much to tell him,” Cunningham says, “that they were right on the money.”
He views PFF analysts as young scouts, and he views their reports like those that come out of the BLESTO and National scouting organizations: more information that can help.
“I’ve been in the game a long time, and numbers have been really important to me,” Cunningham says. “There is so much room for statistical analysis and tape analysis. I had been thinking about investing my own money in something like this when I was done coaching. They beat me to the punch.”
* * *
A former minor league pitcher, Steve Palazzolo had always been interested in baseball’s sabermetrics and stumbled upon Pro Football Focus while looking to see if the same tools existed for football fans to better understand the game. (Jenny Vrentas/The MMQB and Palazzolo/Courtesy photo)
TEWKSBURY, Mass. — Steve Palazzolo spent eight seasons as a 6-foot-10 right-handed pitcher in the minor leagues, working his way up to Triple-A in the Giants and Mariners’ organizations. Now he analyzes football for a living.
The switch came in 2012. He had planned on playing one more season of independent league ball, as a closer for the Worcester Tornadoes, not far from where he grew up in Massachusetts. But the team was allowed only four veterans—those who had played six or more years of pro ball—on its roster, and a certain slugger took his spot.
“Jose Canseco ended my career,” Palazzolo says.
Palazzolo had always been interested in baseball’s sabermetrics, and he stumbled upon Pro Football Focus while looking to see if the same tools existed for football fans to better understand the game. After he signed on as full-time analyst, the major leagues came calling again, asking him to be a pitching coach or a scout. But he turned down those offers.
This season he’s had a primary role in developing Pro Football Focus’s newest project: analyzing college games. It’s a smart business decision that opens up a new potential client base, and it also gives their NFL teams another layer of data to help with their college scouting. The jump to analyzing all Division 1 FBS games—an additional 800+ games on top of the NFL schedule—is a huge undertaking. A third computer sits on Palazzolo’s desk in his townhouse, downloading college game film on one of three terabyte hard drives.
It’s a gold mine of data that can help teams sift through prospects the same way they do in free agency, but by an estimated factor of three or four. The grades can direct teams to a player’s best and worst games—for Jameis Winston, that would be against Boston College (best) and Louisville and Florida games (worst)—and isolate skill-sets such as throwing intermediate routes or getting the ball out while staring down pressure.
“One scouting director told us, ‘We’re looking to unlock gems and find the guy that all of a sudden came out of nowhere his senior year,’ ” Palazzolo says. “I can’t wait until we have enough data that we can go back and look at some of these sleepers. Some of these late-round guys, I’m going to go ahead and guess that if we graded them in college, they probably would have graded well.”
Pro Football Focus ranks Marcus Mariota as the highest-graded passer, and Winston in the middle of the pack. But the information they collect paints a more nuanced picture: Mariota faced nearly 50% less pressure than Winston in 2014, and the Oregon QB performed well, throwing three touchdowns, no interceptions and an average of 11.7 yards per attempt with defenders in his face. Winston, meanwhile, performed better against the blitz than a a traditional rush. Both quarterbacks struggled on rollouts. Mariota used play-action on 51% of his drop-backs, while Winston used it on just 16%. The high in the NFL this season was Alex Smith, who used play-action 31% of the time.
It’s that stat that illustrates why some analysts are doubting the Mariota hype: He did well with what was presented to him in college, but they aren’t sure all of the skills that made him a Heisman winner will translate well to the NFL.
That uncertainty is a gap Pro Football Focus hopes to bridge in the coming years.
* * *
Neil Hornsby abandoned his consultancy business in 2012 and took an 80% pay cut to grow Pro Football Focus. Last October, he moved from England to Cincinnati after forming a partnership with former NFL wideout and current NBC commentator Cris Collinsworth. (John DePetro and Jenny Vrentas/The MMQB)
CINCINNATI — On a train ride home from the University of Liverpool in 1984, Hornsby spent 85 pence on the U.K.’s Touchdown magazine (Dan Marino was on the cover) and became enthralled by the tables of stats that helped inform the highlights he loved to watch. He bought his collection of old NFL record books à la carte on EBay and spent most of the 1990s inputting data into a computer in his spare time, hoping to build a web encyclopedia of NFL records and facts. His hopes were dashed in 2003 when he stumbled upon such a site, ProFootballReference.com.
The idea for Pro Football Focus was born six months later, but the website drew just 80 page views a week (that’s not a typo) when it first launched. Hornsby was living in Luton, England, running a consultancy business that focused on something called “business process.” By 2008, his hobby was evolving into its own business. He and Ben Stockwell, a rabid NFL fan from Ware, England, finished a complete analysis of the ’08 season just a week before the ’09 season kicked off.
Hornsby started paying Stockwell—and later Elsayed and Sam Monson from Dublin, Ireland—out of his own pocket, making a push to analyze each week’s games before the next week kicked off. It was exhausting work, but it was during that ’09 season when Jon Berger, the Giants’ senior director of football information, sent Hornsby an out-of-the-blue message through the website’s rudimentary “contact us” form.
“This was in the days when the NFL didn’t give out player participation until after the season was over,” Hornsby says. “He said, ‘Can we have this info?’ I couldn’t believe it, because I thought every NFL team would have that info. That feeling kept us going.”
In 2012, Hornsby abandoned his consultancy business and took an 80% pay cut to focus on growing Pro Football Focus. He’s spent the last few years adding to his staff, traveling to NFL cities (including trips aboard The MMQB’s training camp RV, which Peter King explains in this week’s Monday Morning Quarterback column) to pitch his product to coaches and front offices. Collinsworth’s stake in the company is why Hornsby moved his wife, Claire, two sons, Alex and Ethan, and dogs Jess and Molly to Cincinnati, where Collinsworth played for the Bengals and still lives.
Former Bengals wide receiver and current NBC commentator Cris Collinsworth invested in Pro Football Focus last summer and helped expand the company’s reach to college football, which can help inform the draft process for its NFL clients. (Gene Lower and Richard Mackson for Sports Illustrated)
Before he invested, Collinsworth was simply looking for a way to get more information on all 32 NFL teams—and faster—to better inform NBC’s national television audience of roughly 21 million. He thought he was being punked when he first heard Hornsby’s thick British accent over the phone, but PFF passed all of his tests, including a side-by-side grading of a game from 2013. Their grades differed very little, and when they did, Collinsworth says, half the time they were right and he was wrong.
“What really impressed me is the fact that 13 NFL teams have contracted with Pro Football Focus for their data,” Collinsworth told The MMQB last August, after becoming a business partner. “I have been around the NFL for over 30 years, I know how hard it is to get behind the wall of those teams. And they’ve got 13 teams to trust their data. That’s huge.”
Collinsworth’s investment enabled the company to start analyzing college football. Down the road, who knows, maybe high schools will be next? But the core of Pro Football Focus’s business remains the NFL—and coming up with new ways to help their clients find an edge and save time. In his basement Hornsby is still scribbling on the whiteboard, trying to unlock the perfect formula to better rank pass rushers. He pulls up a document on a nearby tablet that PFF has made for its teams, showing how down and distance, the quarterback’s drop-back steps, and the position from which a player is rushing directly relate to his ability to get to the QB.
Using their new formulas to normalize data, for instance, a predominantly right-sided rusher like Robert Quinn moves up the rankings because it’s harder to get to the quarterback from that side. A predominantly left-sided rusher, like Charles Johnson, drops a few spots. Then there’s Watt, who rushed the quarterback 669 times in 2014, and no matter how you slice it, grades twice as high as any other defender in the league.
Statistics, of course, don’t provide all the answers, nor do they always give new answers, but they can offer a different perspective to help break down the game.
“Whether a team should go for it on fourth-and-1, there’s been some analysis of that,” Hornsby says. “But the truth of it is, what is the sample size of data for that game being played in Buffalo, at a particular temperature in December, with a right guard who has a dodgy hamstring and the halfback just broke up with his girlfriend the previous day? No amount of statistics can give you that answer. Only the coach can make that decision.
“But what we can do, we can say to a coach, ‘If you see Calvin Johnson lining up as the inside slot receiver on a play in Week 14, and in every other circumstance where he has lined up in that position he has run this route—would that be useful to you?’ ”
Football’s Focus Group
The ultimate outsider—an Englishman who never played the game—abandoned a profitable business to run an NFL advanced stats website. In just a few years he’s gone from 80 page views a week to overseeing an international staff of 80 that’s helping to transform the league
By Jenny Vrentas
CINCINNATI — In a mostly empty basement in suburban Ohio, save for two bookshelves filled with NFL record books dating back to 1942, an Englishman scribbles the names of pass rushers on a whiteboard. Cam Jordan. Jason Pierre-Paul. J.J. Watt. The guy holding the marker knows nearly everything about each snap these defensive ends played this season. Where they lined up. What the situation was. How well they did. Now he’s trying to answer the most incisive question: What does all this information mean?
Neil Hornsby isn’t doing this solely as a labor of love, though that’s precisely how his business, Pro Football Focus, began nine years ago in Luton, England, some 30 miles north of London. Living stateside since October, he now counts 13 NFL teams—40% of the league—as clients, including seven teams from this season’s playoff field. He also oversees a staff of roughly 80 full- and part-time employees who watch countless hours of game footage from their home offices in California, Northern Ireland and seemingly everywhere in between.
There’s a good chance you’ve heard of Pro Football Focus. Last summer, former NFL wideout Cris Collinsworth made a significant investment in the company, and he trusts the analysis so much that he uses its scouting information when commentating on NBC’s Sunday Night Football. PFF’s statistics are widely cited in NFL media reports about players, teams and trends, and Hornsby has appeared in several stories on The MMQB. But what the public sees on Pro Football Focus’s website is just a tidal pool compared to the ocean of information that NFL teams are paying very good money (PFF won’t disclose how much) to access.
It’s more than a data dump. Pro Football Focus will meet virtually any of its clients’ requests, such as measuring hang time to two decimal points for punts and kickoffs, and tracking which direction a center turns after the snap as a potential indicator of which offensive guard is the weaker link. PFF has also created a computer program that diagrammed each of the 32,779 regular-season plays from the line of scrimmage in 2014, depicting details like wide receiver splits, depth of players off the line, motions, and route combinations. That information is then linked to a team’s video system so coaches can put eyes on what the numbers are telling them.
On this Friday afternoon in early January, Hornsby isn’t focusing on the minutiae of game data. He’s looking at the bigger picture (think Moneyball) and trying to enhance the process by which Pro Football Focus grades players. For pass rushers, that means factoring in things such as down and distance, how many steps the quarterback dropped back, and what position they’re rushing from. He’s also trying to establish a player’s value over a replacement player (another idea borrowed from baseball’s sabermetrics).
Sacks weren’t an official NFL statistic until 1982, so analytics is one department in which football lags behind other sports. But a 51-year-old bloke from England—a guy who has never played a down of football in his life—is rapidly changing the game along with a collection of far-flung employees who have only ever met through Skype.
* * *
Based in London, Pro Football Focus’s Khaled Elsayed charts the Washington-New York Giants Thursday night game in September. (Jenny Vrentas/The MMQB)
LONDON — It’s 7 a.m. on a Friday morning in late September, and Khaled Elsayed has been up for a few hours. He awoke at 4:30 local time, right when the Washington-New York Giants Thursday Night Football tilt was ending at FedEx Field in Landover, Md. By sunrise in America, Pro Football Focus’s clients will have access to the preliminary set of information that Elsayed collects from the Giants’ 45-14 victory.
Elsayed, a full-time PFF analyst, works from his flat in the Stoke Newington neighborhood, above a Salvation Army and across the street from a furniture store. His fiancée asleep in the next room, he’s now breaking down plays well into the second quarter. The current play is a mere three-yard run by the Giants’ Andre Williams, but Elsayed’s stat sheet requires that he chart more than two dozen pieces of information.
On his first run-through, he fills in the blanks: Trick look? No. Trick play? No. Was a time out called? No. No-Huddle? No. Spot? L, for the left hash mark. Shotgun? Check. Run position? ML, for middle left of the offensive line. The runner doesn’t change course, but if he had, Elsayed would have recorded the intended point of attack and what initiated the change: F (forced by defense) or V (voluntary). Yards after contact? Two. Who made first contact? No. 31. Run concept? C, a designed cutback.
“One of the things that one NFL team wants us to collect is if a defender is blocked before he makes contact,” Elsayed says. “There are lots of little bits of information that might seem very useless. But NFL teams just love them, and think they’re indicative of performance.”
Where the ball is spotted is another piece of requested information so teams can track tendencies. There’s a different set of blanks to fill in on passing plays, including drop-back depth, time to throw, time to pressure, pass position, and turn of center—there are also checkboxes for screens, play-action, pump fakes, shotgun and pistol.
Elsayed then rewinds the play so he can grade the offensive linemen, giving “statistics” to a position group that is generally bereft of analytical measures. He notices right away that the left guard, rookie Weston Richburg, has done a nice job. Richburg seals off defensive end Jason Jenkins, who is trying to work inside toward the ‘A’ gap. He gets a positive grade. Two of his linemates don’t fare as well. Center J.D. Walton tries to attack a linebacker at the second level but whiffs, and right guard John Jerry allows a defender to squeeze his gap. Both get a negative grade on the play.
These grades sometimes draw the ire of fans, other analytics websites, NFL coaches, and even some of Pro Football Focus’s own clients. The criticism? “They take subjective perceptions and then they puree them into a number with a decimal point that looks very objective,” says Bleacher Report’s Mike Tanier, who has worked for Football Outsiders, another analytics service, since 2005. “There’s a scientific veneer there that doesn’t belong, which is worrisome.”
Pro Football Focus doles out grades for every snap in all of the NFL’s 267 games, assigning incremental marks from -2 to no grade to +2. (Punts are handled with a formula that former NFL punter Chris Kluwe helped devise.) To give some context, only one play in this game is graded at −2; it’s given to Washington quarterback Kirk Cousins for misreading obvious zone coverage and throwing an egregious third-quarter interception.
The idea is to provide a picture of how players are performing, especially when other statistics are either unavailable or misleading. An NFL fan since 1982, when the U.K.’s Channel 4 first showed American football highlights, Hornsby used to read Paul Zimmerman’s evaluations of players in Sports Illustrated that were based on film study. It inspired him to do it for every game. Hornsby’s staff works off a 106-page grading rubric; the guidelines include a 71-word definition for a “wham block” and three specific ways that an offensive lineman can receive a positive grade when double-teaming a defender.
When in doubt, Elsayed will flag plays for further review when the All-22 coaches’ film becomes available. “Am I being generous?” he writes after assigning a +1 grade to Eli Manning on a touchdown pass to Larry Donnell. It was a good throw, but the cornerback made the quarterback’s job easier by hesitating in coverage. On a 20-yard touchdown run by Alfred Morris, safety Antrel Rolle is only seen on the TV broadcast at the end of the play, trailing Morris. “One of the problems is, you can’t just go straight over the top—he’s got to defend the cutback,” Elsayed says. “But I need a different angle to see that. Another analyst might decide, when he looks at the All-22, that he was slow over the top. No grade for Rolle right now.”
Elsayed, 30, completes his analysis in less than six hours. He’s one of Pro Football Focus’ top analysts, having caught the football bug after catching a late-night Falcons-Saints game on TV in what he called “one drunken night at university.” Not long after, in 2007, he met Hornsby through an online forum for NFL fans in the U.K.; a small group of friends who met there now form the company’s core.
“It’s a bit weird,” Elsayed says. “These guys from England helped create a new industry.”
* * *
Two hours south of Green Bay, Nathan Jahnke memorizes shoe colors, hairstyles and even tattoos of players to quickly spot and record which 22 are on the field. (Jenny Vrentas/The MMQB)
MADISON, Wisc. — On an NFL Sunday (or a Thursday or a Monday night), Nathan Jahnke is in his apartment two hours south of Green Bay, filling a spreadsheet with letters and acronyms that serve as a road map for every play. He works mostly in real time, creating a player participation log as the game unfolds.
Jahnke, who has an actuarial science degree from the University of Wisconsin-Eau Claire, memorizes shoe colors, hairstyles and even tattoos of players to quickly spot and record which 22 are on the field. A player’s play count factors into his grade in the same way that the number of yards a running back gains is only meaningful if you know how many times he carried the ball. But Jahnke is taking down more than just personnel groups. He records a player’s position based on where he lines up on the field and in relation to others.
On offense, the positions of receivers and tight ends are specified with L (left), R (right), SL (slot left), SR (slot right), SLi (slot left inside), SLo (slot left outside), and so on. If they’re on the line of scrimmage, they’re tagged with “^”. The alignment of fullbacks is marked as “L” or “R,” and in rare cases when two fullbacks are used, they’re recorded, for instance, as FBOR (outside right) and FBIR (inside right).
On defense, strong safeties are within eight yards of the line of scrimmage; free safeties are beyond that threshold. Safeties and linebackers are marked L (left), R (right) or M (middle). Three-men and four-men lines are labeled differently, and there can be both a RE (right end) and a REO (right end outside), for a player outside the tackle.
There are also tags for what players do on each play. A tight end that stays in to pass block is marked with a “k,” and a tight end that lines up in a two-point stance rather than a three-point stance is marked with an “a.” Any defensive back or linebacker who pass rushes is tagged with a “z,” and a defensive lineman who drops into coverage is tagged with a “v.” A defensive back that initiates contact within a yard of the line of scrimmage is tagged with a “y” to represent press coverage.
All NFL coaches use participation reports as guides when grading players the day after a game. But the official NFL snap count summaries aren’t always ready in time, nor is the report on the following week’s opponent from the advance scout who was sent on the road. Pro Football Focus’s data is ready by 6 a.m.
Jahnke, who grew up on Badgers and Packers football, answered a job ad on Pro Football Focus’s site in 2010. He scored well on a practice game, and then earned bonuses for accuracy as a part-timer before becoming full-time. “I’m doing a fourth of the league every week, so I recognize who everyone is and their tendencies,” he says. Tendencies help form the backbone of all game plans, and the weekly PDF scouting reports that Pro Football Focus gives its clients have pie charts breaking down the number of plays run per personnel group, tables of success rates based on yardage gained, bar charts showing how players are deployed and receptions by route type.
This year, PFF took it a step further. Jahnke spends his Tuesdays tracking advanced player participation stats from the NFL coaches’ film, such as wide receiver splits, the beginning and end points for players in motion, and the technique employed by each defensive lineman (which is represented on a 0-44 scale). Another person works on an all-route analysis, recording each receiving route run by each player throughout the game.
A new computer program—the play analysis tool—processes this data into individual diagrams for every play in every game. It’s sortable by personnel group, or by down and distance, or by formation, and it’s all linked to video of the corresponding play.
On a recent morning elsewhere in the Midwest, one veteran NFL coach was sifting through the information.
* * *
The Lions’ research via Pro Football Focus let them know that Ndamukong Suh and Ezekiel Ansah could pin their ears back when Tony Romo was in shotgun during their wild-card showdown. (Sarah Glenn/Getty Images)
DETROIT — A day after the conference championship games, Gunther Cunningham is sitting at his desk at the Lions’ headquarters. It’s been two weeks since Detroit lost to the Cowboys in the wild-card round, but Cunningham, a senior coaching assistant, is still dissecting the game via Pro Football Focus.
Cunningham is speaking over the phone to The MMQB, but he sends a picture of his oversized computer screens, on which he’s using PFF’s play analysis tool to sort defensive plays based on the Cowboys’ offensive formations.
A self-described “oddball” when it comes to stats and analytics, Cunningham, 68, first got a computer in 1991 to help him explore on-field tendencies when he was the Los Angeles Raiders’ linebackers coach. He started using Pro Football Focus three years ago as the Lions’ defensive coordinator; he found out about it from his friend Jon Dykema, who is staff counsel for Detroit and works with the team president on player contracts.
Cunningham still records some of the same statistics on his own, including time to throw or defensive targets (mostly out of habit), but he and defensive quality control coach Matt Raich lean on Pro Football Focus to shave time off their research. Before the Cowboys game, for instance, it took them less than 30 minutes to pull up all the shotgun plays Dallas had used during its previous five games. They were able to discern the personnel groups, in part, by looking at the play diagrams and corresponding video footage.
“As soon as you get an idea what the team is doing, it helps with guys like Ndamukong Suh and Ezekiel Ansah, because you can say, when they’re in shotgun, pin your ears back and go get ’em,” Cunningham says. “It’s not guesswork.” If the Lions were playing New England, Cunningham adds, he could quickly sort the PFF database for all offensive plays with unbalanced lines to help prepare for the Patriots’ eligible/ineligible gambit.
In his current role, Cunningham prepares scouting reports for the coaching staff. (A 600-page report on the Seahawks’ offense, unused after the Lions were eliminated, sits on his desk). He also helps the general manager grade players, so Cunningham often finds himself in the crosshairs of the debate that plays out in many NFL team headquarters over Pro Football Focus.
Vikings coach Mike Zimmer has been one of the site’s most vocal critics, calling out PFF in the opening statement of a press conference last summer. “I guarantee they don’t know who is in our blitz package and what they are supposed to do,” he said. “I would just ask everybody to take that with a grain of salt, including our fans.” Patriots coach Bill Belichick responded to a question last week about defensive sub-packages with a seeming dig, saying, “I’m sure there’s a bunch of websites that [track] that, Pro Football Extra or whatever they are.”
Cunningham agrees with some of Zimmer’s criticism. Even after spending five seasons as the Lions’ defensive coordinator, Cunningham found it initially difficult to wrap his head around what everyone is supposed to be doing in Teryl Austin’s new system. So PFF analysts grading players without knowing the defensive call is one area, he says, “where they are a little bit short.”
But Cunningham has also seen the reverse: coaches grading favorite players more easily, or giving themselves too much credit for placing a player in the correct position when he makes a play. When one Lions player raised a stink about how Pro Football Focus graded him this season, Cunningham checked the PFF grades against his own analysis. “I wanted so much to tell him,” Cunningham says, “that they were right on the money.”
He views PFF analysts as young scouts, and he views their reports like those that come out of the BLESTO and National scouting organizations: more information that can help.
“I’ve been in the game a long time, and numbers have been really important to me,” Cunningham says. “There is so much room for statistical analysis and tape analysis. I had been thinking about investing my own money in something like this when I was done coaching. They beat me to the punch.”
* * *
A former minor league pitcher, Steve Palazzolo had always been interested in baseball’s sabermetrics and stumbled upon Pro Football Focus while looking to see if the same tools existed for football fans to better understand the game. (Jenny Vrentas/The MMQB and Palazzolo/Courtesy photo)
TEWKSBURY, Mass. — Steve Palazzolo spent eight seasons as a 6-foot-10 right-handed pitcher in the minor leagues, working his way up to Triple-A in the Giants and Mariners’ organizations. Now he analyzes football for a living.
The switch came in 2012. He had planned on playing one more season of independent league ball, as a closer for the Worcester Tornadoes, not far from where he grew up in Massachusetts. But the team was allowed only four veterans—those who had played six or more years of pro ball—on its roster, and a certain slugger took his spot.
“Jose Canseco ended my career,” Palazzolo says.
Palazzolo had always been interested in baseball’s sabermetrics, and he stumbled upon Pro Football Focus while looking to see if the same tools existed for football fans to better understand the game. After he signed on as full-time analyst, the major leagues came calling again, asking him to be a pitching coach or a scout. But he turned down those offers.
This season he’s had a primary role in developing Pro Football Focus’s newest project: analyzing college games. It’s a smart business decision that opens up a new potential client base, and it also gives their NFL teams another layer of data to help with their college scouting. The jump to analyzing all Division 1 FBS games—an additional 800+ games on top of the NFL schedule—is a huge undertaking. A third computer sits on Palazzolo’s desk in his townhouse, downloading college game film on one of three terabyte hard drives.
It’s a gold mine of data that can help teams sift through prospects the same way they do in free agency, but by an estimated factor of three or four. The grades can direct teams to a player’s best and worst games—for Jameis Winston, that would be against Boston College (best) and Louisville and Florida games (worst)—and isolate skill-sets such as throwing intermediate routes or getting the ball out while staring down pressure.
“One scouting director told us, ‘We’re looking to unlock gems and find the guy that all of a sudden came out of nowhere his senior year,’ ” Palazzolo says. “I can’t wait until we have enough data that we can go back and look at some of these sleepers. Some of these late-round guys, I’m going to go ahead and guess that if we graded them in college, they probably would have graded well.”
Pro Football Focus ranks Marcus Mariota as the highest-graded passer, and Winston in the middle of the pack. But the information they collect paints a more nuanced picture: Mariota faced nearly 50% less pressure than Winston in 2014, and the Oregon QB performed well, throwing three touchdowns, no interceptions and an average of 11.7 yards per attempt with defenders in his face. Winston, meanwhile, performed better against the blitz than a a traditional rush. Both quarterbacks struggled on rollouts. Mariota used play-action on 51% of his drop-backs, while Winston used it on just 16%. The high in the NFL this season was Alex Smith, who used play-action 31% of the time.
It’s that stat that illustrates why some analysts are doubting the Mariota hype: He did well with what was presented to him in college, but they aren’t sure all of the skills that made him a Heisman winner will translate well to the NFL.
That uncertainty is a gap Pro Football Focus hopes to bridge in the coming years.
* * *
Neil Hornsby abandoned his consultancy business in 2012 and took an 80% pay cut to grow Pro Football Focus. Last October, he moved from England to Cincinnati after forming a partnership with former NFL wideout and current NBC commentator Cris Collinsworth. (John DePetro and Jenny Vrentas/The MMQB)
CINCINNATI — On a train ride home from the University of Liverpool in 1984, Hornsby spent 85 pence on the U.K.’s Touchdown magazine (Dan Marino was on the cover) and became enthralled by the tables of stats that helped inform the highlights he loved to watch. He bought his collection of old NFL record books à la carte on EBay and spent most of the 1990s inputting data into a computer in his spare time, hoping to build a web encyclopedia of NFL records and facts. His hopes were dashed in 2003 when he stumbled upon such a site, ProFootballReference.com.
The idea for Pro Football Focus was born six months later, but the website drew just 80 page views a week (that’s not a typo) when it first launched. Hornsby was living in Luton, England, running a consultancy business that focused on something called “business process.” By 2008, his hobby was evolving into its own business. He and Ben Stockwell, a rabid NFL fan from Ware, England, finished a complete analysis of the ’08 season just a week before the ’09 season kicked off.
Hornsby started paying Stockwell—and later Elsayed and Sam Monson from Dublin, Ireland—out of his own pocket, making a push to analyze each week’s games before the next week kicked off. It was exhausting work, but it was during that ’09 season when Jon Berger, the Giants’ senior director of football information, sent Hornsby an out-of-the-blue message through the website’s rudimentary “contact us” form.
“This was in the days when the NFL didn’t give out player participation until after the season was over,” Hornsby says. “He said, ‘Can we have this info?’ I couldn’t believe it, because I thought every NFL team would have that info. That feeling kept us going.”
In 2012, Hornsby abandoned his consultancy business and took an 80% pay cut to focus on growing Pro Football Focus. He’s spent the last few years adding to his staff, traveling to NFL cities (including trips aboard The MMQB’s training camp RV, which Peter King explains in this week’s Monday Morning Quarterback column) to pitch his product to coaches and front offices. Collinsworth’s stake in the company is why Hornsby moved his wife, Claire, two sons, Alex and Ethan, and dogs Jess and Molly to Cincinnati, where Collinsworth played for the Bengals and still lives.
Former Bengals wide receiver and current NBC commentator Cris Collinsworth invested in Pro Football Focus last summer and helped expand the company’s reach to college football, which can help inform the draft process for its NFL clients. (Gene Lower and Richard Mackson for Sports Illustrated)
Before he invested, Collinsworth was simply looking for a way to get more information on all 32 NFL teams—and faster—to better inform NBC’s national television audience of roughly 21 million. He thought he was being punked when he first heard Hornsby’s thick British accent over the phone, but PFF passed all of his tests, including a side-by-side grading of a game from 2013. Their grades differed very little, and when they did, Collinsworth says, half the time they were right and he was wrong.
“What really impressed me is the fact that 13 NFL teams have contracted with Pro Football Focus for their data,” Collinsworth told The MMQB last August, after becoming a business partner. “I have been around the NFL for over 30 years, I know how hard it is to get behind the wall of those teams. And they’ve got 13 teams to trust their data. That’s huge.”
Collinsworth’s investment enabled the company to start analyzing college football. Down the road, who knows, maybe high schools will be next? But the core of Pro Football Focus’s business remains the NFL—and coming up with new ways to help their clients find an edge and save time. In his basement Hornsby is still scribbling on the whiteboard, trying to unlock the perfect formula to better rank pass rushers. He pulls up a document on a nearby tablet that PFF has made for its teams, showing how down and distance, the quarterback’s drop-back steps, and the position from which a player is rushing directly relate to his ability to get to the QB.
Using their new formulas to normalize data, for instance, a predominantly right-sided rusher like Robert Quinn moves up the rankings because it’s harder to get to the quarterback from that side. A predominantly left-sided rusher, like Charles Johnson, drops a few spots. Then there’s Watt, who rushed the quarterback 669 times in 2014, and no matter how you slice it, grades twice as high as any other defender in the league.
Statistics, of course, don’t provide all the answers, nor do they always give new answers, but they can offer a different perspective to help break down the game.
“Whether a team should go for it on fourth-and-1, there’s been some analysis of that,” Hornsby says. “But the truth of it is, what is the sample size of data for that game being played in Buffalo, at a particular temperature in December, with a right guard who has a dodgy hamstring and the halfback just broke up with his girlfriend the previous day? No amount of statistics can give you that answer. Only the coach can make that decision.
“But what we can do, we can say to a coach, ‘If you see Calvin Johnson lining up as the inside slot receiver on a play in Week 14, and in every other circumstance where he has lined up in that position he has run this route—would that be useful to you?’ ”
