Tom Brady: The Real NFL MVP?
Although Tom Brady lead the Patriots to the 2017 Super Bowl, data from the season shows that he did not deserve the NFL Most Valuable Player award.
by Abhi Gupta and Natasha Armbrust
May 11, 2018
Each NFL regular season, teams fight through a 16 game slate which pits them against their rivals across the league. The length of the season provides a large sample to truly distinguish the bad, good, and great teams. Yet it is always difficult to recognize who is most responsible for their team’s success. The NFL anoints a “Most Valuable Player” at the end of each season to honor the player who is perceived to have contributed the most to his team. In 2017, the New England Patriots finished with a league best 13-3 record, and their Quarterback, Tom Brady, was awarded the MVP honor. This award is voted on by members of the media. However, these members are subject to biases and each member’s vote may be influenced by personal opinion, group think or player exposure due to the market they play in (consider a player in a bigtime city like New York vs in Kansas City). Throughout the season, many players emerged as candidates for the MVP as their teams entered hot streaks, emphasizing the fact that the team might be the more responsible for the award versus the player themselves. Therefore, we pose the question: Did Tom Brady deserve the 2017 NFL MVP?
To answer this question, we defined metrics that aim to analyze a player’s contribution to his team. We use our metrics to statistically break down a set of quarterbacks who we consider MVP candidates from the 2017 NFL Season. It is important to note that we selected metrics for analysis before looking at the dataset to reduce any biases we may exhibit. Furthermore, we make this analysis available to you, the reader, through interactive graphs that allow you to perform your own analysis. In each section, we will explain our methodology, including how we processed the data. Finally, for those who are not as familiar with football but are fans of numerical arguments, we will provide notes like this to supplement your knowledge of football as you read.
MVP Candidates
Our set of NFL MVP candidates only includes quarterbacks (QBs). We make this decision because QBs inherently have more value in today’s NFL as passing has taken over offensive play calling . Quarterbacks touch the ball on every play and make the most important decisions that influence his team’s success, such as audibles , checking the defense and modifying the play , and getting the ball from the center to the running back or receiver . Furthermore, 22.5 of the last 30 MVP Awards have gone to QBs (0.5 being attributed to Co-MVPs). Although fans of LA Rams Running Back Todd Gurley may take issue with our exclusion of running backs , in order to complete a fair assessment of Brady, we must compare him using metrics relevant to Brady, and as such, we can only compare him against QBs. Our NFL MVP candidate set includes 6 QBs who were selected to the Pro Bowl this past season.
- Tom Brady
New England Patriots NE - Carson Wentz
Philadelphia Eagles PHI - Ben Roethlisberger
Pittsburgh Steelers PIT - Philip Rivers
Los Angeles Chargers LAC - Russell Wilson
Seattle Seahawks SEA - Drew Brees
New Orleans Saints NO
Metrics of Analysis
We will compare Brady to each of the other 5 QBs and gauge his value on 3 metrics:
1. Passing Effectiveness
This dimension analyzes QB accuracy by completion percentage. We break down effectiveness into accuracy by pass distance, down , and a game’s point spread. Each of these graphs highlight a QB’s ability to perform based on pass difficulty (distance), short-term difficulty (down), and long-term pressure (point spread).
2. Efficiency
We gauge a QB’s efficiency based on a series of drive metrics, which includes time per drive , yards per drive , points per drive , and plays per drive . This evaluates a QB on the quality of the drives he plays in by looking at scoring (points per drive), speed (time per drive), length (yards per drive), and rate of progress (plays per drive).
3. Impact
A player’s value can also be quantified by looking at his impact on his team’s win probability. By comparing offense and defense average win probability (WP), the change from defense WP to offense WP may indicate how much that given player impacts their team.
Methodology
To compute our metrics for analysis, we rely on the nflscrapR-data dataset, which contains a play by play documentation across the 2017 NFL Season. The author originally scraped all the data from the NFL API and joined many datasets together to form this one. We filter the quarterbacks we compare in our visualizations by their total number of passes across the 2017 NFL Season. A QB is included if they have attempted at least 100 passes, as this provides a reasonable body of work at around 4 games of playing time. Any quarterback who has not attempted this many passes is not included in our graphs. This narrowed our set of valid passers to 45 QBs, each of which can be seen across every visualization. In addition, for each visualization we compute the NFL Average using the 45 valid QBs. Our work for this project can be found here, which includes our data preprocessing and graph generation.
Metric 1: Quarterback Accuracy by Pass Distance
We begin by analyzing the most obvious metric to any viewer of the game: How accurate is a quarterback the further they throw the ball downfield? To break this metric down, we filtered each QB’s passes by pass distance, measured as the vertical distance travelled downfield from the point of throw to where it was caught or hit the ground. Then, we sorted passes into bins: passes that were less than 0 yards downfield (such as a pitch or a screen play) , 0-10, 10-20, 20-30, 30-40, and 40+ yards downfield. It is important to note that the majority of throws are under 30 yards, and thus all buckets under 30 yards will have a higher number of passes. To make this distribution visible, we place a “total pass attempts” datapoint on each completion percentage bar so it is clear how many passes were attempted in the computation of the completion percentage. Consider the example of a QB with 100% pass completion percentage and 2 total attempts and a QB with 90% completion percentage with 20 total attempts. The comparison between these two QBs should be taken with reservations as small sample sizes are subject to extremes.
As with all of the graphs in our analysis, you can select the quarterbacks whom you are interested in analyzing using the dropdown key on the right of the graphic.
For all MVP candidates, we list completion percentages in the table below. We compute an “average ranking” metric to gauge how a quarterback rates across all categories against all other valid passers. We include Average Rank to aid our understanding of a QB’s aggregate ranking, as a 2nd place rank in all categories is a much better performance than a single 1st place rank and five 8th place ranks. The QB with the top completion percentage amongst MVP candidates for each category is bolded.
| Quarterback | Team | < 0 | 0-10 | 10-20 | 20-30 | 30-40 | 40+ | Avg Ranking |
|---|---|---|---|---|---|---|---|---|
| Drew Brees | NO | 87.4% | 73.7% | 60.5% | 56.8% | 46.7% | 50.0% | 6.2 (1st) |
| Tom Brady | NE | 90.0% | 69.0% | 62.8% | 50.0% | 30.8% | 26.3% | 10.5 (2nd) |
| Russell Wilson | SEA | 93.1% | 66.7% | 51.2% | 40.9% | 33.3% | 33.3% | 16.5 (9th) |
| Ben Roethlisberger | PIT | 83.1% | 73.0% | 57.5% | 31.8% | 30.0% | 18.8% | 19.2 (14th) |
| Carson Wentz | PHI | 73.5% | 65.9% | 58.6% | 51.9% | 27.8% | 17.6% | 21.2 (18th) |
| Philip Rivers | LAC | 73.2% | 68.2% | 56.8% | 51.5% | 21.4% | 29.4% | 21.8 (21st) |
| NFL Average | NFL | 82.8% | 67.0% | 53.4% | 36.6% | 29.9% | 26.3% | N/A |
For passes less than 0 yards in distance, completion percentages are high across the board, with Russell Wilson of the Seahawks performing the best at 93.1%. Passes in this bucket aren’t necessarily difficult, as screen and pitch plays are meant to be high completion plays by design. If anything, this bucket will only help identify QBs who are unable to complete relatively easy passes. Both Carson Wentz and Philip Rivers have low completion percentages, falling below the NFL Average by almost 10%.
Short to intermediate range passes (0-10 and 10-20) saw Drew Brees and Tom Brady respectively take 1st place. These categories are important considering the volume of passes that are thrown in this range. As can be seen with most QBs on the chart, the highest number of passes that are thrown are between 0-10 yards, with the second most passes thrown from 10-20 yards. These passes aren’t as easy as passes behind the QB (< 0) as defenders clog up throwing lanes and force the QB to anticipate defenders' actions.
However, the most difficult passes occur 20+ yards downfield as each pass must have distance and accuracy, especially considering receivers and defenders alike have great catching skills . A bad throw can lead to an interception or a big hit on the receiver. As seen in the graph, completion percentages significantly drop in these buckets due to their difficulty, yet Drew Brees easily takes 1st place in the latter 3 categories, beating the NFL average by more than 20% in some spots. Drew Brees is clearly an accurate QB, something that NFL fans have been known for the better part of 2 decades; however this shows that Brees is accurate not only in the short game, but can complete throws downfield with accuracy rivaled by few in the league. On aggregate, Brees finishes 1st in Average Rank with 6.17 with Tom Brady giving close chase at 2nd place with a score of 10.5.
Metric 2: Quarterback Accuracy by Passing Down
Completion percentage per down helps us understand how a QB performs in a micro-level pressure situation. Specifically, a team will face 1st through 4th down many times throughout a game. QBs who are more accurate on 1st or 2nd down will likely be advancing the ball forward more often, facing fewer 3rd and 4th downs . Alternatively, a QB who is more accurate on 3rd down or 4th down, in comparison to earlier downs, performs better during a high pressure play. If the QB does not complete a pass in these high pressure situations, they may not lose the game, but they will either not score any points on that drive, or will settle for fewer points than a touchdown . Note that sample size again matters, and a QB who attempts and completes 1 pass on 4th down would have 100% completion accuracy while a QB who attempts 2 passes but completes only 1 has a 50% accuracy. We do not want to reward QBs for having low attempts, and thus we enforce that a QB must have attempted at least 3 passes for each down to appear on the graph, otherwise we do not place a point on the graph for that down.
As with the 1st graph, multiple QBs can be selected and compared using the select key on the right of the graph. The QB with the top completion percentage amongst MVP candidates for each category is bolded.
| Quarterback | Team | 1st | 2nd | 3rd | 4th | Avg Ranking |
|---|---|---|---|---|---|---|
| Drew Brees | NO | 74.0% | 73.9% | 67.8% | 80.0% | 3.3 (1st) |
| Tom Brady | NE | 67.8% | 69.4% | 58.7% | 80.0% | 10.3 (3rd) |
| Ben Roethlisberger | PIT | 70.4% | 59.7% | 61.1% | N/A | 13.0 (5th) |
| Philip Rivers | LAC | 64.5% | 62.9% | 61.0% | 60.0% | 17.3 (11th) |
| Russell Wilson | SEA | 60.2% | 67.8% | 56.7% | 50.0% | 21.8 (20th) |
| Carson Wentz | PHI | 59.9% | 56.3% | 66.1% | 55.6% | 23.5 (23rd) |
| NFL Average | NFL | 64.0% | 63.8% | 58.4% | 49.4% | N/A |
This metric shows that there is a large spread for accuracy across downs. Tom Brady is efficient on 1st and 2nd down, but performs at the NFL Average level on 3rd down. On 4th down, his completion percentage jumps to 80%, tied for first with Drew Brees. Ben Roethlisberger only had 1 attempt on 4th down this season, and thus we did not compute a percentage for him in this graph. Carson Wentz, the QB of the champion Philadelphia Eagles, was constantly claimed to be the NFL MVP throughout the season, yet here we can see that his accuracy is subpar on 1st and 2nd down, falling to as much as 6% below the NFL average. He does have better than average accuracy on 3rd and 4th down, however, which alludes to the fact that he is accurate when it matters. The most dominant QB according to this metric, however, is Drew Brees. His completion percentage is the highest for every down, performing 10% more accurate than average on 1st down, 10.1% more accurate on 2nd down, 9.4% more accurate on 3rd down, and a whopping 30.6% more accurate on 4th down, tied only by Tom Brady. Brees has an average ranking of 3.25 across all downs and all QBs, which places him at #1. Tom Brady again comes in 2nd with above average performances in each metric and only truly shining on 4th down.
Metric 3: Quarterback Accuracy by Point Spread
Metric 3 provides a macro-scale perspective on a QB’s performance. In Metric 2, we looked at how a QB performs based on a small scale situation in terms of the down of the current play. But how does a QB perform when the game is close? If the game is within 1 possession of tying or taking the lead, downs matter less because the game may be in the latter portion of the 4th quarter and urgency is key. Is a QB more accurate when the game is out of hand, or when the game is close and the QB cannot afford to make a mistake? We analyze a quarterback based on the score point spread at the time of a pass. Specifically, we look at a QB's team score less the defense team score (how much the QBs team is ahead or behind by) and analyze the completion percentage for each point spread up to 21 points. A positive point spread indicates that the QB’s team is ahead, and a negative point spread indicates they are behind. We do not include a data point if a QB has less than 3 passes for a bucket for the same reasons as Graph 2.
In order to use this graph, you can select QBs using the select picker below the graph, and adjust the scale of the graph by the point spread selector below it. For instance, to analyze close games, you may select the +/- 7 point option, which will change the graph to only include passes with point spreads within this range. The QB with the top completion percentage amongst MVP candidates for each category is bolded.
Of course, this breakdown is a bit difficult to digest in its entirety. To simplify our comparison, we compute the QB accuracy by point spread bins, considering games that fall within specific ranges: tie games (0 pts), 1 possession games (1-7 pts), 2 possession games (8-14 pts), 3 possession games (15-21 pts), and blowouts (22 pts+). We again compute an average ranking metric. Since a QB’s accuracy is much more important in close or tied games than games where his team is up or down by more than two possessions, we only compute the average ranking by looking at games within reach, which we define as the first three buckets (point spreads between 0-14 points). The numbers are listed in the table below.
| Quarterback | Team | 0 (Tied) | 1-7 | 8-14 | 15-21 | 22+ | Avg Ranking |
|---|---|---|---|---|---|---|---|
| Drew Brees | NO | 71.4% | 73.2% | 68.8% | 75.5% | 86.7% | 2.0 (1st) |
| Tom Brady | NE | 72.5% | 65.9% | 62.6% | 67.1% | 54.5% | 7.7 (3rd) |
| Ben Roethlisberger | PIT | 71.9% | 63.1% | 61.9% | 61.9% | N/A | 12.7 (8th) |
| Philip Rivers | LAC | 61.2% | 62.7% | 66.1% | 60.0% | 58.8% | 17.7 (17th) |
| Carson Wentz | PHI | 62.5% | 63.3% | 56.6% | 50.0% | 55.6% | 23.3 (22nd) |
| Russell Wilson | SEA | 53.3% | 61.1% | 66.9% | 92.9% | 47.1% | 23.6 (23rd) |
| NFL Average | NFL | 62.3% | 61.9% | 61.0% | 63.1% | 63.0% | N/A |
Interestingly, this metric provides many valuable insights on each QB’s performances. First, Drew Brees is very effective in close games. His accuracy of 73.2% during single possession games and 68.8% during two possession games leads the league. In tied games, Brees also performs well, at 9.1% above the NFL average. Even in blowouts, Brees performs at an absurd 86.7% accuracy, placing him at the top of our MVP list for this metric. Tom Brady is the most accurate in tied games at 72.5%, but regresses in most other buckets. He still plays at a 2nd place rate for games within one possession, but plays at roughly an average level for games within 2 possessions.
More telling, this metric shows us that Brees and Brady are in their own league compared to other “top” QBs in the league. Rivers, Wentz, and Wilson all play at average or below average levels for nearly every metric, with rankings far below MVP standards. Looking at Wentz, his performance is average for games tied or within one possession, but falls below league average for all other situations. Russell Wilson performs even more horridly, playing at a -9% completion percentage to league average for tied games, not a performance you’d want from an MVP candidate. Wilson provides an interesting example where his performance skyrockets for games between 15 and 21 points, as he has a near 100% completion rate, but is otherwise below average.
Metric 4: Quarterback Effectiveness by Drive
Teams can also be evaluated by their success on drives. A team that is able to score more points quickly outperforms a team that has long drives while scoring less points. As such, we look to evaluate QBs based on the quality of the drives they lead. In this graph we highlight the time per drive, yards per drive, points per drive, and plays per drive. Each of these metrics can help us see how quickly a team scores (time), how much progress a team makes on each offensive possession (yards), the impact the team has on the score (points), and the number of attempts a team takes on each possession (plays).
To compute the average ranking for this metric, we only take into account yards per drive and points per drive. It is not inherently obvious whether more or less time or plays are preferable , and thus we only use objective scales to rank by (more is clearly better).
The graph can be modified by axis, allowing for the selection of a metric to be placed on either the x or y axis. All QB’s are shown, however selecting a QB using the select picker on the right will highlight that QB for easier viewing. Hovering over the corresponding QB’s datapoint will provide further information on the stats used to compute the statistic. The QB that has the largest metric amongst MVP candidates for each category is bolded.
| Quarterback | Team | Time/Drive | Yards/Drive | Points/Drive | Plays/Drive | Avg Ranking |
|---|---|---|---|---|---|---|
| Tom Brady | NE | 2:52 | 45 | 1.8 | 8.9 | 3.5 (1st) |
| Drew Brees | NO | 3:01 | 45 | 1.5 | 8.3 | 5.0 (2nd) |
| Ben Roethlisberger | PIT | 3:03 | 44 | 1.5 | 8.7 | 6.5 (Tied-4th) |
| Philip Rivers | LAC | 2:49 | 43 | 1.4 | 8.3 | 8.5 (Tied-7th) |
| Carson Wentz | PHI | 3:01 | 40 | 2.0 | 8.4 | 9.5 (Tied-9th) |
| Russell Wilson | SEA | 2:28 | 40 | 1.3 | 7.8 | 17.5 (Tied-16th) | NFL Average | NFL | 2:52 | 38 | 1.2 | 8.1 | N/A |
Our MVP candidates are spread widely in this metric. As seen, Brady and Brees lead in yards per drive with 45, well above the NFL Average at 38. Carson Wentz leads the pack with 2 points per drive, but falls with only 40 yards per drive. Russell Wilson struggles with 40 yards per drive and 1.3 points per drive, the lowest amongst MVP Candidates. The average ranking metric is very telling, highlighting Brady as the far and away leader at 3.5, 1st overall. Brees is the next best with a score of 5, placing 2nd. Both Brady and Brees are the highest performers and are far and above the pack. The rankings get close after the top 2, as 4 of our 6 candidates can be found within a 5 point range (5 - 10). Furthermore, Wilson performs horrendously, placing 16th overall in these metrics, a full 7 ranks below the next best QB.
Metric 5: Win Probability Change by Quarterback
In our final metric, we look at how a QB impacts his team’s win probability. Scraped from the NFL API, nflscrapR has created models which produce win probabilities before and after each play . We use this win probability to gauge the impact of each QB on his team.
The graph highlights the average win probability of a team when they are on defense (left column) versus the win probability of the team when the QB is on the field (right column). A line connects these two points to form a slope graph, indicating whether the win probability goes up or down due to the presence of a QB. Multiple QBs can be selected using the select picker.
| Quarterback | Team | % Change |
|---|---|---|
| Carson Wentz | PHI | 1.1% |
| Russell Wilson | SEA | -0.4% |
| Tom Brady | NE | -0.6% |
| Drew Brees | NO | -1.3% |
| Ben Roethlisberger | PIT | -1.8% |
| Philip Rivers | LAC | -2.3% |
| NFL Average | NFL | -1.5% |
Our analysis finds that this metric provides insights only on the extreme performers, showing QBs who severely hurt or help their team’s chance of winning. Particularly, the NFL average for win probability using the nflscrapR model sits at -1.5%, which at first glance would indicate that the average quarterback is a detriment to their team. The models used to calculate win probability are based on historical data and use various metrics (such as time left, timeouts remaining, current score, field position, ...) to calculate win probability for each play. As such, the models are subject to statistical noise and error. We know most of the QBs on this list are highly regarded in the NFL and in general do provide a lift to their team. In fact, the only QB with a positive win percentage difference is Carson Wentz, and even he has only a 1.1% increase. We believe this is due to the fact that a good quarterback will always score points on his respective offensive drives. The more a team scores, the more the team has a chance to win. However, we calculate average win probability at the start of every play, and as such, any points scored by the offense will impact the win probability for the plays AFTER the team has scored. Therefore, a good quarterback’s defense is likely to have a higher win probability than when the QB is on the field.
We do find for the extreme cases we can find meaning in the change in win probability. The QB with the highest positive impact on his team is Jimmy Garoppolo of San Francisco , improving his team’s win probability by 17.7%. It was known that Jimmy G was a strong influence on his new team’s success at the end of the season, and this metric highlights that. The QB with the largest negative impact on his team was Bryce Petty of the New York Jets , decreasing his team’s chances by a cringeworthy 18.9%. In the same manner as Jimmy G, Bryce Petty was known to be a horrendous quarterback; our graph statistic cements this sentiment and provides justification for the Jets recent decision to cut the signal caller .
Once again, this metric provides insight only to the deep extremes of the NFL, where poor performing teams are saved or average teams are severely hurt by their QBs. Our MVP candidates are difficult to analyze using this metric, and based on a simple (and somewhat irrelevant) comparison Carson Wentz ranks highest of the candidates.
Conclusion
Our analysis across all metrics shows that Tom Brady consistently performed well throughout the season. He was within the top 3 of many metrics, and placed 1st in different graphs. However, Drew Brees was the best performing QB of the set of MVP candidates. He performed 1st in accuracy by distance, down and point spread, and 2nd in drive efficiency. Brady and Brees were clearly in a different league from the rest of the NFL, performing at a distinctly higher level than the rest of the QBs yet neck and neck with one another.
We note that of our MVP candidates, Carson Wentz did not come close to performing at the NFL MVP caliber level. Although he helped the Eagles go from a 7-9 team in 2016 to a 13-3 team in 2017, a large change in win percentage was not solely due to him, as seen by Philadelphia’s Super Bowl win despite his absence. Russell Wilson also performed average, yielding many double digit rankings.
When making a purely statistical argument, it is clear that Drew Brees was the true NFL MVP of the 2017 season. However, as mentioned before, the MVP is not awarded solely based on data, and we hypothesize some more political reasons why Drew Brees did not win the award: The Patriots were the best team in their conference, if not the NFL, and as such Brady was accredited with more influence in their success. Furthermore, Brees has always been known for his passing prowess and his absurd consistency in setting passing records. However, this season saw his workload scale down with the rise of his running backs, Alvin Kamara and Mark Ingram . NFL viewers are used to seeing Brees do more than he did this past season, and as such it appeared that the team relied on him less when in fact he may have been even more effective at his lower workload.
The fact that Drew Brees’ dominance was left unnoticed brings into question the current structure of the MVP voting system. Instead of choosing the players who perform the best, the award tends to gravitate towards the players who are “newest” to the scene, considering the candidacy of Carson Wentz who performed below several QBs according to our metrics. By allowing media members to vote for the MVP award, we allow for human biases to interfere with purely numerical arguments for or against a player. Sentiment does not win arguments, but numbers do. We question the meaning of NFL MVP in its current nature. With little to no statistical arguments to back up the awarding of MVP, it is hard to take the award seriously. In a game where we can actually reduce a player's performance to statistics, shouldn't we use their statistics to compute rather than decide the winner? A purely statistical decision does not account for nuances in team dynamics, but a purely human judgement decision does not account for true performance of a candidate. And it is for this reason we address the errors within the current NFL MVP voting and urge the integratation of a numeric arguement to reduce human biases. It is only through integration with numbers that we can truly award the Most Valuable Player the MVP award. ■