To me, the most interesting thing about hockey is how we come to our conclusions about player evaluations. There are players like Ovechkin and Crosby who score tons more points than everyone else, year after year, and so we know these guys are special.
This works for the next tier of players too: You won't find too much discrepancy on anyone's opinion of most of the game's top offensive stars. But for players who come out of nowhere to score 30 goals, for guys out-performing their reputations and (I'd say mostly) for players whose main contributions are defensive, the world of the NHL - from paid analysts to fans - cannot agree on who is good and who isn't.
I've always been interested in how we come to these conclusions. The problem is as obvious as it is insurmountable: how does any one person watch enough hockey - and go beyond the highlights and high-impact plays - to actually know what they're talking about? Never mind that it's proven beyond a shadow of a doubt that the human brain is terrible at sorting through the amount of information committed to short-term memory that you'd take in watching just a single game.
You would think the obvious answer would be statistical analysis, but the world of sports is very anti-intellectual and understandingly sensitive about ceding it's expertise to the one facet of the grown-up world not run by nerds. I don't mean that pejoratively either, I think that's a sensible read of the situation.
Anyways, the evolution of player development has come such a long way that most of us are accepting of stats and are starting to argue about how to use those stats. One such metric is QoC or Quality of Competition. This is a measurement designed to show how players do against various skill levels of the opponents they face.
The inherent assumption is obvious: Players facing weak competition will do better than players facing stronger competition.
It is my experience from writing on Hockeybuzz and talking hockey with a wide assortment of people that, in general, the effect of quality of competition is greatly overblown. My hypotheses* is that QoC has no real bearing on what actually happens in an NHL game. Today, using some research that I have done, I will try to prove that that is true.
First, it must be acknowledged that if good players play bad players, they will dominate them. The reason QoC doesn't really matter is because this is never going to be a realistic game scenario - any opposing coach will skate his best defensive players against the other team's best offensive players (neutralizing any advantage) and so on to the point where it's going to be fairly rare that a very good player gets an extended amount of time against easy competition.
As Garik16 from Hockey-Graphs.com writes "In particular, the impact of quality of competition is often dramatically overstated. This is a little understandable – after all, it should matter a lot who is on the ice against a player at a given time."
Here he is acknowledging that the idea of QoC being mostly irrelevant is counter-intuitive. This is important because if you're going to convince anyone, it's a good idea not to call them stupid. It seems like quality of competition should matter more than it does, and the chart he includes illustrates is nicely. If you click the link I'll include at the bottom, you'll see that players of all qualities do better against bad competition.
The reason that this is irrelevant, however, is because it's almost impossible to get that kind of advantage in a real game. I do think maybe it's possible if a team were deep enough at one position (like, say if the Oilers keep all three of Nugent-Hopkins, Draisaitl and McDavid, and they all develop into their potential) but I would think in a salary cap league this would be a very rare situation.
As Eric Tulsky wrote on NHLNumbers.com "However, for the most widespread competition metrics, no player faces extremely strong or weak competition on average – the measured differences, while real and persistent, are small and scarcely worth correcting for."
This - if true - explains not only why QoC doesn't really matter, but why it is also used so often to explain away the differences between players who are over-performing to their reputation vs. players under-performing to theirs.
Garik16, in his article, says the following:
While it matters if a player is facing Sidney Crosby instead of John Scott at any given moment, the range of competition that a player faces over the course of a season is EXTREMELY SMALL. The gap between the players facing the hardest competition and those facing the weakest competition is the same as facing an average player at most like 4 shot attempts per 60. In other words, the guy with the toughest competition in the league will face an average opponent who is +2 corsi/60, while the guy facing the weakest will face an average opponent who is -2 corsi/60. And nearly all players won’t be in these extremes – most will be within -1 corsi/60 and +1 corsi/60. And as you might expect the gap between opponents who are +1 shot attempts per 60 and those -1 is practically nothing.
What he is saying here is that if you throw out the most extreme examples, the vast majority of players in the NHL are so close in what they contribute over time that it basically doesn't matter at all who you face.
Tulsky, writes that "A simple regression shows absolutely no relationship between a player’s Corsi or relative Corsi and his quality of competition, and multivariable analysis suggests competition has just a very small impact."
If you follow the links he provides, you'll see his work on this subject. This is a guy who has been hired by the Carolina Hurricanes and is considered one of the best hockey statisticians in the world. The point is, the information exists and has been vetted (by other writers who use his work) and can be considered reliable.
That QoC is not a reason to grade a player on a curve is pretty irrefutable: outside of extreme situations, QoC doesn't make enough of a difference to use it as a reason to excuse one players bad performance or to dismiss the great performance of another.
For a real life example, take Jake Gardiner and Morgan Rielly. Rielly gets top pairing minutes and has been the Leafs #1 defenseman this season. Jake Gardiner gets second pairing minutes. Yet, all stats based analysts will tell you that Gardiner is the superior player. Others argue, saying "sure, Gardiner has better stats, but he faces easier competition."
Yet, if we look at WaronIce.com and check their stats, we see something interesting: The average Corsi-For % of Gardiner's opponents is 50.22% and the average of Rielly's is 50.63% - virtually identical.
My theory on this is that hockey is a much more fluid game than our standard "first pairing" "second pairing" vocabulary makes it seem. Gardiner may be on the "second pairing" but he sees opponents from the oppositions entire team (and this makes sense if you think about it since top line forwards play up to twice as much as bottom line forwards, but non-first line players are on the ice for roughly two-thirds of the game. Not to mention that there are generally four offensive lines vs. three defensive lines).
In conclusion, it seems to be pretty cut and dried: Quality of Competition doesn't matter in long-term evaluations because its nearly impossible at the NHL level, given all the factors (last change, parity, opposing coaches, etc.) to actually get a favorable line-matchup for more than a tiny bit at a time. (For instance, if Crosby does end up on the ice against Toronto's fourth line, you can bet Babcock is screaming at them to get off the ice).
Because there is no (what Tulsky calls) "Extreme Usage" there can be no real effect in who plays against who, except in rare and unusual circumstances.
Thanks for reading.
Bibliography:
Tusky, E. (2012). "The Importance of Quality of Competition." Cited electronically from NHLnumbers.com Click Here to see this article.
Garik16. (2014). "Gauging the Relevance of Quality of Competition on a Player's Stats." Cited electronically from hockey-graphs.com. Click Here to see this article.
Waronice.com (2016) General Stats sited from website.
* This is not to imply that this is my original thought - it is, I believe, a widely accepted notion that I agree with and hope to convince you is the correct way to think about the topic.
Blogger • 
Join the Discussion:
»