Wanna blog? Start your own hockey blog with My HockeyBuzz. Register for free today!
 

The Old "Eye Test" and Why It's Ridiculous

September 23, 2014, 11:44 AM ET [52 Comments]
James Tanner
Blogger • RSSArchiveCONTACT



Well, it's not like it was on TV or anything, but the Coyotes did win last night in a meaningless 5-4 shootout victory over the Kings. See: I told you the Coyotes were a playoff team!

Well, maybe that doesn't prove anything, but there were some encouraging results last night nevertheless: Domi, Samuelson and Hanzal all had three points and the two rookies also scored in the shootout.

Good times.

What I want to talk about today is something that Michael Pachla, the Hockeybuzz Sabres Blogger, wrote about today.

In his article, he talked about how Sabres coach Ted Nolan doesn't believe in analytics because what he sees with his eyes is good enough and goals scored are all that matter.

Well, when everyone else had the same view, it didn't. It does today.

Why You Can't Trust Your Own Eyes

Ted Nolan is considered a good coach, but these quotes are ridiculous. I can only imagine the storm that would be brewing today if he coached the Leafs - you know that team that famously didn't care about analytics until the entire "advanced stats" community predicted their downfall and then they went from 3rd in the Conference in February to out of the playoffs?

I am not saying that one example proves anything, just that there was extreme backlash to the Leafs front office and their outdated way of doing things. It was enough that they went out and hired 28 year old "stats guru" Kyle Dubas as assistant GM, as well as hiring blogger Cam Charon and publicly incorporating analytics into their operation.

The correlation between winning and possession is pretty clear cut. Empirical evidence is not something you get to have an opinion on. You either believe it, or you are being willfully ignorant. As John Oliver said when speaking about the scientific consensus on Climate Change: Do owls exist? Are there hats?

This doesn't mean you have to care about advanced stats. If you are a casual fan, I can see how too many charts and talk about Corsi etc. can be boring. But if you are a coach or player in today's NHL, this is inexcusable. It's the equivalent of a sailor saying he doesn't need a compass because he "knows the way." Now, maybe he does, but that's besides the point - why would he risk not knowing even 1% of the time when there's a tool that is guaranteed to be accurate. (Please note that advanced stats are only predictive in the aggregate, and that anomalies will always occur.)

This brings me to "the eye test," something that is often repeated when someone doesn't have interest in learning anything new, using new information or changing their ways.

As anyone who has taken an Intro to Psych course knows, eye witness accounts are very unreliable. Take ten people who witness something and you will get ten different accounts. Mood, attention, context, adrenaline and fear are just some of the things that can affect how we view something.

Then there are anomalies. An anomaly is something that happens when all information suggests it is unlikely. The existence of anomalies is why anecdotal evidence is useless - what you see may not be generally indicative of other people's experience.

But that's not enough. The reason why no one should trust what they see exclusively is called Confirmation Bias. This is a fancy term that essentially just means that humans - every single one of them - have a tendency to ignore information that contradicts what they already believe and to over-value information that confirms it.

There are stats that could tell him for sure, but if Ted Nolan thinks, for example, that Cody Hodgson is "lazy" and Matt Moulson is a "gamer" then nothing he sees is ever going to change his mind. Our minds would be over loaded with information if we didn't do this - it's a biological imperative that we generalize information and fit it into patterns to make sense of it.

However, for the coach of a hockey team, it would serve him well to know the difference between what he thinks he knows and what the stats say. This doesn't mean you disregard what you see - it means you should be an active critical thinker and explore the difference between what you see and what you learn.

Furthermore, there is another good reason for paying attention to stats in hockey: it is a game of extreme speed and there are enough things happening all the time that you couldn't possibly see them all or remember them enough to incorporate them into any kind of useful deduction if you did. It is also a game, which means it is automatically a mixture of luck and skill. Without going too deep into theory, it is just common sense to understand that if you can determine which parts are luck and which parts skill, then you can make strategy to capitalize on this information.

Well, OK that isn't common sense, but it should be obvious to anyone who thinks about it for five seconds.

One of the biggest factors in hockey, and getting back to Nolan, are goals. I am paraphrasing here, but he said a common response to talk about analytics, which is: goals are the only analytic that matters. This would be laughable if it wasn't said by a profession millionaire (Nolan) and then written about my an amateur hundredaire (me).

The fact that goals are so luck based is the very reason advanced metrics are necessary to hockey. Since you can make great play after great play only to be stoned by a hot goalie, or conversely, you can play like garbage, give a poor effort, refuse to back-check and pot 2 goals by total fluke. In this example, it would be obvious to anyone that player B got lucky and player A unlucky - you wouldn't need anything more than your eyes, if you were paying attention.

The problem is, that over the course of a season, this could continue and you might not notice, or you might become blind to it, and when the season is on the line, you could put the wrong player on the ice. Now, over time, we tend to think luck balances out, and it does, eventually. 82 games is a long season, but even that is not really long enough to truly balance everything out - You could easily be a 20 goal scorer who got lucky enough to pot 40, or vise versa.

Analytics, are, at the very bottom, just a tool to account for luck and confirmation bias. If I was the GM of a team that had a coach who wasn't open to information that is demonstratively helpful, I would be looking for a new coach.
Join the Discussion: » 52 Comments » Post New Comment
More from James Tanner
» I am Just Curious If This Works
» NHL At Least Tries to do the Right Thing
» The NHL Cannot Remain Apolitical and Must Show Leadership
» Time for a New Coach to Go Along with the New G.M
» Coyotes Eliminated Following Severe Beating