Those who argue with Advanced stats usually do so because of the eye test. The general vibe given off in these conversations is that a trained eye is better at evaluation because it can pick up nuance that numbers don't capture.
Here's why I find "the eye test" extremely unreliable, in ove video about a card trick:
I've seen people, not just Wild fans, use this picture to say the play leading to the game 5 tieing goal should have been be offsides:
Just for clarification, let's check the NHL rulebook for what constitutes offsides: (emphasis mine)
The position of the player’s skates and not that of his stick shall be the determining factor in all instances in deciding an off-side. A player is off-side when both skates are completely over the leadingedge of the blue line involved in the play.
This is a pretty contraversial opinion in Avs land, so I better have a pretty good reason and logic to back it up. And I do.
First of all, It's important to note how important Puck Possession (measured with either Corsi or Fenwick) is to a team's success. I don't want to go into a long schpiel on it, but Eric T, one of the best analytics minds in hockey right now, has it somewhere between 3-5 times more important than shot quality or goaltending. But it's not just analytics guys, take recent Stanley Cup Winning Coach Daryl Sutter's words for it, from the Blog Nichols on Hockey (emphasis mine):
Via The Edmonton Journal, Sutter says the Kings’ seemingly defend-first game is a “misconception. The big thing in today’s game is you have to be able to forecheck and backcheck, and you have to have the puck. You can’t give the puck up. We don’t play in our zone, so there’s not much defending."
Also: “I’ve coached in three decades now and this stuff where they said Marian had to play in Jacques’s system is a bunch of bull-crap. The game’s changed. They think there’s defending in today’s game. Nah, it’s how much you have the puck. Teams that play around in their own zone think they’re defending but they’re generally getting scored on or taking face-offs and they need a goalie to stand on his head if that’s the way they play."
In other words: to be successful a team needs to have the puck or get brilliant goaltending. While coaching can affect goaltending (And I think Roy deserves a ton of praise for his work turning Varlamov into a reliable goaltender, potentially a Vezina winning one) The real place a coach can have an impact is on puck possession, and shot quality (which, again, is not as important, but not necessarily negligible). So, in my eyes a good team will have a good Corsi when the score is close
The Avs Corsi-close, according to the indispensible ExtraSkater.com: The Avs are 25th, with a Corsi of 47.7%, which is pretty awful. Roy's coaching isn't doing very good for the one thing that he has the most control over.
Now, this is where the argument comes in that his system forces teams to shoot from the outside and those are low quality shots. c6hor8 of Mile High Hockey makes the point pretty eloquently in this comment (i'm excerpting here. It's a good analysis of Xs and Os so I'd recommend heading over and reading the whole thing):
Also, Roy employs something I like to called “controlled possession.” Possession works in two way: 1) having the puck on your stick or 2) having the puck on someone else’s stick but controlling where he goes. The latter is one of the hallmarks of man-to-man coverage. It prevents passes, it prevents space, and it allows the defending player to control (if done well) the puck carrier. Clearly a better defense will take possession more and use the 1st type more than the 2nd. Over time man-to-man coverage is high risk/high reward so issues will arise and more so than in a standard zone coverage.
In fact, in the 32 shots the Avs usually give up I would estimate 25 of those are so routine that most starting goalies save them. It’s the other 7 that need to be stopped and toned down. Those 7 or so are the ones they score on, the ones that look dangerous, and the ones a better defense will prevent or make like the other 25. Varly has played great because he has made those 25 or so saves AND made the other 7 when asked. And those have been great saves. This system is the nemesis of Corsi because it can’t be applied accurately. It looks are something in aggregate when that only tells part of the story. I would argue that the first 2 months of the season looked more like relying on Varly and recently the Avs have discovered their controlled possession system and use it to gain their own possession, move the puck up the ice, and let their impressive offense flourish.
According to this explination, and other similar ones, if what he's saying is correct, The Avs would force a higher percentage of lower quality shots than other teams. This has been an argument for every team whose had unsustainably good goaltending covering bad defense since Corsi emerged. Luckily, recently, more analytics tools have emerged to test this.
Using Ninjagreg's new site I was able to compile a data base for every team, through March 7th, of where the shots they were giving up are and find out which teams give up a larger % of their shots from close in. If c6hor8 is right, The Avs will be giving up more shots from the perimeters and less shots from the slot and right in front of the net.
This is Proportion of Shots Against within 30ft and 20 ft (by percentage) for away data (to remove any home scoring effects) Here's the full data set in a Google Doc and I'll post a truncated version here for formatting sake:
|< 20 ft %||< 30 ft%|
|New York Rangers||37.67%||52.52%|
|New York Islanders||39.58%||58.08%|
In the spreadsheet there's the Average and St. Deviation, which gives a more complete set of data. If you take the standard deviation into effect (something we in the analytics community don't do enough) here's what it means about the Avs:
• There's a 50% chance the Avs give up a lot more high quality shots than the opposing team
• There's a ~68% chance the Avs are no better than slightly below average at restricting shot quality
• There's a ~98% chance the Avs are no better than above average at restricting quality shots.
I'm not trying to argue the Xs and Os of c6hor8's system, but the effectiveness that is being attributed to it is extremely unlikely, given the data.
I think the data makes me conclude a few things: Roy's system is more likely to rely on a goaltender making tough saves, which makes Varlamov's season all the more exemplary (and Roy deserves some coaching Kudos for Varly's turnaround). But it also makes me think Roy's system isn't very effective (or, more likely, the players who run Roy's system aren't very effective). You can call it random chance or great goaltending, or likely both, but it's the combination of the two that is fueling the Avs standings success.
• Todd McLellan should tentetively be the Adams'winner, with a sparkling Corsi AND a sparkling Shot quality against, the sharks are going to be really tough to play against in the playoffs.
• Cooper from TBL is the other Adams candidate and his system, unlike Roy's, is likely helping fuel Bishop's success this season. With him losing Stamkos I'd consider changing my non-existant vote.
Even with all of this here, it's still likely that Corsi-close is, and goaltending likely is, a bigger driver of W/L/Points than the data above. Edmonton has pretty good numbers, but are so over matched in Corsi and top it off with miserable goaltending that any advantage they gain here is lost multiple times over. Toronto has good numbers here, and good goaltending, but is a miserable Corsi team. They're still a minus in goal differential on the season (and down 5-1 tonight as I write this).
Unfortunately, I can't pull out who the best at creating chances is yet, it's entirely possible Roy's system gives up chances in favor of creating scoring chances at the other end, but since the website I pull from updates regularly I have to record all the data on the same day, which is an effort. The next analysis will be doing just that, but for now: Roy's sytem gives up a ton of shots and gives up quality shots disproportionally when compared to other teams.
When the injured Jan Hejda was replaced on the top pair with Nate Guenin I was initially boggled at how very smart hockey people, such as Patrick Roy and his staff, can be so wrong about a player or players as he continues to undervalue the contributions of Tyson Barrie. I was listening to my favorite podcast on the internet, You are not so Smart, when the host talked about Argumentum ad Antiquitatem, or Appeal to Tradition a logical fallacy. Basically something gets valued as true because it's always been valued.
In hockey there seems to be no more greater overvalued ability than toughness, physicality and size. This isn't bad coaches or stupid people, this is an industry wide overvaluation of one attribute an instituitional bias. Nate Guenin, Ryan O'Byrne, Greg Zanon, Scott Hannan are all recent Avs whom coaching staffs seem to have overvalued because they are tough defensive defensemen. It's certainly not just the Avs - Ray Shero and the Pittsburgh Penguins, a very smart organization, just recently traded a second round pick for a washed up Douglas Murray because he's big. Hal Gill had a career that extended well beyond his usefulness because he's big.
Then there's a player like Tom Gilbert, now in Florida. A smooth puck moving fluid defenseman who doesn't hit very much and was never very good at moving bodies. The guy has been a very effective NHL defenseman for years, but he wasn't signed until extremely late for the bargain of $900k. He's been the best bargain of the season. It's not just Gilbert, Tyson Barrie is of the same mold and continues to be underappreciated, both by coaching staff's (Roy's and Sacco's) and by smart fans.
Maybe, in a bygone era, toughness and gritty play was as important as many hockey people think it is now. There was an era where the skating wasn't as good and maybe the more half-court style play led to an increased importance of that toughness. But in today's game, it's less important than being able to move the puck effectively especially in your own zone, a place where Barrie excels and Guenin struggles.Here's an example of how Guenin's play gets over-valued and Barrie's gets undervalued, both from the Calgary game last week:
Guenin goes back into his own zone and is pressured by a CGY player. Instead of passing to Erik Johnson, who is open a short distance away, Guenin skates the puck towards him allowing the Calgary player to cover both players effectively, eliminating his best passing option. Guenin also eliminates his own pass to the wing on the far boards by putting the net between him and the wing. He now has no choice but to try a low percentage chip off the boards. Calgary keeps the chip in the zone and gets at least 4 shots at net including a good scoring chance. You could watch that shift and think Guenin had a good shift. He had a hit, and he effectively tied up his man in front of the net. He blocked a shot. But he wouldn't have had to do any of it with a more effective pass out of his end. (He screwed up a pass the next game against Vancouver which directly led to a back breaking goal)
To me, it appeared as if Guenin didn't make a decision on what to do with the puck until after he got it.
Later in the game Barrie was headed back into his own zone, being pressured heavily on the forecheck. Before getting the puck he looked his options, Holden on his right who had a CGY man coming to try and force a Holden turnover, but he also had Landeskog (maybe O'Reilly) on his left slightly up the boards. He gives a small fake to Holden (the traditional classic play) and chips the puck over the forecheckers stick to Landy, who then skates it harmlessly out of the zone. No blocked shots, no checks, no shots against and no scoring chances. A much more effective play
Michael Lewis' baseball book, Moneyball, is often mistakingly referred too as a book about statistics. It's not. It's a book about identifying and exploiting institutional biases in a professional sport (which are also called market inefficiencies). Statistics was nothing more than the tool he used to identify those biases. Those biases that infest us, all of us, are hard to identify. Physicality is still an important trait for a hockey player, but the importance of it is a bit antiquated. Hockey is slowly shaking that bias, but the Argumentum ad Antiquitatem is still much too common.
Adrian Dater reports, that Stephane Roy's, Patrick Roy's brother, has said that Roy (Patrick, not Stephane) is going to be the next coach of the Colorado Avalanche. The Avs, for their part, are not commenting on it because the organization looked deep into Medusa's eyes years ago rendering them incapable of commenting on anything.
I have mixed feelings on whether or not Patrick Roy will be a good head coach. I have mixed feelings about legends coming back to coach the team they achieved greatness with. I just have no idea what kind of coach Roy will be. The good news is that I think coaches generally get too much credit and too much blame for their teams' success and failure so I don't think it's as big of a deal (on the ice) as some people.
Where I have the most concerns is that I haven't heard of them even interviewing another candidate yet. It's possible they have (they are notoriously secretive) but Dallas Eakins (Toronto Marlies) & Tippett (still Phoenix Coyotes), at least, are both unavailable for interview, so the Avs clearly have not even vetted them yet (Eakins' Marlies were eliminated tonight). Lindy Ruff is, but a high profile coach like that would have likely garnered a report somewhere. Same with a guy like Guy Carbonneau.
Now maybe Roy is the first guy that has been brought in to interview, and is the early favorite to boot, and this is the first step in an truly exhaustive coaching search. In which case this is all just good fun rostercoacherbation. I hope this is the case, and there are leaks about other interviews. If the job goes to roy after that, then that's great news.
But, as of right now, it has a whiff of the Avs business as usual: not doing their due diligence or doing a full search for a coach. In 2008 Tony Granato was hired after no coaching search what so ever. Here's what I wrote about it then:
In fact I have nothing against Granato (we'll get to that in a minute), it was the way in which the search played out that really bothers me. The Avs didn't even bring Dineen, McLellan, or Burns in for a courtesy interview. Is Granato, mastermind of the kitten-killing powerplay, really that impressive that he deserves to be hired before even talking to qualified candidates? He may end up being a fine coach, but then again Kineen, McLellan may too, and waiting for an interview with either candidate certainly certainly would have done no harm. Both are promising young coaches and to not even give them the light of day is really unacceptable.
Then Granato flamed out spectacularly, in one of the worst Avs seasons of all time, and GM Francois Giguere was fired, Granato was fired, and Joe Sacco was hired the day after the GM was hired (meaning Sherman had little to no choice in his own coach). I wrote a more profanity laced post that essentially said the same thing:
Well Pierre Lacroix has clearly gone insane. Today it was revealed that he is promoting an unproven, and some could make the argument an unsuccessful, coach from within the orginization to run the team without even bothering to do a token search for someone who might actually have soem hockey knowledge and happens to be working within a different organization. Somehow promoting this greenhorn, who has very little success as a coach, is going to turn the Avs into contenders.
After Joe Sacco was fired this year Terry Frei, probably the most astute and well informed Avs observer there is, wrote an excellent column in which this he seems nearly prophetic with a single line:
No coaching search can be credible until the direction of the organization and the future makeup of the front office is clear.
Can a credible coaching search consist of one candidate?
Maybe this cynicism is a bit alienating, but this is the third time I've written a variation of this post in the last 5 years. Over that time the Avs have more top-5 draft picks than playoff wins. They have been at or near the bottom in total salary. The front office shake-up was supposed to change things, but the very early first indication is more of the same.
Today AJ over at Mile high Hockey presented an interesting argument for the Avs hiring Dallas Eakins of the Toronto Marlies (the Maples Leafs AHL affiliate) It is an argument that certainly piques my interest. That said, One of the things I really want out of the Avs in this process is a a comprehensive coaching search. In addition to Eakins, I really hope the Avs take a look at a coach who isn't yet available, but may be very soon.
One name that might be available and I think the Avs should look at is the current coach of the Avs much hated rivals Alain Vigneault. I know he's a hated Canuck and all, but I really think he's one of the best coaches in the NHL, and it's because he understand how to best use his offensive players an example:
In 06-07 he started coaching the Canucks and the Sedins. The Sedins were 26, which, in hockey player terms, is the end of their traditional offensive primes. Before AV arrived they had peaked the previous year (age 25) with 75 points for Henrik, and 71 points for Daniel. Their first season with AV they each had greater than 80 (81, & 84 respectively)
What AV did was realize that, even though they are perfectly capable defensively, their talents should really be used in the Offensive zone as much as possible. The Sedins then started increasing in points the next three years until peaking at 29 with 112 pts for Henrik, and 30 with 102pts for Daniel. Hard to tell in a 48 game season, but it appears both are still at their near prime. It'a also important to look at what he's done with Kesler (whom ROR fits a pretty decent mold but with more character!)
The Avs have Matt Duchene, Gabriel Landeskog, and Ryan O'Reilly entering their primes, not leaving them as the case with the Sedins, and they may benefit under Vigneault's willingness to put them in situations that allow them to flourish. Not to be outdone, but the Canucks backline has always been competent as well.
Of concern with AV: he sometimes let public pressure get to him, especially with goal tending. The Luongo/Schneider situation deteriorated when he went with Schneider in the playoffs because, well Canuckistan media. He's also not likely to face that kind of pressure here.
Hopefully the Avs vet both these candidates, and more, before coming to the best decision.no comments
As many here have seen here hockey is starting to warm up to fancier statistics. A ton of really smart, advanced statistical analysis has been done by many people over the last 5-8 years that has seen a great deal of new useful statistics (Corsi, OZone%, PDO) replace more traditional statistics (±) in effective player evaluation. I've had a number of people lately ask me to explain some of them to them, and when a Wild fan finally broke down and asked I figured that it was time to put it in a post, instead of individual e-mails. I'll keep the Wild fan's name anonymous so she doesn't get ostracized from her collective. We'll call her E. Wiener for anonymity's sake. Wait that opens way too many childish jokes, I'll call her Emilie W instead.
One of the key points that really gets lost is that the only math you need to know to understand these statistics is: +, -, x ,÷. Yes, there was a lot of advanced statistical mathematics that went into the development and checking of these stats, but that math is completely unnecessary to put them to use in day-to-day player and team evaluation. I will put some links to some posts that show the math at the bottom, so if you really want you can check it out, but knowing the math behind the stat isn't necessary for knowing what the stat measures.
And that's a key point, a statistic is a measurement. The better the statistic (and the more events that make a statistic), the better the value. Many have heard me say ± is useless, and it's because using ± is like using a sundial to measure a second, or an unmarked yardstick to mark off an inch. The fidelity of ± is so poor that there's no fidelity to the measurement. It's worthless. Luckily for us, some people have developed some statistics that provide a ton more fidelity.
This is not a post where I go out and prove the things I say below. There's a ton of data and thought and math that has gone into a lot of it. I am mentioning conclusions based on those (which I have read, examined a TON of the last 4-5 years, by the way.) So if I say something like "It turns out that teams aren't able to affect their shooting % all that much" it's not just some opinion I'm throwing out willy-nilly. It's based on a lot of math and stats. Not to say there can't be disagreement, but I'm not saying anything lightly here.
So here's a list of the most important fancy stats out there, in my opinion.
What does it measure: Puck Possession
By far the most used and most talked about stats of the advanced stats crowd, Corsi and Fenwick are like fraternal brothers to one another. Corsi is total shots on goal: [shots on goal(including goals) + missed shots + blocked shots)]. (notice only + signs there). Fenwick is the same thing without the blocked shots counted.
Both stats are normally displayed either in a ± fashion (such as on the Behind the Net website) or as a ratio in a % fashion (such as on Hockey Analysis website)
Why it's useful: Both these stats heavily correlate with puck possession. It also turns out Puck Possession correlates with winning, pretty well actually.
Team Corsi is extremely useful for measuring how well a team is playing (given a large enough sample size, by the way). Individual Corsi is good too, but it needs to be put into context a little more (on a team level things like competition level and zone starts even out a lot more than at an individual level)
So if it's similar to ±, why is it useful but ± useless: sample size, sample size, sample size. Sample Size is the key to any good statistic, and there just aren't enough goals scored in any given season, while a player is on the ice, for ± to tell you anything. This sample size thing is hugely important, and even though it is hugely important, many in the advanced stats world can forget it from time to time (myself included, as I'll explain in a later post). You need hundreds of individual events to form a significant sample size.
What it measures: Even Strength Save % of a goalie/goalies
Why it's useful: Wins and GAA are too dependant on team play to give an accurate judgement of goalies. But Save % is something a goalie has almost, almost, exclusive control over. It turns out that teams aren't able to affect their shooting % all that much For example last season. The ES comes in handy because teams PKs vary, mainly in # of instances (and a goalies Sv% does go down on the PK). (Yes that means players do play a hand in shooting quality, but it turns out that most players are about the same on defense, so it pretty much normalizes out with little variation in the quality of chances being given up at the NHL level.).
So ESSV% is the best evaluator of goaltender play. It's why astute Hurricanes fans weren't that worried about Cam Ward being hurt. His ESSV% was only .917, which is pretty pedestrian, so subbing in a decent backup wasn't as big as a downgrade as some feared. Dan Ellis has one recent season where his ESSV% was lower than Cam Ward's current ESSV%. Carolina's play hasn't suffered.
What it measures: Number of time a player starts (takes a faceoff) in the Offensive zone vs starting in the Defensive Zone
Why it's useful: It helps build a picture of how a coach uses a player. In general a player used in the Offensive zone more often is going to score more points than one used in the defensive zone more often. The Sedins took a massive jump in points produced around the same time their Ozone usage went up. In 07-08 they were started in the Ozone in the mid 50% range.
Then the Canucks started shiting them heavily in the Offensive zone starting around 09-10 (check out 10-11 for sure) and their points jumped from the mid 70-low 80 point range to Mid 90-100 point range.
Conversely players in the low %s tend to play more defensive game. and are being used as defensive players. It's why I have defended Stastny's point production this year especially: he's being used very heavily in his own zone this season, the way a 3rd line center would/should be used.
As you can imagine this zone start has a large effect on Individual Corsi ratings, and should almost always be used when looking at individual corsi of players.
What it measures: Corsi of the on-ice competition
Why it's useful: This is, by far, the fuzziest of the stats I have listed here. I don't really trust it as a number, per say, but I usually compare rank on the team. Player X plays the 2nd toughest competition on the team. Player Y plays the softest competition on the team. I usually only use rank on team and not really look at the number too much, unless a major discrepancy shows up.
That said, it's still useful to see who guys are playing against. John Mitchell and Paul Stastny have similar point totals (15, and 17 respectively) but Stastny is playing, by far, the harder competition. Is 17 points playing against the Toews, Datsyuks, Sedins and Parise's of the world more impressive than 15 points whoever is on those teams third lines? Yeah it is. Those points aren't created in a vacuum, and Stastny regularly goes against the other teams toughest competition.
What it measures: Fortune
Why it's useful: PDO is actually two other useful stats combined together: ESSV% + ES shooting percentage. ESSV% we covered, and ESsh% is a cousin of it. It's the on ice shooting percentage of a team. For individuals it's the ESsh% & ESSV% while a player is on the ice. It turns out that teams don't have much control over how well they shoot, and have a lot more control over how much they shoot. Shooting %'s tend to be in the 6-9% range for teams, and those %'s aren't repeatable from year to year. So a team with a giant shooting percentage will regress, and a team with a terrible one will egress to the mean.
a player or team with a PDO far away from 1000 is typically having a streak of luck (good or bad depending on the direction away from 1000). And teams with shooting percentages much higher than 9.5% are said to be shooting well above their true talent level.
The exception here is that really good goalies can maintain a very high ESSV%, this is why teams like Vancouver and Boston in the Thomas years routinely have PDOs over 1000, their goaltenders are head and shoulders better than average.
These 5-6 stats are a really good place to start when it comes to Advanced stats. There's a lot more, and a lot of ways to represent these, but If you start to get a handle on these 5-6 it will really provide a lot of insight to your hockey experience.
If you want to look more in depth into these stats, here's some articles I found interesting and illuminating:
Using Adv Stats to evaluate a players performance - Cam Charron: Canucks Army, Leafs Army, The Province, others.
Randomness of ± - N Greenburg: Japer's Rink
Shot Quality correlation to Corsi - Eric T NHL Numbers/Broad Street Hockey
Shots, Fenwick, & Corsi - JLikens Objective NHL
Forrest vs Trees - Vic Ferrari: Irreverrant Oilers Fans
Saving the best for last:
Zone Time - Vic Ferrari Irreverant Oilers Fansno comments