But of course we need to look at the numbers. And as you can see if you read it, he shows that there is persistence to a goalies miss%. Of course persistence isn't everything. Something being reliable doesn't imply that it's indicative of "talent". There could be other factors which cause it to be repeatable. Something like team effects, which I covered in my last post. For all we know, these numbers are just measuring the team's ability to cause shots to miss the net and not anything the goalie is doing. So we would need to do an analysis like we did last time. Run the correlation between a goalie's miss% and his team without him. But DTM did this. Look at this tweet
— DTM About Heart (@DTMAboutHeart) June 20, 2016
I don't know the exact details behind the numbers. But it looks like it takes a good number of years into account. The sample gets low as you get up there, but the result looks clear. No team effect. Or maybe not?
I say maybe not because I think there is a better way of viewing shots that miss the net. Breaking it up by danger zones. The same Low, Mid, and High made by War on Ice. I guess two thoughts come to mind here. First, War on Ice isn't up anymore. And secondly, they didn't even supply those numbers. And the answer to both of those questions is simple, we can use the play by play files left behind by the War on Ice crew (Of which they deserve a tremendous amount of credit for. It's an amazingly valuable source of info which saves people like me countless number of hours of which we'd spend getting the data ourselves.) to derive these numbers.
Now that we have these numbers we can get to work. Because while miss% as a whole may not show anything, it could be hiding a bias which can only be seen by a more granular look. For example, there may be a real effect in the high danger zone, but because most shots come from the other two zones the signal gets drowned out in regular miss%. Also breaking it up by danger zone is probably a better method in general (as with Sv%).
Before we do that, here's the miss% broken down by Danger Zone and overall (these numbers are from 2007-2008 until 2014-2015 and are 5v5).
Overall_miss% Low_miss% Mid_miss% High_miss%
27.84% 30.18% 29.17% 22.24%
The interesting part here for me is the small difference between Mid and Low. I thought it would be higher. It may be due to the fact that I underestimated how many shots miss the net from the low danger zone by virtue of being blocked and not merely missing. Nevertheless, for those interested, here are the distribution of shots broken down by Zone.
Low_miss% Mid_miss% High_miss%
45.75% 28.33% 25.9%
And this is just a little more slanted towards the low danger zone than shot on goals.
Ok, so now we get to the important part of checking for team effects. Before I do so there is one more thing to mention which the astute observer may have picked up. In his same team analysis done above, DTM only uses road shots to control for rink bias (bias due to the scorer's recording of statistics. For example, some arenas inflate SOG.....etc). Some good work on the subject is shown here. Ideally, I would use that method and get the numbers myself, but it's a bit too involved for me. I can work out my own crude version, but I think just road shots will be fine for this analysis. But, of course, ideally we'd be using rink adjusted numbers.
So as I said be before the numbers I'm using are 5v5 and from 2007-2008 to 2014-2015. In order to achieve a better sample for players with more shots, I coupled up the years. Years 2007-2008 and 2008-2009, 2009-2010 and 2010-2011 are paired up and so on. For each pair, I divided it up by team and calculated the goalies who faced road shots for that team. And then, based on what sample I'm using, I run a correlation between what the goalie did for that team (and that team only, any numbers the goalie accumulated for other teams was excluded) and what the team did without him. So I did that and here are the results (the numbers below are r not r^2 and the sample restriction applies for both the goalie and without him):
n
|
Low_miss%
|
Mid_miss%
|
High_miss%
|
Total_miss%
| |
250+
|
321
|
.048
|
.061
|
.018
|
.057
|
400+
|
256
|
.03
|
.08
|
.032
|
.034
|
500+
|
222
|
.05
|
.165
|
.0275
|
.068
|
750+
|
129
|
.011
|
.123
|
.053
|
.03
|
These numbers corroborate what DTM found. There is practically no relationship. The highest relationship we see is with mid danger but it's still rather tenuous at around ~.10. Don't get me wrong, that is something but it is still small. So we can safely say that team effects only play a very small role even when we break it up by danger zone.
Also, we might as well take a quick look at the reliability of miss% by danger zone. Showing no team effect is nice, but we have to know when splitting it up that we aren't just looking at noise. So I did something similar to what DTM did when he split up games into even and odd groups. Instead I did it at the shot level. I lined up all the shots for each goalie and ran a correlation between the missed% for the even group and the odd group. The cutoff was 1000 shots and the numbers here are road shots only (the numbers below are in terms of r).
Low Mid High Total
.221 .333 .466 .303
This is what we generally expect. High, then mid, and lastly low, with total somewhere in the middle. I also ran it for total shots (not just road) and the trend stays relatively the same (with higher correlations of course). Here they are:
Low Mid High Total
.513 .506 .682 .62
It's interesting because here low seems to be more on par with mid. Maybe rink effects are higher for low? Maybe this is more accurate? I'm not sure. I put more stock in just road numbers but I have to work on getting better estimates. It's also interesting that we get a fair number for low_miss%, as opposed to nothing for Low_Sv%. Nevertheless, both of these show that there is clearly something to these numbers as they are repeatable.
I think all of this (on top of the previous work done by DTMAboutHeart) shows conclusively that goalies have the ability to force shots to miss the net and that team effects are minimal at best. So therefore we should be using it in our evaluation of goalies. And I'd say a better way to look at it is when it's broken down by danger zone. I have some more thoughts on this (so I'd love to hear any thoughts on the subject) and I'll probably post something in the near future on it. Here's a google docs with the numbers-https://drive.google.com/open?id=1yN8fPU4ISVEQpr_GLvr1PTSY7fGNOF1itdNlTY6eXl8
***All Data courtesy of (the now defunct) War on Ice
