Marcel Projections for Goalies

**Also posted on NHLNumbers here

As most of us already know, goalies are voodoo. This makes projecting how our favorite team's goalie will do next year (unless that goalie is Henrik Lundqvist) rather difficult. To help with that we can make projections. Specifically I'll make Marcel projections for goalies. What's Marcel? It's a forecasting model created by Tom Tango (some examples). He used it for baseball and we'll adapt it to hockey.  Of course, this has actually been done before for goalies here by Garik16. But while he used regular Sv% I'll attempt to project Low/Mid/High Sv% (courtesy of Corsica.Hockey) as it gives us a better look at a goalie's performance.

So how does it work? There are three steps. First we weight previous years performance. For example, a player's performance last year matter more than 2 years ago.....and so on. Traditionally only the past three years are used. Here following the suggestion of Eric Tulsky (as did Garik16) I'll use four. Goalies are hard to predict so using some more data makes sense. Second, we regress towards the mean (how much depends on which statistic we're using). And lastly we'll apply an aging curve.

I'll only be using even-strength numbers here. Also as I said before all data is courtesy of Corsica.Hockey.

Weighting Past Performance

Traditionally the weights are 5/4/3 but we can do a little better. So I took the past 9 years of data (all we have) and recorded every instance where a player faced at least 400 total shots in each of the past four years and at least 800 shots in year 5. We can then run a multivariable regression with the past 4 years as the independent variables and year 5 (the year we're trying to predict) as the dependant variable.

But we run into a problem (as some of you might have guessed). The problem is sample size. Due to the fact that we only have nine years of Danger Zone data and because a goalie needs to have played in 5 straight years we start running out of goalies and therefore the weights (well...really only for one of them as we'll see soon) don't make a lot of sense. I tried tweaking the sample restrictions around but the weights still end up moving all over the place.

To account for this, I ran the numbers for players who faced at least 400 total shots in each of the last three seasons (not four). This will give us more of a sample and therefore a better chance of getting some accurate numbers (thankfully it worked). And then we'll take an educated guess at the weight for year four. It's the best we can do here. So I did that and here are the numbers it spits out for High-Danger Sv% (All numbers here are adjusted for league average of that year):

Year n-1: .129
Year n-2: .1058
Year n-3: .0363

Re-weighting that we get 10/8/3. A bit more aggressive than 5/4/3 but It seems fine to me. But how much would year 4 be worth? I don't know. I'll give it a weight of 2. That makes sense to me. And honestly, it won't make much of a difference if the weights slightly differ. Either way, these weights indicate the past two years of a player's performance matters a lot for projecting next years HSv%.

I guess the last step here is Mid and Low Sv%. This may not come as much of a surprise to some of you but they don' get weighted at all. Each season counts the same. This is because Low/Mid Sv% contain a lot less skill than High Sv% making one year of data a really bad judge of talent.

So here are the final weights:

Year     LSv%        MSv%        HSv%
n-1        1                 1               10
n-2        1                 1                8
n-3        1                 1                3
n-4        1                 1                2

Regressing 

When we look at any metric it's important to remember that what we're looking at is a combination of a few forces. The numbers we see are just the observed data. That's made up of: Talent, randomness .....etc. Our job is to try to figure out just the talent component. There are multiple ways to do this, but I prefer to do it according to the following method (Note: If you aren't interested in the math you can skip to the end of this section).

I took all goalie's with at least 1000 total shots (cumulatively) from 2007 until now. I then calculated the standard deviation for each Low/Mid/High Sv%:

Type        SD
Low        .0049
Mid         .0101
High       .0227

I'll then use a technique commonly used by Tom Tango (the same guy who created Marcels). He notes that:

Observed variance= Talent Variance + Luck Variance

Of course, there are more things that effect the observed distribution than just talent and luck. Another is team effects of which I wrote about here (Note: There are still technically more effects but this should do). So now, we have:

Observed= Talent + Luck + Team Effects

Our job now is to solve for talent using the observed variance, luck, and team effects. In order to do so, we need to compute the latter two. Let's first look at "luck". It's important to note that "luck" is the expected spread we would expect if no skill existed among goalies (all goalies have the same skill level). We therefore have to approximate the amount spread in results we'd expect among our population of players if Sv% was random. Since we are dealing with a binomial the standard deviation should equal sqrt(P*Q/N). Where p is the league average probability of a save, and q is the league average probability of a goal (or 1-P), and N is shot attempts. But what number should N be?

Since we are dealing with a bunch of different players with varying amount of attempts it isn't so clear cut. An easy guess would be the straight average among all players in the sample (and this will often be fine) but I tested it out and found that by weighing by the harmonic mean was a better fit (I came across it here and after running a simulation found it was a better fit). So to calculate N I take the harmonic mean(which is 1/n) of every players shot total (for either Low Shots, Mid Shots or High Shots), take the average of that and then take the reciprocal of the average. Here are those numbers.

                 LSv%          MSv%         HSv%
Avg. Sv%     .978          .9254        .8036
N              1135.6          918.6        580
SD            .0043           .0086        .0165

I then estimated the spread in team effects in a similar fashion to the article I linked a few paragraphs back. It only really matter for HSv% (for low and mid it's effectively 0) and even there it's not a big deal. But here's the SD in team effects for HSv%: .0036.

Using the observed spread and the spread in luck and team effects we can solve for the spread in Talent: Here you go:

For example: HSv%:  .0227^2 = Talent^2 + .0165^2 + .0036^2

 Type     SD Talent
Low       .0024
Mid        .0053
High       .015

We then set Talent equal to luck and solve for n (How many attempts do we need for luck to be equal to talent?). We then get the following numbers of attempts for each danger zone (I rounded off):

Low= 3700
Mid= 2400
High= 685

What this means is that we need to add those amount of shots to each players total to regress his numbers. Or we can think about calculating a specific r (correlation coefficient) for each player and then figuring out how much to regress from there (Regressed rate= 1-r). For example, let's a player has 500 High Danger shots. We can then use the equation a/a+c. Where a is the number of player attempts and c is the constant. So we have in this example 500/(500+685) and get r=.442 and and (1-r)= .558. So we regress that player's numbers 55.8% towards to mean.

Lastly, for Low and Mid Shots since the weights for each season is equal, the a in the equation a/a+c is just the sum of the shots faced over the past four years. But for HSv% we know that Year n-1 matters more than n-2....and so on. So a shot faced a year ago matters more than two years ago. So we need to weigh High Danger Shots faced. That's easy, all we have to do is divide each weight by 10. And we now just multiply shots faced by the new corresponding weights (1/.8/.3/.2) and add them up. So if a player faced 300 shots last year and 200 the year before, we do: (300*1)+(200*.8)= 460 Equivalent shots.

Aging Curve

I'll try to keep this section short. Basically I looked at aging recently here. But due to the fact that we only have danger zone data for the past nine years we can't construct any aging curve with it. There isn't enough data. So instead I did it for All-situation Sv% (using data from 1990 until now). From there we can guess at the appropriate numbers for Low/Mid/High Sv%. That's really the best we can do. So I played around with the numbers and settled on (arbitrarily of course) dividing the weights by 4 for LSv%, keeping it the same for MSv%, and multiplying by 2 for HSv%. High danger contains most of the skill so we want most of the aging to be there, Low Danger has the lowest so the aging there should be small, and Mid Danger is somewhere in the middle so I gave it the same weights as I originally found. So here are the weights I settled upon for each.

Age	Change-Low	Change-Mid	Change-High
21	7.5E-05	0.0003	0.0006
22	0.000025	0.0001	0.0002
23	-2.5E-05	-0.0001	-0.0002
24	-7.5E-05	-0.0003	-0.0006
25	-0.000125	-0.0005	-0.001
26	-0.000175	-0.0007	-0.0014
27	-0.000225	-0.0009	-0.0018
28	-0.000275	-0.0011	-0.0022
29	-0.000325	-0.0013	-0.0026
30	-0.000375	-0.0015	-0.003
31	-0.000425	-0.0017	-0.0034
32	-0.000475	-0.0019	-0.0038
33	-0.000525	-0.0021	-0.0042
34	-0.000575	-0.0023	-0.0046
35	-0.000625	-0.0025	-0.005
36	-0.000675	-0.0027	-0.0054
37	-0.000725	-0.0029	-0.0058
38	-0.000775	-0.0031	-0.0062
39	-0.000825	-0.0033	-0.0066
40	-0.000875	-0.0035	-0.007

What those numbers represent is how much a player is expected to increase or decrease relative to league average from last year. For example: Let's say a player's League Adjusted HSv% (Player Sv% divided by Lg HSv%) is 1.02. And let's say he's turning 31 next year. So according to the chart we expect him to drop of .0034. So 1.02-.0034= 1.0166. And if the league average HSv%=.81. We then just do .81*1.0166= .8234. 

Conclusion 

Ok, so that's really it. Here is a google docs with the full projections. Everyone who faced at least one (even strength) shot on goal last year was included. Also, you may notice, I also added a fourth category to those numbers- Adj.Sv%. It's calculated the same way War on ice used to do it. It's just the weighted average of Low, Mid, and High Sv% by it's frequency. This is what we would expect a player's Sv% to be if he faced a league average shot distribution.

But, before I finish, I need to note a few things. First, by the nature of the model the less a player has played the closer he is to average. This is because in the absence of any data my best guess is league average. Ideally I'd regress towards specific priors but this is meant as a simple model. Second, HSv% is the most important component. This is because it's where most of the spread in skill is and because it's the fastest one to reach a signal. Third, it assumes the Sv% for each zone next year will be the same as last year. The overall Sv% last year was .9249 and by danger zone .979/.9251/.8133. So keep those in mind when looking at the projections.

Lastly, some projections will look weird. For example, Joonas Korpisalo is projected to have a .9269 Adj.Sv% even though he only played 31 games in his career (all last year). Why? Well, since he barely played his LSv% and MSv% are very close to league average. But he posted a pretty good HSv% at .8434, and because HSv% accumulated a signal the fastest, he still ends up being a bit above average afterwards. Also since he's 22 we still expect him to improve a little. Add all this up and you get his numbers.

***All Data courtesy of Corsica.Hockey

Goalie Aging Curve

I think we all know aging curves are important in sports. It's a vital component in projecting how players will perform in the future. This type of research has been done for every sport (here's one by Eric Tulsky concerning Corsi ). And actually, a study was once done on goalies by @garik16 here. My study will closely mirror his with a couple of differences. To those people not acquainted with the work done on aging curves I suggest reading the following pieces listed below (along with Garik's piece).

http://www.hardballtimes.com/how-do-baseball-players-age-part-2/ (Also Read Part 1)
http://www.hardballtimes.com/fielding-aging-curves/
http://www.sbnation.com/nhl/2013/11/5/5024738/nhl-free-agency-age-contracts

Due to the dearth of data I'll be using All-Situation SV% available at http://www.hockey-reference.com/. I'm fine generalizing this to even strength Sv% (how to apply it to the different danger zones is a different story). I'll also go back to 1990, so 26 years. Because of that I'll have to adjust for changes in Sv% (The average All Situation Sv% in 1990 was 88.6%, this past year it was 91.49%).

Also, I'll be using the Delta Method here, with each couple weighed by the harmonic mean of their shots. I take each player who played in back to back years (no matter how many shots they faced). I calculate the difference in Sv% and weigh that by their harmonic mean.

Of course there is one concern (and this is where my work differs from Garik). Basically those who survive to play in year 2 aren't only, on average, better players but also luckier players (this is noted here by Tango).  So they likely did better than their actual talent in year 1. This would magnify the difference because in year 2 not only will they decline due to age but they will regress downwards. So the difference between pairs will look bigger than it actually is. We need to try to isolate just the aging component. So I'll have to regress year performance.

The amount I'll regress year 1 performance is based on the following: I took every player who played at 30 games in back to back years. A ran a regression and got an r of .296. The average player in the sample faced about 1450 shots. Doing (1-r)/r* Avg. Shots......we get about 3400 shots. For even strength Sv% the number is about 2950. So this is a little higher. Makes sense to me.

Okay now we can finally get to the numbers. Due to sample size issues I'll only go from the 20-21 couplet to the 37-38 (we'll extrapolate for other ages). Now I'll do something similar to what garik16 did. I plotted the differences for each pair and to smooth it out I fitted a line onto it. Here it is:

As you can see there's a pretty clear downward slope. As we would expect. The older you get the more you are expected to drop off from the previous year. Fitting a line is important because, as you can see, the numbers jump around a bit due to sample size. Goalies don't improve as they turn 33 as the graph shows. It's a blip due to small sample sizes. Note: This isn't me saying that aging is linear. I'm merely using a linear relationship to model the differences between pairs (or how much you age between ages). Those are two different things. If it was linear you would expect a player to decline the same amount each year. Instead here how much a player declines each years increases at a linear pace (The reason I chose to do it this way is because, like Garik, I found the numbers to be more believable. Either way, there isn't that much of a difference). Nevertheless, here are the exact numbers:

Age	Change
18	0.0009
19	0.0007
20	0.0005
21	0.0003
22	0.0001
23	-0.0001
24	-0.0003
25	-0.0005
26	-0.0007
27	-0.0009
28	-0.0011
29	-0.0013
30	-0.0015
31	-0.0017
32	-0.0019
33	-0.0021
34	-0.0023
35	-0.0025
36	-0.0027
37	-0.0029
38	-0.0031
39	-0.0033
40	-0.0035

This corroborates previous work showing that goalies start declining early (Note: I believe this underestimated the amount a goalie should improve between 18-22.....but this is the best I got). One may also notice that this is less aggressive than what Garik found (look at the bottom of his post) For example: At age 30 he has -.002 and I have -.0015 and at age 36 he has -.004 and I have -.0027. This is a consequence of me regressing year 1 performance. As I said before, it narrows the gap because the goalies, on average, likely overachieved in year 1 which then conflates the difference.

I guess the last question here is how to translate all this to Low/Mid/High Sv%. Well........I don't know. One has to take an educated guess here. I played around with the numbers a little and I settled on dividing the weights by 4 for Low_Sv%, using the same numbers as above for Mid_Sv%, and Multiplying by 2 for High_Sv% (Slightly higher might be better). These are obviously arbitrary but they fit the overall aging pattern well and I think it makes sense. High_Sv% should be the highest as it contains the most "skill". Low_Sv% has a ton of noise so the adjustments should be small (honestly one could do without adjusting for age here). And Mid_Sv% should be somewhere in the middle (one could possibly argue a little lower than what I suggested).

To conclude, this post doesn't do much. I merely suggest (to the few of you who care) a slight amendment to Garik's aging curve. I am also fine applying this to Even-Strength Sv%. The Danger Zones are a little trickier. My suggestions are noted above.

***All Data Courtesy of Hockey-Reference

Should We Incorporate Misses into our Evaluation of Goalies?

Back in October DTMAboutHeart posted this interesting article. As usual, please read it if you haven't yet. Basically, the really interesting part of it is his idea of incorporating shots that miss the net into our evaluation of goalies. It's interesting because I think it kind of makes sense (I say this as someone who never played goalie, so take it with what you want). Goalies who position themselves better could force shooters to try to aim for a corner or the side of the net and therefore shoot more pucks wide. And if goalies can force shots to go wide, we should then take notice of it.

But of course we need to look at the numbers. And as you can see if you read it, he shows that there is persistence to a goalies miss%. Of course persistence isn't everything. Something being reliable doesn't imply that it's indicative of "talent". There could be other factors which cause it to be repeatable. Something like team effects, which I covered in my last post. For all we know, these numbers are just measuring the team's ability to cause shots to miss the net and not anything the goalie is doing. So we would need to do an analysis like we did last time. Run the correlation between a goalie's miss% and his team without him. But DTM did this. Look at this tweet

@NMercad @AdamZHerman It's not pic.twitter.com/ztBCHLCDG2

— DTM About Heart (@DTMAboutHeart) June 20, 2016

I don't know the exact details behind the numbers. But it looks like it takes a good number of years into account. The sample gets low as you get up there, but the result looks clear. No team effect. Or maybe not?

I say maybe not because I think there is a better way of viewing shots that miss the net. Breaking it up by danger zones. The same Low, Mid, and High made by War on Ice. I guess two thoughts come to mind here. First, War on Ice isn't up anymore. And secondly, they didn't even supply those numbers. And the answer to both of those questions is simple, we can use the play by play files left behind by the War on Ice crew (Of which they deserve a tremendous amount of credit for. It's an amazingly valuable source of info which saves people like me countless number of hours of which we'd spend getting the data ourselves.) to derive these numbers.

Now that we have these numbers we can get to work. Because while miss% as a whole may not show anything, it could be hiding a bias which can only be seen by a more granular look. For example, there may be a real effect in the high danger zone, but because most shots come from the other two zones the signal gets drowned out in regular miss%. Also breaking it up by danger zone is probably a better method in general (as with Sv%).

Before we do that, here's the miss% broken down by Danger Zone and overall (these numbers are from 2007-2008 until 2014-2015 and are 5v5).

Overall_miss%           Low_miss%        Mid_miss%           High_miss%

    27.84%                      30.18%               29.17%                 22.24%

The interesting part here for me is the small difference between Mid and Low. I thought it would be higher. It may be due to the fact that I underestimated how many shots miss the net from the low danger zone by virtue of being blocked and not merely missing. Nevertheless, for those interested, here are the distribution of shots broken down by Zone.

Low_miss%             Mid_miss%           High_miss%
 45.75%                       28.33%                 25.9%


And this is just a little more slanted towards the low danger zone than shot on goals.

Ok, so now we get to the important part of checking for team effects. Before I do so there is one more thing to mention which the astute observer may have picked up. In his same team analysis done above, DTM only uses road shots to control for rink bias (bias due to the scorer's recording of statistics. For example, some arenas inflate SOG.....etc). Some good work on the subject is shown here. Ideally, I would use that method and get the numbers myself, but it's a bit too involved for me. I can work out my own crude version, but I think just road shots will be fine for this analysis. But, of course, ideally we'd be using rink adjusted numbers.

So as I said be before the numbers I'm using are 5v5 and from 2007-2008 to 2014-2015. In order to achieve a better sample for players with more shots, I coupled up the years. Years 2007-2008 and 2008-2009, 2009-2010 and 2010-2011 are paired up and so on. For each pair, I divided it up by team and calculated the goalies who faced road shots for that team. And then, based on what sample I'm using, I run a correlation between what the goalie did for that team (and that team only, any numbers the goalie accumulated for other teams was excluded) and what the team did without him. So I did that and here are the results (the numbers below are r not r^2 and the sample restriction applies for both the goalie and without him):

	n	Low_miss%	Mid_miss%	High_miss%	Total_miss%
250+	321	.048	.061	.018	.057
400+	256	.03	.08	.032	.034
500+	222	.05	.165	.0275	.068
750+	129	.011	.123	.053	.03

These numbers corroborate what DTM found. There is practically no relationship. The highest relationship we see is with mid danger but it's still rather tenuous at around ~.10. Don't get me wrong, that is something but it is still small. So we can safely say that team effects only play a very small role even when we break it up by danger zone.

Also, we might as well take a quick look at the reliability of miss% by danger zone. Showing no team effect is nice, but we have to know when splitting it up that we aren't just looking at noise. So I did something similar to what DTM did when he split up games into even and odd groups. Instead I did it at the shot level. I lined up all the shots for each goalie and ran a correlation between the missed% for the even group and the odd group. The cutoff was 1000 shots and the numbers here are road shots only (the numbers below are in terms of r).

Low            Mid             High             Total
.221           .333            .466               .303

This is what we generally expect. High, then mid, and lastly low, with total somewhere in the middle. I also ran it for total shots (not just road) and the trend stays relatively the same (with higher correlations of course). Here they are:

Low            Mid             High             Total
.513           .506            .682             .62

 It's interesting because here low seems to be more on par with mid. Maybe rink effects are higher for low? Maybe this is more accurate? I'm not sure. I put more stock in just road numbers but I have to work on getting better estimates. It's also interesting that we get a fair number for low_miss%, as opposed to nothing for Low_Sv%. Nevertheless, both of these show that there is clearly something to these numbers as they are repeatable.


I think all of this (on top of the previous work done by DTMAboutHeart) shows conclusively that goalies have the ability to force shots to miss the net and that team effects are minimal at best. So therefore we should be using it in our evaluation of goalies. And I'd say a better way to look at it is when it's broken down by danger zone. I have some more thoughts on this (so I'd love to hear any thoughts on the subject) and I'll probably post something in the near future on it. Here's a google docs with the numbers-https://drive.google.com/open?id=1yN8fPU4ISVEQpr_GLvr1PTSY7fGNOF1itdNlTY6eXl8

***All Data courtesy of (the now defunct) War on Ice

Team effects and Sv%

A couple of weeks ago a very interesting post was published by @gentleputsch. Please give it a read. Ok, I'm assuming you read it. What interested me the most, as you can probably tell by the title of my post, was the team effect on Sv%. It was bigger than I expected and kind of caught me off guard. With that said, I had a few thoughts on the post (the main part of his post on considering shorthanded Sv% is another thing in itself so I'll leave it out for now) and I want to rerun the numbers a little differently.

First off, running the data just on the past three years seems odd. For all we know, running it on the previous three year stretch could bring back nothing. So we should really incorporate more data. Also, at first I didn't understand why he chose a three year period, but after thinking about it I agree. If we look at one year, we'll end up having a lot goalies who play 50 to 65 games, not leaving much of a team sample. This is lessened for a two year period and even better for a three year (I guess one could actually do it for 2 years but 3 seems like a better choice). With all that said, let's see the numbers.

Just in case some people don't remember, these numbers here are 5v5. The data here is from 2007-2008 until this year. I divided it up into three, three year periods: 2007-2010, 2010-2013, 2013-2016. To be included, a goalie needs to have played all three years with the same team. But instead of just choosing the goalie with the most minutes for each team, I chose goalies with at least 1250 total shots faced. It's arbitrary, I know. But I wanted to get at least ~50 games for each goalie. The total amount of goalies is 84 (so choosing this over the most minutes doesn't make a real difference, we end up with basically the same players anyway). We then run a correlation between a the player's save percentages and that of the team when he's not on the ice. The following numbers are r (not r^2).

n        Low_Sv%    Mid_Sv%     High_Sv%

84          .25              .01         .264


The numbers here seem a bit different than what @gentlepush published. Low_Sv% is about the same, but High_Sv% is lower and Mid_Sv% is nonexistent. It's kind of odd that we get something for low but not mid. Running the numbers for 1 year and for 2 year spans bring back just about the same numbers for mid and high Sv%, Low was practically zero for those years but for some reason gets a spike here. Either way Low_Sv% doesn't matter because we already know that Low_Sv% means almost nothing. As I estimated in my last post it needs about ~13000 shots to regress it (and it's actually slightly higher because of the team effect). That means when a goalie has logged about that many low danger shots we know about half his talent. So we shouldn't even bother with it. With that and Mid_Sv%, which is in the clear, out of the way we now have to deal with High_Sv%.

So the question here is: How do we account for the team effects? I think the first step is thinking about what makes up a player's High_Sv% (or to be fair Sv% in any zone). We know a player's own skill matters, some luck, and as we covered here team effects. So it's kind of like this:

Observed High_Sv%= Talent + Luck + Team Effects

With this in hand, we can estimate the spread of High_Sv% due to talent.

Observed var =  Talent var + Luck var + Team var

The var here is variance. Basically the spread in observed High_Sv% is made up of the spread in talent, luck, and team effects. So let's get some numbers down (those familiar with Tom Tango's work will recognize this type of equation):

One Standard Deviation of observed High_Sv%= .0178 (I thinks it's important to remind everyone I'm doing these numbers over a three year period, for 1 or 2 year the observed SD would be higher).

Luck= p*(1-p)/ Avg # of Shots (this would be the binomial variance)

p here is the average High_Sv% over the period, which is .832. So.....

(.832)(.168)/798.5 = .0000175 (this is the variance, one SD is therefore .0132)

Lastly, team here would just be our correlation multiplied by the observed spread. So we get one SD of Team effect is .0047.

So let's plug it in.

.0178^2 = Talent^2 + .0132^2 + .0047^2

Solving for Talent we get...... SD of Talent = .011

Now all these numbers are nice, but I don't think anyone really cares. But why does this all matter? Because our estimation of a goalie's High_Sv% talent was previously overinflated. We incorporated the team effects into the numbers. And now that we, presumably, got rid of it we could get a better estimate (which I tried two weeks ago until I saw....) as to how much do we need to regress High_Sv%. Mid_Sv% is the same, Low_Sv% doesn't matter, but High_Sv% needs to be adjusted. As usual Tom Tango provides a solution (it seems like his new and old blog are filled with dozens of ways to regress, each a little different)

So let's do it:

Talent SD^2/Observed SD^2= .378 (this would be r^2)

And now.............. (1-.378)/.378 *799 = 1314 Shots

My last estimate of how much to regress High_Sv% was 1121 shots. Now it's about 200 more. Makes sense to me.

At the end of the day, it seems like we were mistaking a little noise for signal in a goalie's High_Sv%. The SD of Talent is really smaller and we just have to regress a little more. I wonder what the numbers would look like for a goalie metric which instead of dividing up by danger zone assigns a probability to each shot (there are a couple of these models out there). I'd assume the team effect is smaller there.

***All Data courtesy (of the now defunct) War_On_Ice

Regressing Sv% by Danger Zone

Goalies aren't easy to evaluate. Or as some might say, they are voodoo. Introduced some time back by the war on ice crew(which will soon be defunct), and used by Nick Mercadante in his Mercad60, they split up shots into low, medium, and high danger based on the position of the shot and whether it was a rebound or a rush shot. And it's been used quite a bit. I've been thinking about it lately, and strangely, I've never actually seen anyone regress it. So I did so and I thought I might as well share it.

What we would expect, from intuition and what we know, is that: Low-Danger is almost all noise, Mid-Danger has a signal but it's weak, and High-Danger is the best and has a fair signal. Of course, we need to confirm this. My first to idea to test all this was to do it the conventional way. Take all goalies and line up all their shots. And run odd-even correlations until r=.5 (that being the correlation coefficient). The amount of shots needed to reach r=.5 would be how much we regress (and we would do so at league average). But the problem is that Sv% (in all zones) take time until a good signal is reached, and there are only so many goalies who have faced a lot of shots. We start to run out of goalies as the sample is too small (I was actually able to get a read on High-Danger Sv% and found it reached r=.5 at 1050......this makes sense as we'll see soon).

Thankfully, there are other ways. On way, which is easier and more convenient, is shown here by Tom Tango. I strongly recommend you read that before continuing (and I also recommend reading his blog in general). Ok (I'm assuming you read it), I ran the numbers on the last 4 years on goalies who faced at least 750 total shots. Here's what we find (It goes without saying....this is all Even Strength numbers). Also I only included sv% for the hell of it and for a self-check (I know it should be about 3000). The best way is regressing each zone independently.

                                  n             Sv%          Low-Sv%        Mid-Sv%           High-Sv%

SD of z Scores:           76            1.307         1.136                1.033               1.298

# of shots regressed:   76            3534          3915                 10127               1001


First thoughts from these numbers is that high danger and regular Sv% (it's near Tango's estimate) make sense. But low and mid seem to be in the wrong spots. Flip it and it makes sense, but it doesn't match up. So what I did was run it on the 4 years previous to that (2008-2012). And what we find is:

                                  n               Sv%          Low-Sv%        Mid-Sv%           High-Sv%

SD of z Scores:           71            1.457           .942             1.268             1.264

# of shots regressed:   71            2638          -11565           1352               1400

For this one Sv% makes a little more sense and High-Danger stays about the same. But as you can easily tell Low-Danger is fucked up and Mid-Danger seems too low. So what we'll do is run them together. We'll just combine the data and see what we find (One note: The percentages used now is for the entire eight year period not each four individually).

                                  n              Sv%          Low-Sv%        Mid-Sv%           High-Sv%

SD of z Scores:           147             1.4            1.046               1.16              1.295

# of shots regressed:   147             2838          12933            2165               1121

And these estimates make the most sense. Sv% is near the 3000 mark, Low-Danger is pretty much just noise, High-Danger is the best and has a fair signal. And Mid-Danger is a weaker signal.

Lastly, I'd like you to think of the spread for each danger zone sv%. Over the past three years the observed Standard Deviation for each one is: Low- .69%,  Mid- 1.317%, High- 2.232%. And what we know is that the amount of noise decreases as we move to the right. So not only does the observed spread increase as we move to High-Danger but we can attribute more of that to talent (the standard deviation due to talent is higher). I think this is important in showing why we should be mainly focusing on High Danger sv% when it comes to evaluating goalies (obviously not only focus on it, just mostly) as it has the biggest spread in talent.

***All Data courtesy of War On Ice

For this who care.....the league average sv% for each zone (and in general) over the past 4 years:

Sv%    Low-Sv%    Mid-Sv%     High Sv%
.923     .9738           .927          .8347