flagyl 100

An Advanced Stat For Goalies?

As we said last week, we are giving readers the opportunity to write guest posts that will be published on Sundays. In this week’s post, Justin (also known as Keeps in the comments section) gives us his perspective on a new advanced stat for goalies called Defense Independent Goalie Ratings (DIGR) that has been circulating around the web. 

Justin is a part-owner of a goalie school up in Albany (www.macgoaltending.com) and he is in the midst of co-authoring a technical manual on the position. Needless to say, this guy knows his stuff.

Enjoy!

For several years now, hockey analysts have searched for a more comprehensive and reliable alternative to traditional goalie rate stats G.A.A. (Goals Against Average) and Save Percentage.  Michael Shuckers, a statistician from St. Lawrence University has released the most advanced metric for evaluating goalies that we have to date: Defense Independent Goalie Rating (DIGR).  This innovative new stat was first presented at the MIT Sloan Sports Analytics Conference in March of this year.

The metric aims to normalize shot distribution, type and team strength across all goalies facing more than 600 shots (based on 30 shots per game, for min. 20 games). It utilizes smoothed spatial mapping of shots taken and the frequency in which shots from those locations produced goals. Unfortunately, the data used was from the 2009-2010 season, and I was unable find this past seasons data at the time of writing this post.  Ultimately, it does not affect the analysis at all, I just prefer the most recent data.

Professor Shuckers used ESPN GameCast shot distribution data to plot every shot taken in the NHL for the ’09-’10 season, along with shot type (backhand, deflection, slap, snap, tip-in, wrap and wrist) and team strength (even strength, PP, or PK).  Empty netters, penalty shots and shootouts were eliminated from the data. The total amount of each shot type taken throughout the year was weighted by frequency into the 30 shot per game sample.

Although not all goalies face the same distribution of shots during the course of the season, Professor Shuckers believes that the sample is large enough to directly compare the probability a goalie will allow a goal on a certain shot type from a certain location compared to all of the other goalies in the sample.

I won’t bore you guys with the complex calculations that Prof. Shuckers uses, but I will give you a familiar example.  In 2009-2010, Henrik Lundqvist was the 5th ranked goalie in the NHL based on DIGR.  His actual save percentage that season was .9208, and his DIGR was .9237. This is not a huge difference, but it suggests that Hank faced a more difficult distribution of shots than the league average, and would be more successful had he faced that distribution instead of the actual shots he faced.

The usefulness of this statistic is that it essentially puts all goaltenders on an even playing field and seemingly removes stingy or porous defenses from the equation.  Unfortunately, it leaves far too many variables unaccounted for to be the all-knowing metric that we have been waiting for.

For example, rebounds and screen shots are not taken into consideration.  I don’t have to explain to you guys that if a point man takes a big slapper through traffic, it has a much better chance of finding twine than if the goalie has a clear lane of vision.  DIGR treats those shots the same.

Same goes for one-timers, the speed and angle of a pass that precedes the shot (for example, did a goal scored from the slot come from a rebound, or a pass from behind the net?) and deflections off the goalie’s own teammates.  The location within the net that the puck crosses the goal line is also unaccounted for.  However, Professor Shuckers suggests that it is possible to add some of these variables to the calculation in the future.

Additionally, the essence of this stat comes from predicting performance rather than observing performance.  Since all goalies did not face the same distribution of shots, you are not observing statistics about the saves a goalie actually made, but what they likely would make.  In my opinion, it does not seem to have much predictive value for future performance, but would serve more as a tool for evaluating past performance and comparisons.

I don’t mean to insinuate that this stat is not incredibly useful or innovative.  Most advanced metrics in sports (which hockey is in it’s infancy of developing) have inherent limitations, and the analysis is all about understanding those limitations and keeping the proper perspective when drawing conclusions from them.

8 Responses to “An Advanced Stat For Goalies?”

  1. RangerSmurf says:

    The biggest issue DIGR faces is that NHL shot location data sucks to begin with. I know he alleged that he accounted for MSG’s atrocious scorer, but every arena has things like top of the crease tip-ins credited at 15+ feet.

    So his data set is compromised to begin with, and as you said, predictively it’s no better than ES sv%.

  2. The Suit says:

    Great post!

    Ultimately I’d like to see a “quality shot” save percentage. I know it would have its own flaws (scorer bias, shot location, etc.), but I still think it would be better than what we are currently working with. Just my opinion…

  3. bob says:

    Well written post.It is one more stat for people to rely on when evaluating a player.But like you said there are still alot of unreliable and unknown variables to take into account.But that can be said about alot of advanced stats.But it is good to see more stats on the goaltending.Sometimes it seems like it is the most forgotten position when it is one of the most important.

  4. Keeps says:

    Evaluating goalies from a statistical standpoint is extremely difficult…Prof. Shuckers attempts to basically replicate FIP (Fielder Independant Pitching) that is used in baseball analysis. The problem is that in baseball, every pitch is isolated so you can better account for the variables which lead to the result.

    Hockey on the other hand, is such a fluid game that can have sustained action for minutes at a time and a play that happened 30 seconds prior to the puck crossing the line can completely differentiate it from a similar play for the purposes of the evaluation.

    It was a lot of fun to try to get under the hood of this methodology, but I think we still have a long way to go before we have something truly reliable.

  5. Glen Miller says:

    The important thing is progress continues to be made. Look at baseball; they’ve been at this for 20 – 30 years and they still haven’t figured out a widely accepted methodology for gauging defensive performance. As long as work continues to be done to make analysis better and more comprehensive then progress is being made.

  6. Glen Miller says:

    Something else to consider; there is a great likelihood there is already software available that can do a better job of determining real shot distribution better than on-ice scorers do. It can also associate accurate speeds on shots and even what part of the net it is aimed for. I’m sure someone can write an advanced algorithm to put all that data together in a useful way.

    • Keeps says:

      Excellent points Glen. Hopefully in the not too distant future we will have that advanced algorithm, which will make this type of evaluation so much more effective. The focus of our research is total efficiency for the position (least amount of expended energy/movement for the highest probability of stopping not only the first shot, but subsequent shots as well).

      There are plenty of NHL goalies who play a grossly inefficient style, but have the athleticism or a strong enough defense to fool the current evaluation models. A more alll encompassing statistic would go a long way towards more comprehensive understanding of projecting/developing young goaltenders

      • Glen Miller says:

        Agree. Hasek comes to mind who excelled playing reflexively. As does Tim Thomas. Hank has been excellent this year at maintaining good position after making the initial save. I attribute that to the coaching of Allaire and work ethic of Hank’s.