25: Exploring shot data

2021.05.04

Every sport has some form of a box score which provides a summary of a game’s statistics. Curling doesn’t exactly have a box score, but for the big events, the foundational data is there. The shot data recorded by scorers is subject to some personal judgment, but even so, it’s actually a decent starting point to interpret what has happened in the game.

For the uninitiated, each shot is scored on a scale of 0 to 4, with points awarded based on whether the intent of the called shot was achieved. That’s a fine start, but shooting percentages are as much about a player’s ability as they are about the difficulty of the shot called, especially for stones thrown later in the end. If your third is asked to throw runback doubles and precise hit-and-rolls all game, they’ll have a worse percentage than the player that gets to throw guards and open take-outs.

Fortunately, the raw data provides enough information to get us started on accounting for the difficulty of each shot. Each shot is given a task, and just by averaging the points awarded for each task at each shot during an end, you can get a reasonable estimate for the difficulty of each shot.

For instance, in the men’s worlds, players averaged a score of 3.65 when throwing a draw on the first shot of the end. And the first shot of the end is a nice controlled experiment. Every time the shot is thrown, we know there are no rocks in play.

For the last shot of the end, players averaged 3.18 on their draws. This is useful knowledge, but using this number to describe the difficulty of a draw on a final shot is not good enough. Because the scoring is based on achieving the shot’s objective, there are a large range of possibilities for the difficulty of the final shot, from being able to touch paint for a score to needing to draw to the pin.

Fortunately, the shots listed earlier in the end can provide some help on difficulty. Basically, if you know how many hits were played in the first 15 shots, you can get a decent estimate on whether a draw will be easier or harder than average. And knowing how a player’s teammates did before their shot helps, too. A bunch of misses will tend to make that final shot more difficult. You can never get to the point where you know if a draw to the pin is needed, but you can get closer to the true difficulty.

It’s that approach that has led me to something I am calling “points over expected” for now. (I’m open to suggestions for a flashier name.) Basically, take the average difference between the points awarded and the expected points on each shot thrown. With this approach you can more fairly compare the contributions from players at different positions. You can more fairly compare how players are doing on certain shots or certain turns. The possibilities are endless.

Here’s the leaderboard for my yet-to-be-named stat based on all shots thrown at the women’s worlds through Monday:

    Switzerland +0.276 6-0
         Sweden +0.173 5-1
       Scotland +0.097 4-2
         Canada +0.087 2-5
          China +0.084 4-2
            RCF +0.082 6-0
          Japan +0.032 2-3
  United States +0.025 4-3
        Germany -0.040 3-4
        Denmark -0.095 2-4
          Korea -0.112 2-5
 Czech Republic -0.179 2-4
          Italy -0.205 1-6
        Estonia -0.239 1-5

With this data, we can have a more interesting conversation about what’s happened so far.

There are two unbeaten teams, but one of them has played much better than the other. Switzerland has been a machine so far. When we look at the individual player ratings by this method, the Swiss have been outstanding at each position. Melanie Barbezat has been far and away the best lead at the worlds and Esther Neuenschwander has been the best second. Silvana Tirinzoni has been the second-best third (Japan’s Onodera Kaho is the surprising leader there) and Alina Pätz has been the second-best fourth (to Eve Muirhead).

The Swiss will come back down to earth a bit. I mean, if they don’t, they probably won’t lose a game, and that doesn’t seem realistic. But it’s clear that Switzerland has been the best team so far.

The order of the unnamed stat lines up pretty well with the overall standings with the obvious exception of Canada. Whatever the issue is with Canada’s poor start, it’s not that they’ve played poorly. They haven’t played like the best team in the world, but they’ve played like a playoff team. Late-game rock luck hasn’t gone their way to a large extent. Exhibit A is Kovaleva’s second-to-last shot in the eighth end that propelled The Federation to a comeback win.

Can Canada recover to make the playoffs? On that question, it seems everyone is under the impression that Canada can’t lose again if they want to finish sixth, but I think it’s more likely than not that if Canada finishes 7-6, they are in the playoffs. For one thing, there are only six teams over .500 right now. And if Canada is winning games it means they are beating some of those teams trying to claw their way to 7 wins.

Also, I’ve been holding out hope for Korea – who at 2-5 has the same record as Canada – to make a run, as well. But based on expected points, the Koreans have played like the 11th-best team thus far. So their situation is quite a bit different from Canada. They don’t have much room for error and they have to play much better the rest of the way.

Much of Canada’s fate depends on the U.S. since Canada would lose a head-to-head tiebreaker to the Americans due to an earlier loss. The U.S. has already played Estonia and Italy and has not played the powerhouses of Switzerland, Sweden, or The Federation, which is just the opposite of Canada’s schedule. Canada will be favored in every game the rest of the way and while that’s a long way from saying they’ll win every game the rest of the way, they might not even need to do that to make the playoffs.

Postscript: I hope to publish shot-by-shot data with the expected points derived from my method for the men’s and women’s worlds (and other past events) in the near future. There are many more potential uses for it than just identifying which team is playing the best.

In the meantime, here’s what an 8-ender looks like. Notice Denmark’s shot difficulty falling into the abyss as they continue to miss late in the end, while Switzerland’s shots are somewhat easier as they continue to draw freely into the open rings.

User comments
  1. D · 2021.05.04 · 4:25 am

    I’ve always been really interested in shot data and I’m looking forward to seeing what else you’ll do with it! I believe it was the Slams who used to (or still?) assign difficulty scores to shots, along with percentages, but I’m not exactly sure how those were calculated. Unfortunately, a lot of players still don’t believe in shot data because “statisticians are just volunteers who don’t know what they’re doing” :/

    1. kenpom · 2021.05.04 · 4:55 pm

      I did not see that data for the Slams this year, but I’ve seen it in the past on CurlingZone. They also have sweeping numbers as well.

      I have seen some wonky scoring decisions (more so in the Slams than the Worlds) so I can sympathize with players who don’t want to buy into the data. But over many shots there’s some meaningful information there.

  2. Derrick McLean · 2021.05.04 · 9:51 am

    This is a very interesting approach to assign shot difficulty without having a second variable input and then designing a weighting system thereafter. How is the shot expectancy actually calculated for a shot in the middle of the end (say, shots 7 and 8 in the figure above)? I can’t tell if it is a regression based or Markov Chain approach or something entirely different. Are you just looking at the previous two shots or across the entire end? Also, what does Chc(4) represent?

    1. kenpom · 2021.05.04 · 4:33 pm

      Chc(4) and Chc(0) are the chances of scoring 4 or 0 on the shot. It is a regression for now, though I’ve though about trying Markov Chain.

      For shots in the middle of the end, it’s mostly based on the average for that shot type and that shot. Take the average of all draws at shot 7 and boom, that’s your expectation for a draw at shot 7, or at least close to it.

      This is not true for every shot, though. Peels are pretty much the same shot no matter when they are thrown, so the average for a peel is taken over all shots.

    2. Derrick Mclean · 2021.05.05 · 11:46 am

      Great stuff. I would suspect some level of correlation between the previous shot outcome and the next shot outcome. I’m not sure exactly how your model is set up, but lagging the previous shot score as another variable could be a quick way to test for this autocorrelation. If you are using multilevel modeling (parsing out games and ends as nested variables), then you might be able to detect autocorrelation (even the simple AR(1) effect) and further validate the expected score parameter.

  3. Josh · 2021.05.04 · 10:35 am

    The shot you shared the video of was truly amazing. The commentators really didn’t think she could make it (and neither did I), but once she did it seemed like Einarson got a little chuffed and thought, “If she can do it, so can I,” even though it wasn’t the best choice.

    One time at a men’s five and under spiel in the Northeast, a friend had me score their team’s game via an app. Needless to say, the scores were not that great, and the friend wasn’t too happy with me after the game, but I don’t know what else he expected… we were all five and unders. *shrug emoji*

    1. kenpom · 2021.05.04 · 4:24 pm

      Man, I’d love for somebody to score my games, fully accepting that my percentages would be in the 20’s. At least I’d have a way to measure my progress.

  4. Nuclearbdgr · 2021.05.05 · 11:03 pm

    Ken, thanks for digging into this. Is there objective criteria for the score (e.g. what makes a 3 a 3 and not a 2)?

    1. kenpom · 2021.05.06 · 12:50 am

      Oh yes…right here

      Although that document is only for WCF events. Canadian events are a bit different, and in my experience with the slams, the attention to detail is not as strong. In the worlds, a peel attempt to blank an end is always labeled as ‘clearing’. In the slams, it could have been any one of ‘clearing’,’hit and roll’, or ‘takeout’ depending on the scorer. The slams scorers also were generally more forgiving from what I saw. All of that complicates my efforts.

Join the discussion!