Well, it’s been a while since I did one of these, eh? We’re back into a normal tournament season (after the “play random formats” for half of Winter Court 2013), and that means I get to go back to doing statistics. We’ll be following along all the L5R tournament action this Kotei season, tracking what Clans people are playing and how often Clans make the cut. And also, because people do rather care, how often the different Clans win.
For this first weekend of Kotei, “the Clan that’s winning” was obviously Crane, which completely dominated the universe. There were six Kotei. Three were won by Crane. Two were won by Lion (aka, the only Clan that goes first on Crane), including one after having to beat three of the four Crane who made up half of T8. And the other one was won by the same player who always wins that Kotei, and who doesn’t play Crane. Not only did Crane do extremely well, but they did well with a multitude of decks, including dishonor (sporting a lot of Scorpion Personalities), straight military, and straight honor. Other than “play Lion” it seems tough to meta against, especially since there isn’t much one can do to meta against “your Stronghold produces extra gold” or “your Sensei is really good and for some inexplicable reason doesn’t have a drawback, unlike every other Sensei.”
On the other hand, it’s one weekend. Which is especially important when we’re talking about statistics, because one weekend worth of Kotei is very far from statistically significant. Even more so where the total turnout of those Kotei was abysmal – that’s a topic for its own post/episode, but for our purposes here its worth noting because smaller Kotei are less statistically significant. I’ll highlight this by noting that, technically, Phoenix is currently in first place in my vaunted “Made Cut %” value, because only four people were willing to play Phoenix over the weekend, but two of them managed to make the cut. Maybe we’ll play another ten weeks and that figure will hold up, but I’d lay money it won’t.
Since the significance of some of the entries in these charts may not be immediately obvious, and this is the first one in a while, let me note what each column is. The ones on the left should be pretty obvious – the total number of players for that Clan, what percentage of the field was playing that clan, how many players of that clan made the cut (note: there’s no such thing as a “play-in” round anymore), and the number of tournaments that players of that clan have won.
The next column – “% made cut” – is the portion of the players of a particular clan made the cut. The last column is the most confusing. The problem with just “% made the cut” is that the percentage of players who make the cut fluctuates based on things like attendance and how that interacts with the qualification for someone to make the cut. For example, if you have at least 41 players, but not enough to do 7 rounds, then everyone who goes 4-2 or better makes the cut, and that’s a gigantic portion of the field compare to doing a 5-2 cut. Those sorts of tides raise everyone’s boat. That’s fine within an environment, but it’s a problem when comparing between environments. The purpose of the “relative chance of making the cut” figure is to be something that can be compared across environments – for example, was Embassy Crane more or less broken than Khol Wall or Warrens of the One Tribe (hint: much, much less).
Historically, I have expressed this figure as a percentage – zero would be average, with a positive number being a stronger performance and a negative number being a weaker performance. However, this has also historically proven difficult for folks to grok. So I’m trying out a different way of expressing the exact same number. Under the old system, if Clan X was twice as likely to make the cut as a member of the average clan, then Clan X would have a value of 100%. If Clan X was half as likely, they would have a value of -50% (never making the cut would have been -100%). Now, I’m doing to ditch the percentage, and normalize to 1. So I’ll take the average rate of making the cut (per clan), and that rate will be 1.00. A Clan that never makes the cut will be 0.00. A clan that makes the cut twice as often as the average rate will be 2.00. Please do let me know if you think this new representation makes the figure any easier to grok.
Ivory Edition | ||||||
Players | % of Field | Made Cut | Won | % Made Cut | Relative Chance of Making Cut | |
Crab | 24 | 11.0% | 1 | 4.0% | 0.18 | |
Crane | 46 | 20.3% | 20 | 3 | 43.5% | 1.96 |
Dragon | 29 | 12.8% | 4 | 13.8% | 0.62 | |
Lion | 30 | 13.2% | 6 | 2 | 20.0% | 0.90 |
Mantis | 24 | 10.6% | 2 | 8.3% | 0.38 | |
Phoenix | 4 | 1.8% | 2 | 50.0% | 2.25 | |
Scorpion | 26 | 11.5% | 7 | 1 | 26.9% | 1.21 |
Spider | 17 | 7.5% | 3 | 17.6% | 0.80 | |
Unicorn | 27 | 11.5% | 4 | 15.4% | 0.69 | |
228 | 49 | 21.6% |
Just above the chart, I made reference to the average rate of making the cut “per clan.” If you look at the bottom row, you’ll see that the total number of player is 228, with 49 making the cut. But if you do the math, you’ll see that the make the cut rate for the field is therefore 21.5%.
But if you look at the make the cut % for each of the nine Clans and average that, you will get a figure of only 21.0%. I call the first figure something like the make the cut rate per player, and the second figure the make the cut rate per clan. The rate per player is (in practice) always higher because players gravitate towards what are perceived as the best clans. If, for example, a tournament has one Phoenix player and ten Crane players, and 5 Crane players make the cut, then 45.5% of the players made the cut, but the Clan average was 25%. In my “make the cut” rankings below, the “AVERAGE” figure is this per Clan number, because I’m comparing Clans.
1 ) Phoenix 50%
2 ) Crane 44%
3 ) Scorpion – 27%
AVERAGE – 22%
4 ) Lion – 20%
5 ) Spider – 18%
6 ) Unicorn – 15%
7 ) Dragon – 14%
8 ) Mantis – 8%
9 ) Crab – 4%
By request, here are my current tallies for glory accumulation for the Renewal story prizes. Players is the number of players (1 point per player). Awards includes points for making and progressing in the elims and also honor contest points. Total is just that. Projected is a super-simple projection of how many points each clan would end up with if it continued to score at this rate. Note that this takes into account only how many kotei have been played and how many total there are. It does not include anything like adjustments for the expected size of kotei so far versus kotei to come, nor does it take into account the fact that the second half of Kotei season will be in a different environment.
Players | Awards | Total | Projected | |
Crab | 25 | 5 | 30 | 315 |
Crane | 46 | 61 | 107 | 1123.5 |
Dragon | 29 | 14 | 43 | 451.5 |
Lion | 30 | 20 | 50 | 525 |
Mantis | 24 | 8 | 32 | 336 |
Phoenix | 4 | 1 | 5 | 52.5 |
Scorpion | 26 | 18 | 44 | 462 |
Spider | 17 | 13 | 30 | 315 |
Unicorn | 26 | 4 | 30 | 315 |
Four Crane, One Lion, and one Scorpion win? I’d swear it was three Crane and two Lion.
Well, yes, that is what it says in the chart. Not sure why I got the number wrong in the text, which has now been fixed.
One thing that might also be interesting is projected points per clan from the Renewal storyline. Just a thought.
Wasn’t sure how helpful it would be to keep track long term if Dineen keeps up with the official reports, but I can add my scoring so far into the post.
I suspect that Mantis will soon surprise us all…
http://en.wikipedia.org/wiki/Normalization_%28statistics%29
I’ve updated these with the AEG tallies, which in some places contradict what the reporting from the event at the time was. Note that my “make the cut” numbers will generally disagree with the “official” ones, because those figures are only tracking exactly T8 or T4, and exclude other players who made the cut (for example, finished 13th and then lost in the T16 round)
It would be much more interesting to see the % of playtest group players that made the cut… That’s an elite clan that significantly impacts things much more than is visible to the average observer.