Kotei 2012/Emperor Edition Meta-Game Update #8

Quite a few Kotei this weekend, and a little bit of catch-up data too.  Very few changes this time – Crab continue to make the cut and win tournaments at a brisk clip, Dragon manage to win a lot without a noteworthy rate of making the cut, and Crane continue to flounder.

For a primer on the chi square standard residual, and its significance, check out last week’s updateBold entries indicated a better than 95% confidence that the results in question are – to use a technical phrase – out of whack (in fact, for the two applicable ones in this data set, Crab and Crane, it’s a 99% confidence).

Emperor Edition Environment
Players % of Field Made Cut Won % Made Cut Chi-Squared Standardised Residual
Crab 242 15.1% 70 7 28.9% 2.61
Crane 131 8.2% 14 0 10.7% -2.61
Dragon 166 10.3% 37 7 22.3% 0.31
Lion 205 12.8% 49 3 23.9% 0.84
Mantis 161 10.0% 34 4 21.1% -0.02
Phoenix 163 10.1% 40 2 24.5% 0.93
Scorpion 202 12.6% 41 3 20.3% -0.28
Spider 143 8.9% 20 1 14.0% -1.87
Unicorn 172 10.7% 31 2 18.0% -0.90
Unaligned 22 1.4% 1 4.5%
1607 337 20.4% 11.9

Making the Cut Rate Rankings
1) Crab – 29%
2) Phoenix – 25%
3) Lion – 24%
4) Dragon – 22%
5) Mantis – 21%
AVERAGE – 20%
6) Scorpion – 20%
7) Unicorn – 18%
8 ) Spider – 14%
9) Crane – 11%

11 thoughts on “Kotei 2012/Emperor Edition Meta-Game Update #8

  1. So if I understand the significance of the Chi Square numbers, while Crab and Crane situation is statistically certain to be linked to unbalance, Dragon actual dominance (relatively) could still be more of a freak event. And Spider poor results is almost certainly due to unbalance.

    1. Mostly yes, depending on what you mean by “certain,” but excluding the Dragon part. Technically, the figures means that we can say, with high (and statistically significant) confidence that Crab and Crane are not balanced with regards to making the cut (one can, of course, still argue with the underlying value of my making the cut metric). The Spider figures are, I think, strong enough for casual conversation (there is a 90% confidence). I don’t know if I’d call that last “almost certain,” but like I said, probably good enough for most casual conversations.

      With regards to the Dragon, I would note that, for the relevant statistics, there is not dominance. Dragon has a lot of wins. Dragon has not done much better than average at making the cut. The last column in the chart is exclusively about making the cut, and does not involve raw wins. So what the chart says about Dragon, if you take making the cut to be a measure of balance, is that the Dragon are balanced – anything less than a 1 means that the faction in question is less than one standard deviation away from balanced.

      1. What it potentially also says is that Dragon has good matchups against the other decks which make the cut. So although it is balanced overall it is the matchups which give it the edge in overall wins.

        I’m sure you could do something with the data to work out power rankings for any given clan vs any other clan. That should help in sifting out whether the wins above are matchup related or just oddities of the data/chance.

  2. So First I would like to say Thank you to all who put in the work to collect and post this.

    Now my question: your overall making the cut % is over 20… Uh how is this possible. If the floor rules are followed the highest possible percentage should be 19.9% and that is if every Kotei has an attendance of 21, 41, 81 or 161 exactingly. (OK technically if you had ALLOT of tournaments of less than 9 people it could be over 20%)
    Realistically it should be around 15%. So whats up?

    1. It sounds like you’re talking about a “traditional” cut (not that it’s really traditional, since it still doesn’t let as many people in as they did before Colson started screwing with the swiss structure several years ago) – which, to be fair, is all you’ll find if you look at the Floor Rules section on L5R.com, since they never updated that part from the CE Floor Rules (and, to be frank, I can’t seem to find the updated EE Floor Rules pdf on their website at all now).

      The EE Floor Rules permit an “X-2” cut, where (once the tournament hits 40something players) you play 7 rounds and all the 5-2 (or better) players make the cut. This is a substantially larger cut than the “traditional” method, and is used by most Kotei, which is why the make the cut rate is much higher than 15%.

  3. I think your Chi-Square numbers might be off. How did you calculate it?

    These are the numbers I get:
    Crab 1.74
    Crane -2.25
    Dragon 0.29
    Lion 0.64
    Mantis 0.03
    Phoenix 0.78
    Scorpion -0.15
    Spider -1.53
    Unicorn -0.64
    Unaligned -3.59

    1. First, exclude Unaligned entirely. Then, figure out what percentage of clan-aligned decks make it into the cut. Then for each Clan, figure out the expected number of players in the cut based on that Clan’s attendance. Then for each Clan (actual players in cut – expected number) / (sqrt(expected number)).

      1. I see; you’re not including Ronin. I suppose if your hypothesis is testing environment balance (sans Ronin), then you’re right not to. I think you should, regardless — especially, since Ronin (Oni) has been relevant in the past (and is more relevant today than most people think).

        1. With only 22 players (and one making it in), lumping the ronin decks back in only reduces the average rate of making the cut by ~0.23% Even if you include unaligned decks, it hardly budges the numbers, so there must be something else different you’re doing.

          1. You’re right. I had a slightly different, incorrect formula I was using.

            Regardless, I still think you should include Ronin — take it as you will.

          2. I disagree on adding ronin – for the purpose of statistics the null hypothesis is ‘all factions should have an equal chance of winning’. I don’t think there’s any DT intent that this is true for Unaligned players, so adding them doesn’t help prove anything.

Leave a Reply