Deep Analysis: Comparing Metagame Methodology

The plan for this week was to update my metagame data tables with the results of the Grand Prix. However, Jason surprised all of us with the actual metagame update. Not that I’m complaining, mind you; it was a very nice surprise. I’m glad the more rigorous and comprehensive table is back in action. In light of this development, we can do something unusual: compare results. My article series focused only on the results of high-level paper tournaments in the United States. Jason has those results, and a host of others, providing an opportunity to see if my focused data accurately reflects the metagame at large. It will also provide an excellent example of how perception of data can affect reaction to the data.

To be honest, I’m actually rather relieved. The results of GP Oklahoma City are unusual, and would have really warped my metagame projection. Don’t misunderstand me; it is important and useful data. But the temptation would have been to read deeply into the results. By the time the GP rolled around, big-mana decks had mostly dropped off to inhabit the lower end of Tier 2. Even if I were aggregating them, they would have only been mid-Tier 2. No, Eldrazi Tron isn’t really big-mana; it’s more stompy than ramp. A resurgence, especially to this degree, is definitely worth looking into.

However, it has to be taken with a grain of salt. Wizards is not reporting the Day 2 data, without which we don’t know if this phenomenon was a function of population or actual success. If non-Eldrazi Tron and Valakut decks made up most of Day 2, of course they would dominate the finals. If that wasn’t the case, there are number of interesting possibilities. As it stands, it’s best to evaluate the result in a vacuum. As I said, I’ll come back to the GP later. For now, let’s look at the two metagame projections.

Comparative Analysis

When comparing anything, it is important to compare relatively similar things. No matter the quality of the apple, it makes a poor orange. I would like to have similar data to compare, but that really isn’t possible. I was collecting the Top 16 results of high level events while the full update has many levels of competition. My data is also primarily US-focused while Jason has expanded our metagame sampling overseas. By their nature, the two sets measure different things, and a true side-by-side is disingenuous.

Instead, their differences should be embraced. The metagame update should be regarded as a reflection of the “true” Modern metagame, or as close as can be accomplished. My data is a specialized subset, measuring high-level success within the greater metagame. This analysis will inform us of how well that data reflects reality, and what it may imply about the set itself. Oftentimes this method reveals hidden quirks and other interesting discussion points. For reference, my final table with the Roanoke Open results will be posted below. The total metagame is in the sidebar on the right of the screen and in the Top Decks page. Or go read Jason’s article, which features plenty of links.

DeckTotal
Affinity22
Jeskai Control20
Gifts Storm14
Grixis Death's Shadow14
Eldrazi Tron11
Counters Company9
Humans8
Infect7
UW Control7
Burn6
Abzan5
UG Merfolk4
Bant Company4
Jund4
Elves4
Titan Shift4
8-Rack4
GB Tron3
Ad Nauseam3
GW Company3
Death and Taxes3
Mono-G Tron3
Mardu2
UR Breach2
BW Eldrazi2
GR Ponza2
BW Eldrazi and Taxes2
Titan Breach1
Saheeli Evolution1
Temur Aggro1
Knightfall1
5-Color Death's Shadow1
GR Devotion1
RW Prison1
Bant Eldrazi1
Abzan Company1
Grixis Control1
Living End1
Skred Red1
Bogles1
4-Color Company1
RG Vengevine1
GR Tron1
Naya Company1
4-Color Knightfall1
GW Hatebears1
Grixis Delver1
4-Color Death's Shadow1
Dredge1
Mono-Red Prison1
4-Color Control1
Enduring Ideal1
Lantern Control1
WG Taxes1
Mono-Blue Merfolk1

Similarities

First, our top 3 decks are the same. Affinity is even on top by a decent margin. It would be higher by roughly three decks if I’d done what Star City doesn’t and separated Jeskai Geist/Tempo from actual Jeskai Control. That would have separated Affinity, but not dropped Jeskai from second place. In fact, Tier 1 for both sets is very similar. Seven of nine top decks are the same, even if the order is different—the order’s a relatively small consideration, reflective of noise in my sample that the weighing system seeks to eliminate for the metagame update. But that’s where the similarities end. While the Top Level Tier 1 accurately reflects the international metagame, the rest of my set deviates wildly.

Differences

The most notable variation affects Burn and RG Valakut/Titan Shift. Burn is the fourth-place deck in the metagame, but Tier 2 in the Top Level. Looking back, Burn is consistently Tier 1, but never gets any respect at high levels. This disdain is very counterintuitive, as you would think that players would gravitate towards winning decks. Dissecting why Burn sees so little play at high levels is an article unto itself, but I will note that it catches some undeserved flak for being a “kids’ deck.” Valakut is at best at the bottom of Tier 2 in my sample, yet it made Tier 1 by a decent margin. It’s particularly glaring considering that it won. In its place, my data has Infect and UW Control, both of which are actually Tier 2.

My mid and lower tiers are very different from Jason’s overall. Some of that may be my splitting of Tron variants, but even with that, many of Jason’s lower-Tier 2 and 3 decks only appear as singletons in my data. My Tier 2 decks tend to appear as Tier 3 in the metagame. The simplest answer to these variations is that my data set isn’t big enough to accurately reflect the metagame. As I’ve said before, the more data you have the less likely you are to miss something small but important, or to mitigate the effect of outliers. That said, the update looks at the total metagame, including international and online results, while my data is mostly SCG circuit results. The populations are very different and may not reflect each other well.

Meaning

If you are only going to examine high-level results, you will miss a lot of the metagame. They simply don’t capture the whole picture. However, they do accurately reflect the top of the metagame. Consider this: the same decks that reach the finals of an Open or GP are likely to perform throughout the meta. Each high level tournament is a grueling fight through 15 rounds of high-level competition. The overall metagame is built by decks succeeding in the face of many levels of competition. If a deck can rise in the face of constant pressure from all sides in the total metagame, it *should* do similarly well at a given competition. It’s also worth noting that when a deck is doing well it does attract players, which will boost its numbers. More players equals more visability.

Of course, anything can happen in a given tournament. That’s why you should not draw broad conclusions from any individual result. You need multiple data points to draw a meaningful conclusion. That’s why I waited until mid-November to really make statements about the metagame and the official update uses so many different events. This also highlights a flaw in my methodology. While Tier 1 was very similar, the other tiers were far off. This is expected. The lower tiers are by definition both less popular and less successful, meaning if you’re only using the highest results, they’re less likely to appear. When you cast a wider net they have more opportunities to appear. Therefore, my data is valuable for tracking Tier 1, which in turn determines much about player’s reactions to the meta. Let’s be honest, nobody metagames against or reacts to Tier 2 or 3; all eyes are on 1. This can be important for predicting how the metagame will evolve, but not to show what the overall metagame is doing.

GP Oklahoma City

Now, on to current events. Since Titan Shift and traditional Tron variants dominated the Top 8, we may learn something from their metagame positioning. The overall Top 32 looks far more like the metagame we’ve been reporting, featuring lots of Jeskai variants. I would guess that the Tron pilots expected to face plenty of Jeskai, and as a bonus got the gift of the many Lantern Control decks that coverage claimed were present. There’s not really any evidence of that in the data. Only two Lantern decks made Top 32, but it wouldn’t surprise me if there had been a lot of Lantern players. Sam Black hyped the deck and then played it at the Invitational. Some have even called it the best deck in Modern. Then, proponents discovered that Tron was a horrible matchup. Titan Shift isn’t much better. If there really was all that Tron food running around, I’m not surprised by the result.

What I’d Have Brought

As noted, the metagame has been defined by decks with poor Tron matchups over the past month. I couldn’t go to the Grand Prix, but if I did, I would have played Jeskai Geist with big mana in mind. I could see the trend as well as anyone, so I assumed that players would gravitate to anti-Jeskai lists. 8-Rack had been the weapon of choice at the SCG Classic, but that deck is soft to an open metagame. Going that deep is too risky, but Tron and Valakut are fine decks in their own right, and now they’re well positioned. That’s a perfect example of repositioning metagaming.

I had intended to reposition my deck around the big mana decks. Rather than go with the recent plan of Ceremonious Rejection and Disdainful Stroke, I’d planned to go for the cheap win.

Jeskai Tempo, David Ernenwein (Test Deck)

Creatures (11)
Snapcaster Mage
Spell Queller
Geist of Saint Traft

Sorceries (4)
Serum Visions

Instants (21)
Lightning Bolt
Path to Exile
Spell Snare
Lightning Helix
Logic Knot
Electrolyze
Cryptic Command

Lands (24)
Celestial Colonnade
Arid Mesa
Flooded Strand
Steam Vents
Hallowed Fountain
Spirebluff Canal
Sacred Foundry
Island
Plains
Mountain
Sideboard (15)
Blood Moon
Vendilion Clique
Supreme Verdict
Relic of Progenitus
Negate
Stony Silence
Elspeth, Sun’s Champion
Celestial Purge
Buy deck on Cardhoarder (MTGO)Buy deck on TCGPlayer (Paper)

That’s right: three Blood Moons. Normally it’s not a great card against Tron because they can just keep making land drops, but Jeskai can really apply pressure. The fact that it randomly hits a lot of decks is gravy. It can be painful for Jeskai too, but in the matchups where we’re a massive dog, it’s just what we need. That’s also why I have Arid Mesa instead of Scalding Tarn. In those matches I have to find my Plains. We’re likelier to naturally draw Island and we need white mana to cast the important threats. Given the choice between fetching Island and Plains when I had Blood Moon in hand it was always better to get Plains. Hence, extra ways to find them. I would probably not recommend anything this extreme going forward, but I stand by the plan for Jeskai mages looking specifically to beat Tron and Valakut.

Looking Forward

While metagame trends can be identified, actually using them is tricky. A lot of players saw the trend towards interactive fair decks and Affinity and away from goldfish styles and reacted. I anticipated big mana being a force but didn’t predict the (alleged) Lantern surge. The Lantern players were ready to crush the interactive decks, which is what they do, but not the big mana decks. Ramp, in turn, was ready for anything but Storm. I speculate that was the key to its success. It isn’t as strong against one thing as a dedicated metagame deck, but it’s powerful enough on its own merits that if it didn’t hit its awful matchups, it didn’t matter.

I believe this strategy will be the norm going towards the Pro Tour. The trend of Jeskai, Death’s Shadow, Storm, and Affinity being the top decks is likely to continue, but now players are reacting to this trend and will try to take advantage. More decks that don’t fit the trend will rise, weakening said trend. What happens next is impossible to forecast. Whether this development occurs before or after the Pro Tour will prove interesting to watch.

David began playing Magic during Odyssey block, quit playing Magic when Caw Blade ruled the world, and returned to Modern shortly before Deathrite was banned. He’s made an appearance at the Pro Tour, made money at GP Denver, and is constantly grinding and brewing in Modern.

Leave a Reply