The following guest article is written by Jeroen Aga (Geronimoo on Isotropic).
Caravan/Mountebank v. Warehouse/Mountebank
This is a response to the first annotated game on theory‘s blog. That game raises an interesting question: on a board that seems to revolve around the Mountebank, should you go with yaron‘s Caravan/Silver opening, or theory‘s more aggressive Warehouse/Silver?
Here’s what theory had to say:
…Warehouse also cycles my deck, letting me repeatedly play my Mountebank. I think it’s a pretty critical card in sets like these that lead to decks filled with crap but with a sprinkling of a couple crucial cards.
Then he goes on:
Caravan is the best $4, but it’s barely better than the $3′s. Although it’s ordinarily a great card, easily spammed, its utility is limited in this set, since the extra card is going to be pretty bad most of the time.
However, a keen observer in the comments had these arguments in favor of the Caravan:
What happened to you in the game is the absolutely best draw available. Not only did you trigger the reshuffle at Turn 3, the 4 cards that missed the shuffle are all Coppers and Estates.
…
Consider other possible draws. It would be bad if you draw your Warehouse on Turn 4 (since your Warehouse now misses a reshuffle, as well as the things you buy that turn), or in your Warehouse hand you happened not be able to draw your Silver. It’s easy to see over half of the time the bad things can happen, and that is why I won’t recommend opening Warehouse+Silver.
So it remains an open question. I decided I was going to find out and I’m inviting you on the journey. If you’re not the adventurous type, just skip to the bottom of this post and you’ll find a neat-looking graph with the final result. Bye to you, skippers!
Hello to you, fellow journeyman! First some introductions; I’m a software engineer for a big insurance company and have been on the bench lately (no more budgets, crisis, or just idiot managers, who knows). So I wrote a Dominion simulator to keep me busy. After a few hours of keyboard molestation (my colleagues giving me the stare, “what the hell is he working on so furiously???”) I had my first few thousand virtual games running (Smithy wins against the Village Idiot!! Wheeee!). The computer was making some awful plays though, so I found help at the Simulate Dominion blog. Their “rule based algorithm” was exactly what the doctor ordered and my simulator was soon firing on all cylinders.
At this point theory had published his first annotated game and I knew I had a project to spend some quality bench time on.
Single card strategies
The first thing I did was try out the Warehouse strategy by letting it play against Big Money (this guy completely ignores the Kingdoms and only buys treasures and green cards).
The Warehouse strategy looks something like this:
Buy Rules (Warehouse): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Gold -buy a Warehouse (if you've bought at least a Silver) -buy a Silver -buy nothing
So how does this work??? The simulator just goes through these Buy Rules top-to-bottom and does the first thing it is capable of and then stops reading. If he has extra buys he starts again from the top. The play rules are also simple:
Play Rules (Warehouse):
-play Warehouse
-discard Warehouse(s) if this allows a Province buy
-discard the worst cards
If the simulator has extra actions he will repeat (so extra Warehouses will be played unless this would prevent a Province buy). This is not an optimal discard strategy but it probably isn’t far off.
For reference here are the buy rules for Big Money:
Buy Rules (Big Money): -buy a Province (if you've bought at least a gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Gold -buy a Silver -buy nothing
Big Money has no Play Rules of course.
I fired up the simulator and let it play some games; 10,000 to be exact, which would take a human player about a year to finish but only 3 seconds for my creation. A little to my surprise, Warehouse got completely crushed by Big Money:
This graph shows the amount of money ($) a player has available each turn
The Warehouse player has serious cash flow issues while the Big Money player races to Province-buying territory. I soon realized discarding crap with Warehouse isn’t so great when your draws are equally crappy. I toyed with the Warehouse player’s buy rules for a bit and found these buy rules to be optimal:
Buy Rules (Warehouse): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Gold -buy a Warehouse (if you've bought at least two Golds) -buy a Silver -buy nothing
Surprisingly this strategy buys his first Warehouse when the game is already in the end phase. Now the Warehouse player wins 50% of the time while the Big Money player only wins 41% (9% are ties):
Warehouse outbuys Big Money from Turn 8
On with the Caravan! Here are the optimal buy/play rules (I won’t bother you with my first feeble attempts):
Buy Rules (Caravan): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Gold -buy a Caravan (if you've bought at least three Silvers) -buy a Silver -buy nothing
Play Rules (Caravan): -play Caravan (well d'uh)
So opening with Caravan/Silver was really bad and you should open with Silver/Silver for best results. This was quite surprising, but then I remembered all those times I opened games on isotropic with Caravan/Silver and got stuck buying Caravans for several turns. The problem is very similar to that of Warehouse: card advantage is great but not when you’re drawing mostly crap.
Now we’re getting to the fun part! Here’s the result of the Caravan-Warehouse clash:
This graph shows the average VP (victory points) gained each turn
Warehouse comes out a small winner (3%). This illustrates how Warehouse’s pseudo-trashing trumps Caravan’s pure card advantage.
Let’s throw the big baddie into the mix: the Mountebank. First we’ll try the single card strategy (buy only Mountebanks/treasures/VP’s).
Here are its buy/play rules (borrowed from simulatedominion.wordpress.com):
Buy Rules (Mountebank): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Mountebank (buy maximum 2) -buy a Gold -buy a Silver -buy nothing
Play Rules (Mountebank): -play Mountebank
As most people know Mountebank is better than Gold, so the simulator will often pay $6 for a Mountebank. This simple strategy is brutally effective: it beats Big Money 9 games out of 10:
Big Money starts out ahead, but then gets overwhelmed by Curses and Coppers.
Mixed card strategies
Now let’s enhance the Mountebank strategy by adding the card drawers. First up is the Caravan. Again, the buy/play rules:
Buy Rules (Mountebank-Caravan): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Mountebank (buy maximum 2) -buy a Gold -buy a Caravan (if you've bought at least one Silver) -buy a Silver -buy nothing
Play Rules (Caravan-Mountebank): -play Caravan -play Mountebank
This strategy has no problem opening with Caravan/Silver; the pure Caravan strategy needed to open Silver/Silver to get to Gold quickly enough. The goal for this strategy, however, only needs to get to $5 as soon as possible. This strategy wins 93% against Big Money’s 5% (2% are ties).
Switching Caravan for Warehouse results in these buy/play rules:
Buy Rules (Mountebank-Warehouse): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Mountebank (buy maximum 2) -buy a Gold -buy a Warehouse (if you have more Silvers than Warehouses) -buy a Silver -buy nothing
Play Rules (Mountebank-Warehouse):
-play Warehouse
-discard Warehouse(s) if this allows a Province buy
-discard the worst cards (this includes extra Mountebanks)
-play Mountebank
Again the need to get to $6 isn’t that great, so you can open with Silver/Warehouse instead of Silver/Silver. I had some trouble finding an optimal yet simple buying strategy but eventually settled on alternating between buying Warehouses and Silvers. This wins 94% against Big Money’s 5% (1% are ties), so it has a tiny edge over Caravan-Mountebank in the Big Money matchup.
Finally we come to the big fight:
Drum roll…..
Caravan/Mountebank outbuys Warehouse/Mountebank from turn 10
What the deuce?! Caravan wins 51% to 44%??? I didn’t believe these results and upped the number of simulations to 100,000 games but with the same result. Than 1,000,000 but Caravan kept its edge. I considered 10 million games, but waiting a whole hour for the same ugly numbers wasn’t worth it, so I went back to the drawing board.
I played around with the Warehouse Player’s buy rules and ultimately cracked it:
Buy Rules (Mountebank-Warehouse): -buy a Province (if you've bought at least a Gold) -buy a Duchy (if there are only 5 Provinces left) -buy an Estate (if there are only 2 Provinces left) -buy a Gold -buy a Mountebank (buy maximum 3) -buy a Warehouse (if you have more Silvers than Warehouses) -buy a Silver -buy nothing
The new Warehouse outbuys Caravan in the early turns.
The extra Coppers and Curses from the opponent’s attacks severely hinder your buying power and getting some solid money in your pile is more important than being able to attack back as soon as possible. Buying Golds before Mountebanks gives the edge back to the Warehouse Player and enables it to get a head start as you see in the graph.
I tried the same changes for the Caravan Player but buying Golds before Mountebanks really hurt its win rate (43% to Warehouse’s 53%) . The explanation is probably that the Warehouse allows you to dig deeper for those Mountebanks so you can afford to buy them later.
In conclusion I declare the Warehouse/Silver-opening the winner over Caravan/Silver…. but it’s only by a nose-length (or even a pimple’s length on said nose).
But that’s not all we learned:
- Silver is often your best buy in the early game
- Editor’s note: I believe this is true only when pursuing Big Money-type strategies. More complex Action chains tend to benefit more from non-terminals like Haven or Great Hall once you get to Gold.
- Card drawing is only good when you can draw into something useful
- Warehouse is really strong in the absence of trashing cards
- An opponent’s strategy might force you to change your winning strategy
See you on Isotropic (my nick’s Geronimoo)!
Dominion Strategy 
That’s really interesting, thank you for sharing. I was wondering if it would be too much to ask to see the code you’re using for your simulations – I didn’t see it at the simulate dominion blog. If you used python or vba or something simple, I’d be interested to fiddle with some of the rules. Though if you’d rather not share the code, I still really appreciate you sharing the results!
The post is great. Did you consider trying to add some tactical play of the warehouse? Like, if you have one mountebank in hand, and you know one is in the deck, don’t play the warehouse.
Awesome analysis! I wish I could be part of these kinds of projects…
But one question comes to mind: *Should* you always play a first Warehouse? (I think your Warehouse play rules say so, but I’m asking if that is optimal.)
Say I draw Warehouse + any 4 cards that give me $8+. Strategically speaking, is it always worth playing the Warehouse in order to cycle past any Coppers and Victory cards after considering the chance that I might also draw unnecessary Golds (which the computer should be able to calculate)? It seems like there could be situations where it might be better to not play a Warehouse. Maybe when you have 1-3 cards left in your deck and you know one of them is a Gold, or another Warehouse, for example.
Or are you trying not to consider things like deck-tracking? That would be understandable, since it would make the strategies involved “harder” for humans to abide by.
Sorry – that comment should have been top-level, not a reply to rrenaud!
If you want to repost as a top level response, I can use the admin tool to delete the sub thread about the mis-parenting, but I can’t actually re-parent your post. Argh wordpress ;(.
Not sure if you were just simplifying the logic that’s actually implemented, but does this mean that a 4/3 opening would be Silver/Silver but a 3/4 opening would be Silver/Caravan?
I love this analysis. But it is getting way to close to the calculations I do at work, which makes it less fun. Just don’t calculate a correlation function, and we’re good.
Less facetiously, I think there may be an under-appreciated value for warehouse’s ability to support more mountebanks. I wonder if they simulation bears this out.
Hasn’t this webpage consistently advocated against junking your deck with too much silver? What if your silver buy rule was to buy unless gold >= 1? Wouldn’t that improve the rule-based strategies?
I assume ppl are working on bots to run on isotropic? These rule-based strategies would be a good start. How about some coding magic rrenaud, so we can have a simple language to describe rules, and it creates the bot for us? 🙂
If I was going to do a Dominion AI, it would be more sophisticated. I’d try to do some machine learning rather than the manually specified strategy style that is so prominent on BGG. But I am currently pretty happy just hacking on dominion statistics stuff.
A Dominion machine learning project would be AWESOME.
I hear that Dominion is IBM’s next project, now that they’ve locked up Jeopardy.
Excellent work and great to see.
Just one point going forward. Adding more simulations won’t change the result unless something is very wrong.
Each simulation is essentially a single sample from the population of all possible samples. Excluding ties this is basically a Bernoulli trial with a win probability of p for one of the players (and 1-p for the other). The highest variance is if the probability of a win is exactly equal and is .25 in that case. You can construct a reasonably small margin of error with even 1,000 tries. The margin of error should be less than 1% with 10,000 tries. This isn’t the exact case because the real distribution is really a multinomial but the principle is not much different.
Stick with 10,000 tries and modify your approach as you needed. I think what you are doing is great and the graphs are wonderful.
However, I would add a very crucial caveat. You need to be very careful to draw conclusions from this data. You are drawing some very broad conclusions without really addressing the real issues. How do we really know Silver is the best buy? What evidence do you have that card drawing is not good unless you have things to draw into? These things might be true but you really need to confirm your results with structured experiments and even then you have to be careful with your conclusions.
– Drew
@jambarama:
The code was written in JAVA. I might publish it some day, because it doesn’t contain any ground breaking stuff.
@rrenaud and @Bulb:
I tried to keep the strategies simple for 2 reasons: 1) to avoid mistakes and bugs and 2)If I let the simulator use all the information he has available he might start playing better than a human player. For example I could let the computer calculate the probability of drawing extra Mountebanks with Warehouse when he has one in his hand already and let his decision to play the Warehouse be based on that, but no human player is capable of doing that stuff in the middle of a game. So if my super smart simulator wins with Warehouses, a human player might not have the same win rate with that Warehouse buy.
@zzz:
I tried to keep the buy rules easy to read so they might not be as complete as the actual JAVA-code
@ksf:
I experimented with more Mountebanks in the Warehouse strategy, but more than 3 Mountebanks resulted in a worse win rate
@cleanest:
The strategies described in my post are all enhanced Big Money strategies so they will always benefit from more Silvers. Once you start with King’s Courts and other shennenigans, Silver is a liability.
@Drew:
I knew of course that extra simulations wouldn’t alter the results that much. Just trying to spice up the article a bit. And you’re absolutely right about drawing conclusions without real evidence… I’ll try to avoid that in the future.
@everyone:
I originally started this project to make a bot that can actually play real Dominion, but it proved way too complex. I’ve had some success using Genetic Algorithms for a Magic:the Gathering Automated Sealed Deck Builder, but Dominion is too different to apply the same logic.
I disagree that a human can’t keep track of how many mountebanks are left in his or her deck, and decide whether or not to play warehouse accordingly. It’s even pretty easy to count your deck and work out the exact odds on the fly. Anyone trying to be good at Dominion should be thinking about what’s left in their deck and whether or not drawing cards is a good idea in every game.
If you’re really able to track what’s left in your deck past the early turns, you’re quite the genius, but I doubt it will have the same impact on your win rate as making better buys. If one of the lvl40+ on isotropic could comment on that? Are you guys really tracking your whole deck during game play or just making better buys than your opponents? Or both?
I’m not level 40+ (I’m within 5 spots of you, actually), but I keep pretty good track of what’s left in my deck. In early turns (oh, the first 10 or so), there’s a good chance I know exactly what’s left. I definitely keep track of “important cards” so I would know exactly how many mountebanks are left, even in late game (at least before all the curses are out). In more complicated decks, like those bombarded with curse-givers and masquerades, I will definitely lose track. And I keep track of VP really well in the vast majority of decks, but I probably couldn’t tell you if I’ve drawn 3 provinces and an estate or 2 of each – I only keep track of how much green I’ve drawn.
I guess my point is that human players aren’t going to have exact percentages, but we can get pretty close approximations.
I’d have to agree. Keeping exact odds obviously gives a computer an advantage, but if I have only purchased 1-2 Mountebanks, it is very easy to know whether or not I still have one in my deck.
At any rate, I am curious as to how much stuff like this even matters, in general, and this would be a good chance to demonstrate. Obviously I have no idea how hard it would be to code, but if it is relatively easy, I think it might be interesting to see how the win rate changes if you prevent “guaranteed” drawing of good cards with Warehouse when you already have $8. Maybe it only improves its win rate by 0.1%, but maybe more… 🙂
There is a difference between being able to calculate the odds perfectly and being able to have some general deck sense to make informed tactical decisions.
While you aren’t going to beat the computer at straight up computing odds, you can still outplay it because it has to make the right decisions based on the information, and good decision boundaries can be complicated.
A human with imperfect information can still play better than a somewhat poorly implemented algorithm with perfect knowledge.
One of the things I sort of have a problem with is that you seemed to keep going with your trials until they told you what you seemingly already knew; this is, of course, somewhat bad science, BUT on the other hand, I can’t say it’s totally invalid if you’re really sure that one is better than the other. It’s just that there are a couple of caveats: 1) it doesn’t seem THAT clear to me that this warehouse strategy is clearly better than a warehouse one (and a mix between the two may be even better); 2)if you’re doing this to tell you something you already know, the simulations are no longer justifying you’re feeling; rather, you’re justifying the correctness of your simulation by showing that it gives results that we already know to be true.
The whole point of this exercise was to get a better and more precise understanding of the power of certain cards. Everyone who’s played a few games knows Warehouse will be great if there aren’t any trashing cards, but how great??? I think these simulations indicate it will be on par or even better than Caravan…
Maybe I didn’t state it clear enough, but Warehouses+Mountebanks is clearly NOT the best possible strategy on this board, just a tiny bit better than Caravan+Mountebanks…
For starters I’m guessing combining the Caravan and Warehouse will probably be stronger than both single minded strategies.
I think the worst part from a “scientific” perspective is that you stopped iterating the changes when warehouse was ahead.
In general, between two different families of parameterized strategies, where you fix one, optimize the other to beat the first, then fix the other and optimize the newly choosen member of the first family, etc, the family that you optimize last is going to have an advantage.
I did try to make the Caravan strategy better after it got beaten by the Warehouse (and I mentioned that in the article), but to no avail. If it bought more Mountebanks its win rate went down and when I changed the buy order of Mountebank and Golds its win rate also went down. The only thing that made it win again was buying Warehouses too, but that might be for my next article (advanced mixed card strategies 🙂
Sounds like we need some Machine Learning! 🙂