Mathhammer - Taking Variance Into Account

A couple months ago I listened to an episode of The Best General that blew my mind and made me question how I managed my pregame and in-game expectations. Specifically, it was Episode 14, where Adam spoke with Eric Lofgren of Variance Hammer; Eric brought up that there is an inherent issue with looking purely at averages when evaluating units in Warhammer because averages rely on the law of large numbers, which overlooks the variance in our actual, on the table results.

For me, the idea of variance morphed into the question of just how accurate are our expectations when based on mathhammer? For example, I know the average result on 2D6 is 7 and that if I'm looking for 4+s I'll expect half of a given pool to pass and half to fail. What I don't know off the top of my head is how often those actual results occur.

A telling example can be seen when using "traditional" mathhammer for a series of rolls that don't average to whole dice. That is, if I have 10 shots hitting and wounding on 3s, math says I get 6.67 hits and 4.44 wounds. Then if my target has a 5+ save after modifiers, the average will be 2.96 unsaved wounds. Obviously you can't have fractions of a die, so I would generally round and expect 4 wounds with 3 unsaved. But, if we look at the actual possible outcomes we get a slightly different picture.

Starting with the to-hit roll: on 10 shots needing 3s, the most frequent result is 7 hits, which occurs 26.0% percent of the time. Meanwhile, 6 hits will be the result 22.8% of the time, and 8 hits will occur 19.5% of the time. Depending on this first outcome, it will obviously impact the to-wound roll expected outcomes.

Using averages, with 7 hits you'll actually then expect 4.67 wounds. But if you get 6 hits you'll expect 4 wounds, and 5.3 wounds if you got 8 hits. Looking at the actual outcomes, we can see that 4 wounds will be the result 24.1% of the time, with 5 being 23.1% of the time.

Finally, after saves, with 4 wounds, we'll expect 2.67 unsaved, or 3.33 unsaved if we got 5 wounds. Looking at probability again, we see 3 unsaved wounds is the most likely, 26.7% of the time, and 2 unsaved wounds nearly as likely, with that outcome occurring 23.8% of the time.

Here's a full table of possible outcomes:

Hits (%)	Wounds (%)	Unsaved (%)
7 (26.01%)	4 (24.09%)	3 (26.67%)
6 (22.76%)	5 (23.13%)	2 (23.76%)
8 (19.51%)	3 (17.21%)	4 (19.65%)
5 (13.66%)	6 (15.42%)	1 (12.54%)
9 (8.67%)	2 (8.07%)	5 (9.93%)
4 (5.69%)	7 (7.05%)	6 (3.48%)
10 (1.73%)	1 (2.24%)	0 (2.98%)
3 (1.63%)	8 (2.11%)	7 (0.84%)
2 (0.30%)	9 (0.38%)	8 (0.13%)
1 (0.03%)	0 (0.28%)	9 (0.01%)
0 (0.00%)	10 (0.03%)	10 (0.00%)

From these overviews, we can see that the most frequent outcomes line up with our expectation based on averages. However, it's important to note that the expected outcome is still only happening about one in four times, or half if you're expecting either 2 or 3 Unsaved Wounds.

That seems like a pretty small difference, but this is a game of nested odds. Many times we still have one or two more rolls to resolve; variable damage, and ignoring wound rolls. Let's say that our 10 shots were with an AP-2 Damage 3 weapon, targeting a Rhino. If I'm just averaging it I probably roughly expect 3 unsaved wounds (2.96) resulting in 9 damage. But, taking variance into account, we know that our expected outcome of 3 unsaved wounds only happens about a quarter of the time.

Building on this, an idea Nick brings up regularly in his articles on the Brown Magic is not being risky and not relying on the odds; don't let the dice beat you. Nick repeatedly talks about over-committing to something if it's important. That is, if his plan hinges on killing a unit in the current turn, he's going to commit more than what's needed just by math, to make sure he succeeds.

To help quantify this building in of redundancy, we can look for a confidence level for each outcome; that is, the percentage of outcomes that are a given result or better (I.e. 75% chance of scoring at least 3 wounds). Going back to our example before of shooting a Rhino with a S8+, AP-2, D3 weapon, I can say that at least 80% of the time, I will get 5 or more hits (92.3%), 3 or more wounds (89.4%), and 2 or more unsaved wounds (84.5%).

Using an 80% confidence level (or any level you choose) to estimate outcomes, rather than just the most likely result, can help to better allocate resources to ensure success. So, again, allocating on the Rhino, my 80+% confidence level for unsaved wounds with 10 shots is going to only be 2, resulting in 6 damage. If I absolutely need to kill that Rhino I should look to double the attacks I'm allocating at it, and use 20 shots to make sure I have a very likely chance of doing enough damage (88.6%). Using simple averages and mathammer, I may only allocate 14 shots, assuming that should be enough, but that reduces my odds to 63.3% that I will succeed.

To do all this calculation and comparison, I spent entirely too much time creating some Google Sheets that will run the math given a set of dice and rolls needed. I haven't incorporated any re-rolls, but if you would like to take a look (and I suggest making a copy for your own use), you can find that here.

I hope this was helpful. This has definitely been a bit of a rabbit hole for me as I analyze the numbers. Let me know what you think of the tool, and if you have any suggestions for improvements. Also definitely let me know if I messed something up, it's entirely possible.