本文深入探讨了D&D游戏中英雄的成功概率预测机制，以及影响英雄选择的关键因素。通过分析不同“仙灵”的预测方法，揭示了运气、技能、以及预测者偏见对英雄最终命运的影响。文章还提到了数据生成规则，以及玩家在解决这一问题时所采用的策略，最终探讨了游戏设计和玩家分析之间的互动。

🧙‍♂️ 英雄的成功取决于六个隐藏变量（Int, Wis, Cha, Str, Con, Dex）以及计算出的“能力值”。

🔮 不同的“仙灵”使用各自独特的公式来预测英雄的成功几率，例如Amy看重运气，Bella侧重智力与魅力，而Holly则采用整体方法。

🍀 运气在预测中扮演重要角色，尤其对低能力值的英雄影响显著，但“仙灵”的预测也存在偏差，导致英雄选择出现选择性偏差。

🧑‍💻 玩家通过分析数据，识别出Holly的预测最有用，Colleen的预测存在冗余，最终成功解析了游戏中的预测机制。

Published on May 26, 2025 9:43 AM GMT

This is a followup to the D&D.Sci post I made nine days ago; if you haven’t already read it, you should do so now before spoiling yourself.

Below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted).

Ruleset

Skills and Competence

The success of a hero is decided by six hidden variables: Int, Wis, Cha, Str, Con, and Dex^[1]. These are used to produce Competence via the formula

Competence = 8Int + 9Wis + 10Cha + 10min(Int,Wis,Cha) + 11max(Str,Con,Dex) + 11Str+12Con + 13Dex[2][2]

Competence, Luck, and Success

The Challenge of saving a world is given by

Challenge = 1d20 + 1d40 + 1d60 + 1d80 + 1d100 + 1d200 + 1d400

If a hero’s Competence exceeds the Challenge, they succeed.

27.1% of generated heroes are Lucky. A Lucky hero, if they don’t succeed fairly, rolls a d8; on an 8, they succeed anyway through sheer force of Fate. This makes Luck a major factor for low-Competence heroes, but much less important to high-Competence ones.

The Fae and their Predictions

Amy

Amy believes fervently in Luck - particularly its ability to ameliorate otherwise hopeless situations - and nothing else. She predicts a 1% success rate for everyone who doesn’t have it, and a 99% success rate for everyone who does.

Bella and Liboulen

Bella (named for Peter Pan’s Tinkerbell) and Liboulen (named for Cleveland Quixotic’s Olliebollen) prioritize a hero’s mind over their body.

Bella is excessively cynical, and doesn’t care for Wisdom. Her predictions are given by

BellaPred = Cha+Int-1

Some bad experiences with over-clever heroes have left Liboulen falsely perceiving Intelligence as an active liability. Their predictions are given by

LiboulenPred = 40.7+5Cha+Wis-Int

Linestra and Colleen

Linestra also prioritizes the mind over the body, linearly combining Int, Wis and Cha with a random element as below:

LinestraPred = 22.5 + 1.2Int + 2.4Wis + 3.6Cha + 0.11d12 - 0.1

Colleen copies Linestra’s predictions, creating a nearly collinear column. Her guesses are Linestra’s, plus 1.7%; when Linestra predicts over 90% success chance, she adds a further 0.2%.

Fizz, Ziqual, and Ister

Fizz, Ziqual and Ister are physicalists, prioritizing physical stats over mental. Their predictions are given by

FizzPred = 41+2Dex+min(Str,Con)ZiqualPred = StrCon - ((StrCon)>12)*random.choice([0,1])IsterPred = 50+Str

Holly

Holly takes a holistic approach. Her prediction makes use of all non-Luck stats, but also contains a random element strong enough to obfuscate the signal: this makes her the best single predictor (after adjusting for her systematic underestimation of strong heroes and underestimation of weak ones), but also makes her redundant with the other fae, whose predictions are collectively sufficient to perfectly reconstruct success chance without her.

HollyPred = 20+Str+Con+Dex+Int+Wis+Cha+1d12

Sick days

Whenever Linestra took a sick day, Colleen would predict 50% and hope no-one noticed. After the 5540^th batch, the fae got a live-in healer, and none of them have missed a day since then.

Choice of Chosen

The fae’s ranking of heroes is decided by taking a weighted average of their predictions of success chance, in which Holly’s predictions are considered twice as important as everyone else’s. This ranking is used to decide which heroes to send in the frequent circumstances when they have more available heroes than they do imperiled worlds. The result is a small but non-negligible selection bias, which particularly affects Amy: though Luck is important, a disproportionate number of Lucky-but-otherwise-underwhelming heroes are Chosen just because Amy’s exuberance outweighs everyone else’s sober assessments, producing the net result that Luck ends up looking almost useless in the set of heroes sent.

Answerkey

The best potential Chosen to choose is Candidate #11, with a success chance of ~90.2%; the next best is Candidate #19 (whom the fae would have chosen in your absence) with a success chance of ~84.2%, closely followed by Candidate #2 with a success chance of ~83.7%.

Holly’s predictions are the most useful when considered in isolation. The redundant fae are Colleen (copying Linestra’s work) and Holly (all information conveyed by her is conveyed with higher fidelity by someone else).

(Congratulations to everyone who figured any of this out, and particular congratulations to aphyer for figuring all of it out plus more of the generation structure than I thought I'd exposed.)

Reflections

This was planned as an ML challenge; the intended path to failure - aside from the perennial problem of picking poor hyperparameters - was judging characters’ usefulness using GBTs’ Feature Importance (will usually say Holly is the most valuable just because early-planted decision trees take the easiest route away from the starting value) and/or unpenalized linear regression’s coefficients (will invariably imply Colleen is more useful than everyone else combined due to collinearity effects’ pathological behavior); the intended path to success was identifying Holly by ablating advisors and noting changes in performance metrics, and identifying Colleen by Just Looking At The Data.

These paths were mostly left untravelled; shortcuts untaken, traps unsprung^[3]; even players who eventually resorted to ML seemed to lean strongly towards pure Analysis. I didn’t try to make this scenario a puzzle, but players played it as one regardless, and got further with human cleverness than they could have with the machines I made it for. I hope it was a fun one: feedback on this point, and on any other point, is greatly appreciated.

Scheduling

I plan to run my next scenario from the 20^th to the 30^th of next month . . . unless someone strongly prefers a different timing, and/or has one of their own they want to run in June.

1. ^{^}

   The generation process for these variables is mostly irrelevant to the task; feel free to look through the generation code if you want the full story.

2. ^{^}

   Everyone knows Dexterity is the most important trait a hero could possibly have.

3. ^{^}

   The one trick anyone managed to fall for was misidentifying Amy as redundant due to selection biases, which was an emergent effect I didn’t explicitly plan.

Discuss