文章探讨了情报分析中可能出现的认知偏差，尤其是在面对缺乏证据的情况时。文章以二战时期对日裔美国人的拘留为例，分析了在没有证据表明存在破坏活动时，为何官员反而可能认为存在精心策划的阴谋。通过贝叶斯定理的视角，文章指出，缺乏证据并不一定意味着风险降低，反而可能导致对更危险、组织更严密的威胁的担忧增加。文章最后强调了在评估风险时，要考虑多种可能性，并警惕单一证据带来的误导。

🕵️ 文章首先引用了一个历史案例，即二战时期对日裔美国人的拘留事件，以此引出核心讨论：在缺乏证据的情况下，如何解读和评估潜在的威胁。

💡 随后，文章构建了三个假设情景：没有反战活动、存在非协调的反战活动、存在协调的第五纵队阴谋。通过对这三种情况的概率进行量化分析，揭示了缺乏证据可能带来的认知偏差。

📈 文章运用贝叶斯定理，计算了在没有观察到间谍活动或破坏行为的情况下，各种假设的后验概率。结果表明，虽然整体反对的可能性降低，但对更危险的、有组织的阴谋的担忧却可能增加。

⚠️ 作者强调，单一的指标，例如“坏事程度”，并不能完全反映风险。在评估风险时，需要考虑多种可能的结果，并警惕缺乏证据可能带来的误导。

Published on July 13, 2025 1:42 AM GMT

The following is a nitpick on an 18 year old blog post.

This fable is retold a lot. The progenitor of it as a rationalist mashal is probably Yudkowsky's classic sequence article. To adversarially summarize:

It's the beginning of the second world war. The evil governor of California wishes to imprison all Japanese-Americans - suspecting they'll sabotage the war effort or commit espionage.It is brought to his attention that there is zero evidence of any subversion of any kind by Japanese-Americans.He argues, rather than exonerating the Japanese-Americans, the lack of evidence convinceshim that there is a well organized fifth-column conspiracy that has been strategically avoiding subversion to lull the population and government into a false sense of security, before striking at the right moment.However, if evidence of sabotage would update him towards believing in the presence of opposition among Japanese Americans, then a lack of evidence necessarily must update him away from that belief.There's no way that a lack of evidence for a conspiracy could ever cause you to be more worried about said conspiracy.So the governor is a stupid evil cringe idiot with incoherent beliefs.

I agree with the broad takeaway. The provided heuristic seems useful for detecting when you might be not updating coherently. The governor in question probably wasn't well-founded in his beliefs, and probably was a bad person. From a quick skim of the wikipedia article, Japanese-American internment seems like it was absolutely horrific, unjustified, unreasonably inhumane, and shouldn't have been done.

That aside, this specific example is critically flawed.

 

 

Let's model the governor's reasoning explicitly. We have three disjoint hypotheses:^[1]

$H_1$: There is no significant anti-war subterfuge by Japanese-Americans.$H_2$: There is significant anti-war subterfuge, but it's not well coordinated.$H_3$: There is significant anti-war subterfuge, well coordinated and organized into a fifth-column conspiracy.

Assuming we wait a week, we would expect to see some evidence of espionage or sabotage under each world with probabilities:

$P\left(E|H_1\right)=0.02$ (measurement isn't perfect, reality has some random noise. 1 in 50 chance of seeing something, even though there's nothing)$P(E\mid H_2)=0.85$ (no one's coordinated, so it's pretty likely that one of the many saboteurs will do some sabotage in this particular week)$P(E\mid H_3)=0.05$ (maybe the coordinated conspiracy will choose to act in this given week - but it's far more likely they'll keep their members in line, holding back anyone who might act on their own, and instead wait for an opportune moment later in the war when they can do more damage with a single large attack)^[2]

You believe you live in the no-opposition world with $P\left(H_1\right)=0.6$, the uncoordinated opposition world with $P\left(H_2\right)=0.3$, and the coordinated conspiracy world with a tiny $P\left(H_3\right)=0.1$. You wait a week, and see no evidence of espionage or sabotage.

 

$\begin{array}{cc}P\left(E\right)& =P\left(E|H_1\right)P\left(H_1\right)+P\left(E|H_2\right)P\left(H_2\right)+P\left(E|H_3\right)P\left(H_3\right)\\ & =0.02\cdot 0.6+0.85\cdot 0.3+0.05\cdot 0.1=0.27\\ & \\ P\left(\neg E\right)& =1-P\left(E\right)=0.73\end{array}$

 

$\begin{array}{cc}P\left(H_1|\neg E\right)& =\frac{P\left(\neg E|H_1\right)P\left(H_1\right)}{P\left(\neg E\right)}=\frac{0.98\cdot 0.6}{0.73}=0.81\\ & \\ P\left(H_2|\neg E\right)& =\frac{P\left(\neg E|H_2\right)P\left(H_2\right)}{P\left(\neg E\right)}=\frac{0.15\cdot 0.3}{0.73}=0.06\\ & \\ P\left(H_3|\neg E\right)& =\frac{P\left(\neg E|H_3\right)P\left(H_3\right)}{P\left(\neg E\right)}=\frac{0.95\cdot 0.1}{0.73}=0.13\end{array}$

 

So the odds of there being any opposition -$P(H_2\vee H_3)$ - have gone down.

Obviously, it would be good to live in world $H_1$. We're a bit worried about $H_2$, but uncoordinated opposition really isn't a big deal, and doesn't pose much of a threat. The chance of a well-coordinated fifth-column conspiracy is what really keeps you up at night, and you suspect it would be about $100$ times as bad as uncoordinated opposition. Before seeing nothing for a week, we have an expected badness[^3] of:

$⟨\text{badness}⟩=0+1\times 0.3+100\times 0.1=10.3$

After seeing no evidence of any sabotage or espionage for a week, we have an expected badness of:

$⟨\text{badness}⟩=0+1\times 0.06+100\times 0.13=13.06$

The lack of any evidence of sabotage has decreased the overall probability of any opposition, but also shifted that remaining probability mass towards a more dangerous and organised operation. The amount that we should be worried has gone up.

In this case, our worry has gone up by about 25%. If the governor was even more convinced that there was some high underlying level of anti-war sentiment - that were it not for an organised conspiracy coordinating action, someone would at least try to bomb a bridge or something - then the prior $P\left(H_1\right)$ can be pushed down further, and the badness increase caused by not seeing any evidence of conspiracy can go to the moon.

 

 

 

The original article is still mostly correct. For any operator $\hat{a}$ - such as "badness" - with expectation

$⟨\hat{a}⟩={\int }_{H}a\left(H\right)\cdot P\left(H\right)dH$

if observing $E$ causes the expectation of that operator to increase, then observing $\neg E$ must cause the expectation to decrease. Conservation holds.

$⟨\hat{a}|E⟩>⟨\hat{a}⟩⟹⟨\hat{a}|\neg E⟩<⟨\hat{a}⟩$

It's just that $1\times H_2+1\times H_3$ isn't the only operator we can care about. I did warn you this was a nitpick.

1. ^{^}

   You can generalise this further to a continuous probability distribution over the size, shape, and level of coordination of any conspiracy, but three discrete possibilities is enough to illustrate the point. 

2. ^{^}

   A perfectly optimal conspiracy will, of course, act in a way that minimises the amount of info you can gain, so in this situation it would be better for them to show just enough small sabotage to convince you that there is no large conspiracy. At which point you'll be on the lookout for a suspiciously perfect amount of medium-sized sabotage, and around and around until Nash equilibrium. Of course both you and the real conspiracy will have constraints on your behaviour, so this isn't without merit.

Discuss