本文探讨了在面对认知局限时，我们如何做出明智的决策。文章深入分析了“无知”的概念，以及它与不确定性和“无头绪”的关系。通过对不同观点的提问和解答，作者引导读者思考在不确定性面前，我们应该如何评估行动的价值、制定决策，以及避免过度依赖精确的概率和期望值，从而为我们提供在复杂世界中做出更明智选择的框架。

🤔 什么是“无知”、不确定性和“无头绪”？文章首先定义了“无知”、“不确定性”和“无头绪”这些关键术语，为后续讨论奠定基础。

💡 “无知”与“无头绪”有什么联系？文章探讨了“无知”如何导致“无头绪”，以及除了“无知”之外，是否存在其他导致“无头绪”的原因。

❓ 如何在不确定性下做决策？文章讨论了在面对不确定性时，如何使用概率区间和价值区间，而不是单一的数字，来评估行动的价值，并提供了在“无知”的情况下做出决策的指导。

⚖️ 精确的概率和期望值是否合适？文章质疑了依赖精确概率和期望值的做法，认为这种做法可能存在武断性，并探讨了替代方案。

🔄 避免主导策略和金钱泵陷阱。文章讨论了在不确定性下，如何避免采取可能导致损失的策略，以及如何避免陷入“金钱泵”陷阱。

Published on July 7, 2025 9:54 AM GMT

I’ve argued in my unawareness sequence that when we properly account for our severe epistemic limitations, we are clueless about our impact from an impartial altruistic perspective.

However, this argument and my responses to counterarguments involve a lot of moving parts. And the term “clueless” gets used in various importantly different ways. It can be easy to misunderstand which claims I am (not) making, in the context of previous EA and academic writings on cluelessness.

So, as a “guide” to these arguments, I’ve written this list of questions and resources that answer them. Caveats:

Most of the resources are my own work — not because I necessarily think I’ve given the best answers, but because the precise claims and framings that other works use might be subtly yet importantly different from mine. I also include references to writings that I have not (co-)authored, for more context. But these authors don’t necessarily endorse my claims.When I link to a reply to someone else’s comment, I don’t mean to claim that the person being replied to endorses the exact statement of the objection I’ve given in this post.

What are unawareness, indeterminacy, and cluelessness?: The basics

What’s the connection between unawareness and cluelessness? Are there arguments for cluelessness besides the argument from unawareness?
Comment by meMogensen (2019)Further reading:
Roussos (2021)“Motivating example” in “Should you go with your best guess?: Against precise Bayesianism and related views”
What’s the difference between (A) accounting for unawareness, or having imprecise credences, and (B) just being really uncertain, or needing to think more before acting? You say we should use intervals of {probabilities} / {values of outcomes} / {expected values} instead of single numbers. What do these intervals mean?
“Unawareness vs. uncertainty” in “The challenge of unawareness for impartial altruist action guidance: Introduction”“The structure of indeterminacy” in “Should you go with your best guess?: Against precise Bayesianism and related views”Further reading:
“Degrees of imprecision from unawareness” in “Why intuitive comparisons of large-scale impact are unjustified”Tarsney et al. (2024, Sec. 3)
If you don’t use EV (or heuristics meant to approximate EV), how do you make decisions?
“Unawareness-inclusive expected value (UEV)” in “Why impartial altruists should suspend judgment under unawareness”“Suspending judgment on total effects, and choosing based on other reasons” in “Should you go with your best guess?: Against precise Bayesianism and related views”Further reading:
Clifton (2025a)Bradley (2012, Sec. 5)
What’s the connection between …
… indeterminacy and imprecision / imprecise probabilities?
“Indeterminate Bayesianism” in “Should you go with your best guess?: Against precise Bayesianism and related views”
… indeterminacy/imprecision and incompleteness?
“Appendix: Indeterminacy for ideal agents” in “Should you go with your best guess?: Against precise Bayesianism and related views”
… indeterminacy/imprecision and incomparability?
“Degrees of imprecision from unawareness” in “Why intuitive comparisons of large-scale impact are unjustified”
What’s the positive motivation for having indeterminate/imprecise credences, or assigning indeterminate/imprecise values to outcomes?
“Motivating example” in “Should you go with your best guess?: Against precise Bayesianism and related views”“Degrees of imprecision from unawareness” in “Why intuitive comparisons of large-scale impact are unjustifiedFurther reading:
Bradley (2012, Sec. 4.3)Bradley (2017, Sec. 11.3-11.4)
You say we should have imprecise credences (etc.) because picking a precise credence is “arbitrary”. Are you saying we need to justify everything from precisely formalizable principles? That seems doomed.
“Why not just do what works?” in “The challenge of unawareness for impartial altruist action guidance: Introduction”“Non-pragmatic principles” in “Winning isn’t enough”Further reading:
“Reasons for belief” in Clifton (2025a)Comment by me

Why aren’t precise credences and EV the appropriate response to these problems?

Sure, we don’t have an exact probability distribution over possible outcomes with exact values assigned to them. But aren’t we still ultimately aiming for the highest-EV action? And can’t we do that using best-guess proxies for the EV?^[1]

“Unawareness vs. uncertainty” in “The challenge of unawareness for impartial altruist action guidance: Introduction”

“Okay, But Shouldn’t We Try to Approximate the Bayesian Ideal?” in Violet Hour (2023)

Further reading:

Comment by Clifton

Why not aggregate our interval of {probabilities} / {values of outcomes} / {expected values} using a meta-distribution? (E.g., just take the midpoint.) Don’t we leave out information otherwise?
“The “better than chance” argument, and other objections to imprecision” in “Why intuitive comparisons of large-scale impact are unjustified”“Maximality is too permissive” in “Should you go with your best guess?: Against precise Bayesianism and related views”Further reading:
“Aggregating our representor with higher-order credences uses more information” in “Should you go with your best guess?: Against precise Bayesianism and related views”Clifton (2025b)Mogensen and Thorstad (2020, Sec 4.4)Bradley (2017, Sec. 13.2)
Can’t we always say which action is net-better as long as our intuitions are at least somewhat better than chance? Or, as long as there’s some similarity between promoting the impartial good and decision problems we’re much more familiar with?
“The “better than chance” argument, and other objections to imprecision” in “Why intuitive comparisons of large-scale impact are unjustified”“Meta-extrapolation” in “Why existing approaches to cause prioritization are not robust to unawareness”Further reading:
“Aggregating our representor with higher-order credences uses more information” in “Should you go with your best guess?: Against precise Bayesianism and related views”
Aren’t your credences just your acceptable betting odds, which are precise?
“Background on degrees of belief and what makes them rational” and “Suspending judgment on total effects, and choosing based on other reasons” in “Should you go with your best guess?: Against precise Bayesianism and related views”Further reading:
Eriksson and Hájek (2007)Carlsmith (2021)
You say that picking a precise credence/EV is arbitrary. Isn’t the cutoff between the numbers you include vs. exclude in imprecise credences/intervals of EVs also arbitrary?
“Indeterminate Bayesianism” in “Should you go with your best guess?: Against precise Bayesianism and related views”Comment by meFurther reading:
Lyon (2017)Bradley (2012, Sec. 4.3.6)
If you have imprecise credences or incomplete preferences, can’t you get money-pumped or otherwise take a dominated strategy? (And if you apply some patch to avoid dominated strategies, aren’t you just acting like a precise EV maximizer?)
Petersen (2023)“A money-pump for Completeness” in Thornley (2023)“Avoiding dominated strategies” in “Winning isn’t enough”Further reading:
Bradley and Steele (2014)Bradley (2022)

Hedden (2015)^[2]

Sure, you don’t need to have precise probabilities and evaluate actions based on EV to avoid money pumps. Still, don’t coherence/representation theorems collectively suggest that precise EV maximization is normatively correct? (As Yudkowsky puts it, “We have multiple spotlights all shining on the same core mathematical structure [of expected utility]”.)^[3]

“Unawareness vs. uncertainty” in “The challenge of unawareness for impartial altruist action guidance: Introduction”

“Avoiding dominated strategies” in “Winning isn’t enough”

Further reading:

Rethink Priorities (2023, Sec. 3.2)

Hájek (2008)

Aren’t we not clueless (in practice) because…?

We’re surely not entirely clueless in mundane contexts. And it would be arbitrary to posit a sharp discontinuity between those contexts and promoting the impartial good. The complexity of a decision problem is continuous and on a spectrum. Thus, aren’t we not entirely clueless about promoting the impartial good?
“When is unawareness not a big deal?” and “Why we’re especially unaware of large-scale consequences” in “Why intuitive comparisons of large-scale impact are unjustified”Further reading:
Comment by Daniel
Sure, there’s some imprecision in our estimates, but aren’t at least some interventions good by a wide enough margin that the imprecision doesn’t matter?
“Reasons to suspend judgment on comparisons of strategies’ UEV” in “Why impartial altruists should suspend judgment under unawareness”Further reading:
“Case study revisited” in “Why existing approaches to cause prioritization are not robust to unawareness”
Why not just use the strategies (or credence-forming methods) that work best, either empirically or in toy experiments resembling our situation?
“Heuristics” in “Winning isn’t enough”“Meta-extrapolation” in “Why existing approaches to cause prioritization are not robust to unawareness”Further reading:
Williamson (2022, Sec. 1.4.2)
Come on, do you really think [obviously good/bad thing] is no better/worse than staying at home watching cat videos? Isn’t this just radical skepticism?
“When is unawareness not a big deal?” and “Why we’re especially unaware of large-scale consequences” in “Why intuitive comparisons of large-scale impact are unjustified”Further reading:
“Maximality is too permissive” in “Should you go with your best guess?: Against precise Bayesianism and related views”
Why not wager on the possibility that we’re not clueless?
“The “better than chance” argument, and other objections to imprecision” and “Appendix A: The meta-epistemic wager?” in “Why intuitive comparisons of large-scale impact are unjustified”Further reading:
“Meta-extrapolation” in “Why existing approaches to cause prioritization are not robust to unawareness”
Superforecasters do better than chance at predicting complex outcomes, so aren’t we not clueless?
“Precise forecasts do better than chance” in “Should you go with your best guess?: Against precise Bayesianism and related views”“Unawareness and superforecasting” in “Why intuitive comparisons of large-scale impact are unjustified”Further reading:
“Mechanisms, Metaculus, and World-Models” in Violet Hour (2023)
Shouldn’t we treat the unknown unknowns as canceling out in expectation, since we can’t say anything about them either way? Or at least, can’t we extrapolate from what we do know? Even if we’re biased, it would be surprising for our biases to be highly anti-inductive in expectation.
“Symmetry” and “Extrapolation” in “Why existing approaches to cause prioritization are not robust to unawareness”Further reading:
“Problem 1: Modeling the catch-all, and biased sampling” in “Why impartial altruists should suspend judgment under unawareness”

Who, and which interventions, are these problems relevant to?

Isn’t cluelessness only a problem if you’re trying to directly shape the far future? But I’m not doing that, I’m trying to (e.g.) stop x-risks in the next few years.
“Case study: Severe unawareness in AI safety” in “The challenge of unawareness for impartial altruist action guidance: Introduction”“Extremely limited understanding of mechanisms” in “Why intuitive comparisons of large-scale impact are unjustified”Further reading:
“Focus on Lock-in” and “Case study revisited” in “Why existing approaches to cause prioritization are not robust to unawareness”
Isn’t cluelessness only a problem for sequence thinking?
“Appendix E: On cluster thinking” in “Why existing approaches to cause prioritization are not robust to unawareness”
Isn’t it robustly positive to …
… try to prevent bad lock-in events (like AI x-risk)?
“Focus on Lock-in” and “Case study revisited” in “Why existing approaches to cause prioritization are not robust to unawareness”
… do more research, spread better values or decision-making practices, gain more influence on AI, or save money?
“Capacity-Building” in “Why existing approaches to cause prioritization are not robust to unawareness”
… follow strategies whose least conjunctive effects are positive?
“Simple Heuristics” in “Why existing approaches to cause prioritization are not robust to unawareness”
Your case for indeterminacy appeals a lot to “arbitrariness”. I’m fine with some arbitrariness in my beliefs and preferences. Isn’t that enough for me to not be clueless?
“Permissive epistemology doesn’t imply precise credences / completeness / non-cluelessness”
What about extremely small decisions, like helping an old lady cross the street? If we help the old lady, isn’t it reasonable to treat the expected value of the off-target effects as so negligible that the benefit to the old lady dominates?
Yim (2019)Comment by Aird

What implications do these problems (not) have for our decisions?

What’s decision-relevant about saying it’s indeterminate whether A is net-better or worse than B, if you have to choose something anyway?
“Practical hallmarks of indeterminacy” in “Should you go with your best guess?: Against precise Bayesianism and related views”
What’s decision-relevant about your arguments about unawareness, if they don’t say it’s bad to keep doing what we’re doing?
“Appendix A: The meta-epistemic wager?” in “Why intuitive comparisons of large-scale impact are unjustified”“Conclusion and taking stock of implications” in “Why existing approaches to cause prioritization are not robust to unawareness”
Are you saying we should default to inaction?
Comment by me

1. ^{^}

    Note: I’m not sure the references included here fully respond to this question. But it’s not yet clear to me what people mean by this question, so I encourage anyone who finds the included references inadequate to say in the comments what they have in mind.

2. ^{^}

    This work argues against the view that diachronic (i.e., sequential) money pump / dominated strategy arguments, such as the arguments against incompleteness, are normatively relevant in the first place.

3. ^{^}

    Note: Again, I’m not entirely sure what the argument for this objection is supposed to be, so it’s hard to say whether these references adequately address it.

Discuss