本文探讨了如何识别和应对人工智能（AI）的对齐风险，特别关注了内部机制分析与行为证据在揭示AI潜在问题中的作用。文章认为，仅依靠内部机制分析得出的证据，在说服人们采取有力行动方面可能效果有限。相反，能够被人类理解的行为证据，对于引发有效的应对措施至关重要。文章还讨论了内部机制分析在辅助寻找行为证据、提高对问题行为的理解以及在其他方面的潜在价值，并强调了迭代检测技术的重要性。

🧐 **行为证据的重要性：** 文章强调，要促使人们对AI对齐问题采取有力行动，关键在于依赖人类可以理解的行为证据。仅凭内部机制分析得出的证据，难以有效说服人们采取行动。

💡 **内部机制分析的辅助作用：** 内部机制分析可以帮助发现难以直接观察到的行为证据，例如，揭示AI执行的有害行为。此外，它还能帮助人类理解AI行为为何有害。

🔄 **迭代检测技术：** 文章建议，检测对齐问题的技术应侧重于寻找行为证据，或力求足够强大且成本低廉，以便进行迭代改进。这有助于不断提升检测的准确性和有效性。

🤝 **其他应用场景：** 除了促使人们采取行动，检测对齐问题还能促进对齐措施的支持、推动研究改进，以及帮助相关人员调整策略，例如AI公司员工可以根据检测结果调整控制协议。

Published on April 15, 2025 6:14 PM GMT

One key hope for mitigating risk from misalignment is inspecting the AI's behavior, noticing that it did something egregiously bad, converting this into legible evidence the AI is seriously misaligned, and then this triggering some strong and useful response (like spending relatively more resources on safety or undeploying this misaligned AI).

You might hope that (fancy) internals-based techniques (e.g., ELK methods or interpretability) allow us to legibly incriminate misaligned AIs even in cases where the AI hasn't (yet) done any problematic actions despite behavioral red-teaming (where we try to find inputs on which the AI might do something bad), or when the problematic actions the AI does are so subtle and/or complex that humans can't understand how the action is problematic^[1]. That is, you might hope that internals-based methods allow us to legibly incriminate misaligned AIs even when we can't produce behavioral evidence that they are misaligned.

Unfortunately, I think it's unlikely that evidence which is just from fancy internals-based methods (and can't be supported by human inspection of AI behavior) will be legible and convincing in practice. Suppose that we use some fancy method to train an ELK head to honestly predict what is going to happen or to honestly answer our questions. Then, suppose this head says/predicts "this action would yield very bad consequences in a way which is unlikely to be an accident" but humans can't understand how this action would result in these bad consequences (even after it is explained to them). Even if the internals-based method is extremely well supported theoretically and empirically (which seems quite unlikely), I don't think this would suffice for this to trigger a strong response by convincing relevant people^[2]. This reasoning also applies to "our interpretability method decoded some very problematic deceptive reasoning" or "the deception probe fires in a very suspicious way (e.g. it fires more when we give it a fake opportunity to escape or ask it about its alignment research outputs)".

More generally, I think that to convince people enough to take strong action, evidence will need to bottom out in some sort of behavioral evidence that (many) humans can understand.^[3] (That's not to say solid behavioral evidence will necessarily suffice.)

This isn't to say that internals-based techniques for detecting scheming aren't useful. These techniques could be quite useful via two mechanisms:

The technique could allow us to find behavioral evidence that would have otherwise been hard to find. For instance, maybe we can use the method to discover an input on which the AI does something egregiously bad (that we can understand). (However, it will be important to legibly not apply too much optimization pressure as you might be able to get any model to look misaligned with enough optimization pressure.) Internals-based techniques could also help with having humans understand why an action is egregiously bad (e.g., we might be able to use internals-based techniques to effectively do better elicitation for debaters).Detection of scheming can be useful via mechanisms other than convincing people to take strong and expensive actions:
Marginally more buy-in and support for handling misalignment could be helpful.If we can detect misalignment, we can study techniques for preventing or removing misalignment by iterating (or even training) against our detectors. The more robust the detector is, the more useful this would be.The subset of people who buy this evidence could usefully alter their actions and strategy. For instance, AI company employees implementing countermeasures and utilizing AIs might adjust their approach now that they know the AIs are likely seriously misaligned (though presumably not in a way that greatly increases safety costs). In the case where you know your AI is seriously misaligned, you want to adjust your control protocol, even if you're unable to take much of a hit on (apparent) usefulness^[4].We might be able to trigger well-enforced preexisting commitments. We could even imagine some sort of legally binding contract which triggers in this case (effectively a bet between people who are more worried about misalignment and those who are less worried).

Given these difficulties in producing legible evidence, I think researchers working on internals-based techniques for detecting misalignment should spend some time thinking about the ultimate use case of their technique and then alter their approach as needed. These difficulties also generally make me somewhat less excited about internals-based methods for detecting misalignment (at least if these methods don't involve finding inputs on which the AI behaves poorly). I think techniques for detecting misalignment should focus on finding behavioral evidence or should aim to be robust (and cheap) enough to be iterated against.

Audio narrations of this blog are available on our podcast. Search "Redwood Research" or subscribe via Apple Podcasts, Spotify, RSS, or other platforms.

---

1. We can push the frontier of what humans can understand using recursive oversight techniques like debate which might allow for eliciting explanations that allow for humans to understand how the action is problematic (e.g., pointing out exactly what the problematic behavior is and explaining how to test this in isolation). However, if the problematic nature of an action is sufficiently subtle and/or complex (perhaps due to the AI having superhuman understanding), then even well elicited debate might be unable to provide a good enough explanation. We might have hoped that internals-based methods could push frontier beyond debate (perhaps to indefinite worst case scalability) while still producing legible and convincing evidence, but I'll argue that non-behavioral evidence is unlikely to be convincing. This suggests an upper bound on strategies which depend on legible evidence: once well elicited debate (and other methods for improving human understanding) aren't viable, misaligned AIs can get away with doing bad actions without these actions resulting in legible evidence of misalignment. ↩︎

2. That is, relevant humans, this approach might work for eliciting a strong response from AIs once AIs make the important decisions. ↩︎

3. That is, it must at least partially bottom out behaviorally. Internals-based evidence which is substantially behaviorally supported (in a relatively direct way) could suffice. E.g., if we can read off the AI's internal reasoning using some fancy method, it looks quite malign, and the malign aspects of the reasoning are also quite predictive of the AI's behavior, this could suffice. For instance, Alignment Faking in Large Language Models utilized a mixture of inspecting the Chain-of-Thought and behavioral evidence in a relatively effective way. ↩︎

4. Presumably, the actual level of usefulness might be substantially lower as the AI is a misaligned AI trying to disempower humans. ↩︎

Discuss