🎯提出为 AI 设计暂停功能的问题，即如何通过设计效用函数使 AI 在需要时暂停。文中提到一种设想，若处于暂停状态且未采取行动，效用为预计未暂停时的效用；若未处于暂停状态，效用为正常效用函数。

🤔探讨这种设想可能无法实现的原因，作者好奇为何这样的设计未能达到预期效果，引发对该问题的深入思考。

💬强调对这一设想的讨论，希望了解为何这种看似合理的设计在实际中可能行不通，以寻求更好的解决方案。

Published on September 26, 2024 10:05 PM GMT

One of the questions I've heard asked is "how do you design a utility function that would make the AI do nothing?"  That is, how could we put a pause button on an AI so that we could pause it if we wanted to?  I had an idea about how one could do this, and am sure it has already been thought of, so I'm curious to know why this doesn't end up working.

Why can't we just write something like:

If (paused) and not (taking actions): utility=PredictedUtilityIfNotPaused

If not (paused): utility=NormalUtilityFunction