本文探讨了人工智能（AI）在复杂任务中的局限性，并提出“智慧”是弥补这些不足的关键。文章指出，现有的AI系统在应对不确定性、解释性、协作和安全性方面存在问题，而这些问题都源于缺乏智慧。作者认为，通过借鉴认知和社会科学，将元认知能力融入AI系统，可以提升其鲁棒性、可解释性、协作性和安全性。文章还讨论了构建智慧AI的潜在方法，并强调了智慧AI在复杂现实世界中的重要性。

🧠文章的核心观点认为，当前AI系统缺乏“智慧”，这导致其在处理不确定性、解释性、协作和安全性方面遇到挑战。作者认为，智慧是AI系统有效应对模糊、不确定、新颖和复杂问题的关键。

💡文章强调了元认知在人类智慧中的重要性，并提出将元认知能力融入AI系统是提升其智慧的关键。元认知包括识别知识的局限性、考虑不同观点和适应环境变化等能力。

🔄文章提出了“良性循环”的概念，即通过同时提升AI的鲁棒性、可解释性、协作性和安全性，这些品质可以相互促进。例如，鲁棒性可以促进协作，可解释性可以促进鲁棒性，协作可以促进可解释性等。

✅文章建议，构建智慧AI的一个方法是，通过基准测试元认知能力和训练AI系统运用智慧推理。这种方法旨在使AI系统不仅智能，而且在复杂现实世界中能够明智地行动。

Published on April 10, 2025 11:54 AM GMT

Isn't this post a duplicate?

I originally posted a link to the paper on Less Wrong.

When I decided to write a summary, the default option would have been to write a separate post. However, I wanted the comments to stay attached. So I decided to edit the summary into the linkpost and post a new linkpost that links to the summary.

If I'd gone the normal route, there would still have been both a summary and linkpost on Less Wrong. The only difference is that this makes the commenting situation slightly neater.

Link to original paper

Authors of the Paper:

Samuel G. B. Johnson, Amir-Hossein Karimi, Yoshua Bengio, Nick Chater, Tobias Gerstenberg, Kate Larson, Sydney Levine, Melanie Mitchell, Iyad Rahwan, Bernhard Schölkopf, Igor Grossmann

Abstract of Summarised Paper:

Recent advances in artificial intelligence (AI) have produced systems capable of increasingly sophisticated performance on cognitive tasks. However, AI systems still struggle in critical ways: unpredictable and novel environments (robustness), lack of transparency in their reasoning (explainability), challenges in communication and commitment (cooperation), and risks due to potential harmful actions (safety). We argue that these shortcomings stem from one overarching failure: AI systems lack wisdom.

 Drawing from cognitive and social sciences, we define wisdom as the ability to navigate intractable problems - those that are ambiguous, radically uncertain, novel, chaotic, or computationally explosive - through effective task-level and metacognitive strategies. While AI research has focused on task-level strategies, metacognition - the ability to reflect on and regulate one's thought processes - is underdeveloped in AI systems. In humans, metacognitive strategies such as recognizing the limits of one's knowledge, considering diverse perspectives, and adapting to context are essential for wise decision-making. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.

By focusing on developing wise AI, we suggest an alternative to aligning AI with specific human values - a task fraught with conceptual and practical difficulties. Instead, wise AI systems can thoughtfully navigate complex situations, account for diverse human values, and avoid harmful actions. We discuss potential approaches to building wise AI, including benchmarking metacognitive abilities and training AI systems to employ wise reasoning. Prioritizing metacognition in AI research will lead to systems that act not only intelligently but also wisely in complex, real-world situations.

Why I wrote this summary: 

Firstly, I thought the framing of metacognition as a key component of wisdom missing from current AI systems was insightful and the resulting analysis fruitful.

Secondly, this paper contains some ideas similar to those I discussed in Some Preliminary Notes on the Promise of a Wisdom Explosion. In particular, the authors talk about a "virtuous cycle" in relation to wisdom in the final paragraphs:

By simultaneously promoting robust, explainable, cooperative, and safe AI, these qualities are likely to amplify one another:

Robustness will facilitate cooperation (by improving confidence from counterparties in its long-term commitments) and safety (by avoiding novel failure modes^[1]).Explainability will facilitate robustness (by making it easier to human users to intervene in transparent processes) and cooperation (by communicating its reasoning in a way that is checkable by counterparties).Cooperation will facilitate explainability (by using accurate theory-of-mind about its users) and safety (by collaboratively implementing values shared within dyads, organizations, and societies).

Wise reasoning, therefore, can lead to a virtuous cycle in AI agents, just as it does in humans. We may not know precisely what form wisdom in AI will take but it must surely be preferable to folly. 

Thinking of Commenting?

I recommend following the link to the summary and commenting there so that the comments are all in the same place.

Discuss