文章探讨了在人工智能快速发展背景下，如何确保人类有效监督AI系统。核心概念是“增强监督”，即利用AI来放大人类的监督能力。文章提出了两种关键机制：一是“评分员辅助”，即让人类评分员借助AI助手来评估AI输出；二是“混合”，结合人类和AI评分员的判断，根据任务实例预测其相对评分能力。文章强调了人机互补的重要性，并指出了设计有效的人机协作协议所面临的挑战，这些挑战涉及人机交互、认知科学等多个领域。通过跨学科合作，可以更好地解决这些社会技术难题，确保AI安全。

🤖 增强监督：为了应对日益强大的AI系统，需要利用AI来放大人类的监督能力，确保AI与人类价值观对齐。

🤝 人机互补：人类和AI在能力上各有优劣，通过结合两者优势，可以产生比单独使用人类或AI评分员更强的监督信号。

✍️ 评分员辅助：让人类评分员使用AI助手来评估AI输出，AI助手可以指出AI输出的缺陷或自动化部分评分任务，从而提高效率和准确性。

🎛️ 混合机制：结合人类和AI评分员的判断，根据任务实例预测其相对评分能力，从而更有效地利用不同评分员的优势。

🤔 跨学科合作：设计有效的人机协作协议面临诸多挑战，需要人机交互、认知科学、心理学等多个领域的知识和合作。

Published on December 24, 2024 9:57 AM GMT

By Sophie Bridgers, Rishub Jain, Rory Greig, and Rohin Shah
Based on work by the Rater Assist Team: Vladimir Mikulik, Sophie Bridgers, Tian Huey Teh, Rishub Jain, Rory Greig, Lili Janzer (randomized order, equal contributions)

 

Human oversight is critical for ensuring that Artificial Intelligence (AI) models remain safe and aligned to human values. But AI systems are rapidly advancing in capabilities and are being used to complete ever more complex tasks, making it increasingly challenging for humans to verify AI outputs and provide high-quality feedback. How can we ensure that humans can continue to meaningfully evaluate AI performance? An avenue of research to tackle this problem is “Amplified Oversight” (also called “Scalable Oversight”), which aims to develop techniques to use AI to amplify humans’ abilities to oversee increasingly powerful AI systems, even if they eventually surpass human capabilities in particular domains.

With this level of advanced AI, we could use AI itself to evaluate other AIs (i.e., AI raters), but this comes with drawbacks (see Section IV: The Elephant in the Room). Importantly, humans and AIs have complementary strengths and weaknesses. We should thus, in principle, be able to leverage these complementary abilities to generate an oversight signal for model training, evaluation, and monitoring that is stronger than what we could get from human raters or AI raters alone. Two promising mechanisms for harnessing human-AI complementarity to improve oversight are:

Rater Assistance, in which we give human raters access to an AI rating assistant that can critique or point out flaws in an AI output or automate parts of the rating task, andHybridization, in which we combine judgments from human raters and AI raters working in isolation based on predictions about their relative rating ability per task instance (e.g., based on confidence).

The design of Rater Assistance and/or Hybridization protocols that enable human-AI complementarity is challenging. It requires grappling with complex questions such as how to pinpoint the unique skills and knowledge that humans or AIs possess, how to identify when AI or human judgment is more reliable, and how to effectively use AI to improve human reasoning and decision-making without leading to under- or over-reliance on the AI. These are fundamentally questions of Human-Computer Interaction (HCI), Cognitive Science, Psychology, Philosophy, and Education. Luckily, these fields have explored these same or related questions, and AI safety can learn from and collaborate with them to address these sociotechnical challenges. On our team, we have worked to expand our interdisciplinary expertise to make progress on Rater Assistance and Hybridization for Amplified Oversight.

 

Read the rest of the full blog here!

Discuss