The Alignment Project是一个由多国政府、企业、风投及慈善机构支持的全球性基金，规模超过1500万英镑，旨在加速AI控制与对齐研究的进展。该项目聚焦于AI安全领域中被低估的研究方向，并鼓励全球研究人员提交项目申请。其研究目标包括AI控制（如何防止AI系统产生危害性行为）和AI对齐（如何设计AI系统使其不产生危害性行为）。项目涵盖信息论、计算复杂性理论、经济学、博弈论、概率论、学习理论、强化学习评估、认知科学、可解释性、基准设计和训练后方法等多个学科领域，以期通过跨学科合作推动AI安全的关键性突破。

💡 **AI控制与对齐是核心目标**：该项目旨在解决两个关键问题：一是如何防止AI系统采取可能威胁集体安全的行为（AI控制），二是的设计AI系统使其从根本上避免此类行为（AI对齐）。这两种方法互为补充，共同致力于提升AI的安全性。

🌍 **多方资金支持的全球性项目**：The Alignment Project获得了来自英国AI安全研究所、加拿大AI安全研究所、Schmidt Sciences、亚马逊AWS、Anthropic、Halcyon Futures、Safe AI Fund以及英国先进研究与创新局等国际政府、行业、风险投资和慈善基金的支持，显示了其在全球AI安全领域的广泛影响力和重要性。

🔬 **聚焦被低估的研究领域**：项目特别强调对AI安全领域中可能被低估的研究方向的关注，鼓励研究人员提出新颖的研究思路。文章列举了信息论、计算复杂性、经济学、博弈论、概率论、学习理论、强化学习、认知科学、可解释性、基准设计和训练后方法等具体研究领域，以期激发更广泛和深入的探讨。

🤝 **跨学科合作推动研究突破**：项目认识到AI安全研究的复杂性需要跨学科的视角和工具，因此鼓励来自不同领域的顶尖人才参与。通过将研究领域细分为不同学科的问题，项目旨在吸引更广泛的专业知识，共同推进AI安全技术的进步。

🚀 **提供资金支持与合作机会**：The Alignment Project不仅提出了研究方向，还为符合条件的研究者提供了资金支持。文章鼓励对AI安全有研究兴趣并能将其研究与项目挑战联系起来的个人提交项目提案，并提供了战略与运营岗位的申请链接，为有意参与者提供了实际的参与途径。

Published on August 1, 2025 9:52 AM GMT

The Alignment Project is a global fund of over £15 million, dedicated to accelerating progress in AI control and alignment research. It is backed by an international coalition of governments, industry, venture capital and philanthropic funders. 

This sequence sets out the research areas we are excited to fund – we hope this list of research ideas presents a novel contribution to the alignment field. We have deliberately focused on areas that we think the AI safety community currently underrates. 

Apply now to join researchers worldwide in advancing AI safety. 

For those with experience scaling and running ambitious projects, apply to our Strategy & Operations role here.

Our research goals

In-scope projects will aim to address either of the following challenges:

AI Control: How can we prevent AI systems from carrying out actions that pose risks to our collective security, even when they may attempt to carry out such actions?AI Alignment: How can we design AI systems which do not attempt to carry out such actions in the first place? 

Making substantial breakthroughs in these areas is an interdisciplinary effort, requiring a diversity of tools and perspectives. We want the best and the brightest across many fields to contribute to alignment research, so have organised these priority research areas as a set of discipline-specific questions. We suggest clicking ahead to your specific areas of interest, rather than reading linearly. Sections are roughly ordered from most theoretical to most empirical. 

Some of the subfields below have more detail than others about subproblems, recent work, and related work. This should not be read as a signal about which areas we believe are more important: much of the variance is due to areas our alignment and control teams, or our collaborators, have focused on to date. So, for example, lots of the alignment questions focus on scalable oversight / debate. We want to bring other areas up to similar levels of detail, and will attempt to do this in future versions of this agenda.

We’re excited about funding projects that tackle these questions, even if they aren’t focused on a problem outlined below. Feel free to look at others’ lists and overviews — e.g. Google DeepMind, Anthropic, or Redwood Research — for ideas. If you see connections between your research and these challenges, we encourage you to submit a proposal.

Research areas

Information Theory and Cryptography: Prove theoretical limits on what AI systems can hide, reveal or prove about their behaviour.Computational Complexity Theory: Find Formal guarantees and impossibility results behind scalable oversight protocols.Economic Theory and Game Theory: Find incentives and mechanisms to direct strategic AI agents to desirable equilibria.Probabilistic Methods: Bayesian and rare-event techniques for tail-risk estimation, scientist-AI, and formal reasoning under uncertainty.Learning Theory: Understand how training dynamics and inductive biases shape generalisation.Evaluation and Guarantees in Reinforcement Learning: Stress-test AI agents and prove when they can’t game, sandbag or exploit rewards.Cognitive Science: Map and mitigate the biases and limitations of human supervision.Interpretability: Access internal mechanisms to spot deception.Benchmark Design and Evaluation: Translate alignment's conceptual challenges into concrete, measurable tasks.Methods for Post-training and Elicitation: Refining, probing and constraining model behaviour.AI Control: Current alignment methods can't ensure AI systems act safely as they grow more capable, so the field of AI Control focuses on practical techniques—like restricting AI capabilities and using oversight models—to prevent catastrophic outcomes and test systems before they can cause harm.

Our backers

The Alignment Project is supported by an international coalition of government, industry, and philanthropic funders — including the UK AI Security Institute, the Canadian AI Safety Institute, Schmidt Sciences, Amazon Web Services, Anthropic, Halcyon Futures, the Safe AI Fund and the UK Advanced Research and Innovation Agency — and a world-leading expert advisory board.

 

View The Alignment Project website to learn more or apply for funding here.

Discuss