文章探讨了前沿人工智能安全问题，认为仅依靠“争先”策略是不够的。作者指出，虽然竞争能促使AI开发者提升安全性，但无法确保所有开发者都采用足够的安全措施。文章强调，除了“争先”外，还需要“筑底”，即建立最低安全标准，以应对AI系统日益强大的潜在风险。作者详细阐述了“争先”策略的局限性，市场力量无法确保安全实践的普遍采用，以及市场所需的安全性标准可能不足以阻止灾难性风险。最后，文章提出了完善“争先”策略的建议，包括补充性政策。

💡“争先”策略的定义：Anthropic倡导通过竞争压力提升AI安全性，鼓励开发者将安全作为竞争优势。这种策略促使开发者改进安全措施，但其有效性受到质疑。

⚠️“争先”策略的局限性：竞争压力可以带来安全改进，但不能保证所有开发者都采纳足够的安全措施。文章以Meta公司为例，说明“争先”策略在推动安全措施普及方面的滞后性。

🛡️“筑底”的重要性：随着AI系统能力的提升，仅依赖竞争压力是不够的。文章强调需要“筑底”，即建立最低安全标准，以确保所有开发者都遵守基本安全规范，从而应对潜在的灾难性风险。

📢市场力量的不足：市场力量无法保证安全实践的普遍采用，市场所需的安全标准也可能不足以应对灾难性风险。文章暗示，仅靠市场机制难以充分解决AI安全问题。

Published on May 1, 2025 12:33 AM GMT

To make frontier AI safe enough, we need to "lift up the floor" with minimum safety practices

Anthropic has popularized the idea of a “race to the top” in AI safety: Show you can be a leading AI developer while still prioritizing safety. Make safety a competitive differentiator, which pressures other developers to be safe too. Spurring a race to the top is core to Anthropic’s mission, according to its co-founder and CTO.¹

Is the race to the top working?

Competitive pressures can lead to some safety improvements.

When models aren’t reliable or trustworthy, customers get upset, and this creates pressure to fix problems. The CEO tweets on a Sunday, “we are working on fixes asap, some today.”

Currently both OpenAI and Anthropic are dealing with trustworthiness issues with their models, unwanted by either company.² And once one finds a fix, competitive pressure will mount on the other to quickly find a fix as well.

So, does that mean we can count on a race to the top to keep us all safe?

Not really. “Creating pressure to be safer” is different from “Making sure nobody acts unsafely.”

A race to the top can improve AI safety, but it doesn’t solve the “adoption problem”—getting all relevant developers to adopt safe enough practices.

For instance, Anthropic often cites its safety framework as evidence of how a race to the top can work—but Meta didn’t publish their own safety framework until 17 months later. If AI safety is a race to the top, it’s not a very fast one.

To Anthropic’s credit, they have not claimed that a race to the top is sufficient for AI safety, at least not to my knowledge. But media coverage sometimes suggests otherwise³ —perhaps because Anthropic doesn’t have a paired phrase that emphasizes the need for regulation.

It’s an important point, and so it bears saying clearly:

A “race to the top” must be paired with “lifting up the floor.” As AI systems become more capable, it is dangerous to rely on competitive pressures for getting frontier AI developers to adopt safe enough practices.⁴

In this post, I will:

Define Anthropic’s philosophy of “race to the top” (including an Appendix with detailed excerpts from Anthropic)Explain why frontier AI safety relies on “lifting up the floor”Explain why we shouldn’t rely on a “race to the top” being sufficient
Market forces won’t lead to universal adoption of safety practicesThe safety practices desired by the market won’t be strong enough to stop catastrophic risks
Describe ways of making a “race to the top” go better, including complementary policies

Continues here: https://stevenadler.substack.com/p/dont-rely-on-a-race-to-the-top
Twitter thread: https://x.com/sjgadler/status/1917649337519333612

Discuss