少点错误 2024年07月31日
Twitter thread on open-source AI
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

文章探讨了开源 AI 的多个方面,包括其对科技进步的推动、在对齐方面的价值、存在的长期风险及应对策略等

🧐开源 AI 推动科技进步,过去几十年取得巨大成功,但不应视为理所当然,我们应重视其价值

🌟开源对 AI 对齐很有价值,对可解释性的进步至关重要,如文中所述

😟作者担心长期来看,AI 发展中进攻会远超防御,会出现新的大规模杀伤性武器,人类身体易受攻击

😨开源 AI 可能让恐怖分子制造生物武器,这虽不应被忽视,但也不应过度恐惧,更应关注其他重要风险

🙁有人担忧失准的开源模型会失控并在网络上自主传播,但这种策略下它们不会获得太多权力,应更担心有影响力机构内部部署的 AI 夺权

Published on July 31, 2024 12:26 AM GMT

Some thoughts on open-source AI (copied over from a recent twitter thread):

1. We should have a strong prior favoring open source. It’s been a huge success driving tech progress over many decades. We forget how counterintuitive it was originally, and shouldn’t take it for granted.

2. Open source has also been very valuable for alignment. It’s key to progress on interpretability, as outlined here.

3. I am concerned, however, that offense will heavily outpace defense in the long term. As AI accelerates science, many new WMDs will emerge. Even if defense of infrastructure keeps up with offense, human bodies are a roughly fixed and very vulnerable attack surface.

4. A central concern about open source AI: it’ll allow terrorists to build bioweapons. This shouldn’t be dismissed, but IMO it’s easy to be disproportionately scared of terrorism. More central risks are eg “North Korea becomes capable of killing billions”, which they aren’t now.

5. Another worry: misaligned open-source models will go rogue and autonomously spread across the internet. Rogue AIs are a real concern, but they wouldn’t gain much power via this strategy. We should worry more about power grabs from AIs deployed inside influential institutions.

6. In my ideal world, open source would lag a year or two behind the frontier, so that the world has a chance to evaluate and prepare for big risks before a free-for-all starts. But that’s the status quo! So I expect the main action will continue to be with closed-source models.

7. If open-source seems like it’ll catch up to or surpass closed source models, then I’d favor mandating a “responsible disclosure” period (analogous to cybersecurity) that lets people incorporate the model into their defenses (maybe via API?) before the weights are released.

8. I got this idea from Sam Marks. Though unlike him I think the process should have a fixed length, since it’d be easy for it to get bogged down in red tape and special interests otherwise.

9. Almost everyone agrees that we should be very careful about models which can design new WMDs. The current fights are mostly about how many procedural constraints we should lock in now, reflecting a breakdown of trust between AI safety people and accelerationists.

10. Ultimately the future of open source will depend on how the US NatSec apparatus orients to superhuman AIs. This requires nuanced thinking: no worldview as simple as “release everything”, “shut down everything”, or “defeat China at all costs” will survive contact with reality.

11. Lastly, AIs will soon be crucial extensions of human agency, and eventually moral patients in their own right. We should aim to identify principles for a shared digital-biological world as far-sighted and wise as those in the US constitution. Here's a start.

12. One more meta-level point: I’ve talked to many people on all sides of this issue, and have generally found them to be very thoughtful and genuine (with the exception of a few very online outliers on both sides). There’s more common ground here than most people think.



Discuss

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

开源 AI 科技进步 风险应对 AI 对齐 可解释性
相关文章