本文是NAO内部于2024年11月撰写的一份策略备忘录，探讨了随着LLM（大型语言模型）能力提升，AI加速自身发展对NAO工作的影响。文章认为，AI的快速发展将改变NAO的应对策略，需要重新调整优先级。作者建议NAO优先关注那些可能成为未来AI加速工作瓶颈的领域，如数据收集和合作伙伴关系。通过预先规划，NAO可以更好地利用AI带来的优势，从而更有效地应对潜在挑战。

💡作者认为，随着LLM能力的提升，AI将加速自身发展，可能在未来几年内实现巨大进步。

⚠️作者最初认为AI对NAO的主要影响在于加速潜在对手的发展，因此强调快速建立全面的检测系统的重要性。然而，作者现在认为需要重新调整优先级。

🔑作者建议NAO优先考虑那些可能成为未来AI加速工作瓶颈的领域。例如，如果NAO拥有更大的虚拟计算团队，那么数据收集的广度和深度将变得至关重要，因为AI可以更有效地从数据中提取见解。

⏳作者强调，尽早开始大规模数据收集至关重要，因为AI无法提供历史数据。此外，建立更多合作伙伴关系以获取样本和数据也需要优先考虑，因为这需要真实的人力投入。

Published on April 19, 2025 11:40 AM GMT

Cross-posted from my NAONotebook.

This is an internal strategy note I wrote in November 2024 that I'mmaking public with some light editing.

In my work at the NAO I've been thinking about what I expect to see asLLMs continue to become more capable and get closer to where they cansignificantly accelerate their own development. I think we may see very large advances in the power ofthese systems over thenext few years.

I'd previously thought that the main impact of AI on the NAO wasthrough accelerating potential adversaries, and so shorter timelinesprimarily meant more urgency: we needed to get a comprehensivedetection system in place quickly.

I now think, however, that this also means the best response involvessome reprioritization. Specifically, AI will likely speed up someaspects of the creation of a detection system more than others, and soto the extent that we expect rapid advances in AI we should prioritizethe work that we expect to bottleneck our future AI-accelerated work.

One way to plan for this is to imagine what would be the mainbottlenecks if we had a far larger staff. Imagine if each seniorperson had AI support equivalent to all the smart junior people theycould effectively manage. Or even (but my argument doesn't depend onthis) AI systems that are as capable as today's experiencedresearchers. I think if in a year or two we found ourselves in thissituation we would wish that:

We had collected a lot more data, because with a very large virtualcomputational staff future-AI-assisted-NAO can wring insights out ofdata far more efficiently than present-NAO.

We had started large-scale collection sooner, because, even if AIaccelerates sequencing's price decreases and we can collect a lot moredata in the future, it can't give us historical data.

We had a lot more partnerships for bringing in samples and data,because these take real-human time to scale up.

While I don't think this is the only way things could play out, Ithink it's likely enough that we should be taking these considerationsvery seriously in our planning.

April 2025: since initially drafting this we've started anambitious effort to scaleup our pilot system.

Comment via: facebook, mastodon, bluesky

Discuss