本文提出了一个衡量AI能力的框架，核心是通过AI加速AI研发劳动的能力来分级。它定义了“X倍AI”的概念，即AI在研发中能达到人力X倍的效率，并区分了3倍AI、10倍AI、顶尖人类专家主导AI（TEDAI）和超人类AI四个等级。文章强调，这种分级方式侧重于AI本身的能力，而非受外部因素影响的整体进展速度。同时，文章还探讨了这些不同等级的AI可能具有的特点以及与t-AGI、Anthropic ASL等其他框架的联系，旨在为讨论AI风险和能力提供更清晰的视角。

🚀AI研发加速能力：文章核心概念，指AI在研发中能达到的效率倍数，例如“3倍AI”表示AI的研发效率相当于人类的3倍。

🧠能力等级划分：文章将AI能力划分为3倍AI、10倍AI、顶尖人类专家主导AI（TEDAI）和超人类AI四个等级，每个等级都描述了AI在智能、速度、知识等方面的特征。

🤔替代框架比较：文章对比了t-AGI、Anthropic ASL等其他AI能力衡量框架，指出了各自的局限性，并强调了AI研发劳动加速框架的优势，即更侧重于AI本身的能力，减少外部因素的干扰。

⚙️等级能力推演：文章对每个等级的AI进行了推演，描述了它们在实际应用中的表现，例如，3倍AI可能像一个初级工程师，但速度更快、知识更丰富。

🔮未来发展展望：文章还对不同等级AI的发展速度进行了展望，认为我们可能会先看到3倍AI，然后才逐步发展到超人类AI，但具体过程仍具有不确定性。

Published on December 22, 2024 8:56 PM GMT

In a variety of conversations about AI misalignment risks, I find that it is important to be able to clearly point at different levels of AI capability. My current favorite approach is to talk about how much the AI accelerates AI R&D^[1] labor.

I define acceleration of AI R&D labor by Y times as "the level of acceleration which is as useful (for making more powerful AIs) for an AI company as having its employees run Y times faster^[2] (when you allow the total inference compute budget for AI assistance to be equal to total salaries)". Importantly, a 5x AI R&D labor acceleration won't necessarily mean that research into making AI systems more powerful happens 5x faster, as this just refers to increasing the labor part of the production function, and compute might also be an important input.^[3] This doesn't include acceleration of hardware R&D (as a pragmatic simplification).

Further, when I talk about AIs that can accelerate AI R&D labor by some factor, that means after being given some reasonable amount of time for human integration (e.g., 6 months) and given broad usage (but keeping fine-tuning and elicitation fixed during this integration time).

Why might this be a good approach? Because ultimately what we're worried about is AIs which can greatly accelerate R&D in general, and AI R&D in particular is worth focusing on as it could yield much faster AI progress, quickly bringing us to much greater levels of capability.

Why not just talk about the overall acceleration of AI progress (i.e., increases in the rate of effective compute increases as discussed in the Anthropic RSP) rather than just the labor input into AI R&D? Because for most misalignment-related discussions, I'd prefer to talk about capability levels mostly independent of exogenous factors that determine how useful that level of capability actually ends up being (i.e., independent from the extent to which compute is a bottleneck to AI research or the fraction of progress driven by scaling up hardware rather than algorithms). Rather than talking about overall AI progress or software progress labor acceleration, we could talk about the overall acceleration of just AI software progress (just algorithms, not compute increases)^[4], but this just adds the potential for compute bottlenecks without much benefit in discussions related to technical measures of misalignment. AI R&D labor acceleration doesn't fully avoid exogenous factors, but it avoids many such factors while still getting at a relevant and specific task.

I'll compare this approach to several alternatives later.

So, now we can talk about levels of capability like "3x AI R&D labor AIs". I'll call such systems "3x AIs" as shorthand.

Beyond discussing AI R&D labor acceleration, I think it is often useful to talk about the point when human cognitive labor is totally obsolete. Thus, I think it also makes sense to separately talk about Top-human-Expert-Dominating AI (TEDAI): AIs which strictly dominate top human experts in virtually all cognitive tasks (e.g., doable via remote work) while being at least 2x cheaper and faster^[5]. It is very unclear what level of AI R&D labor acceleration would occur with such systems, and this would be heavily dependent on factors like cost, speed, and the parallelizability of research^[6]. Sometimes the term AGI is defined such that TEDAI is equivalent to AGI, but I think defining a different precise term is useful for clarity.

Beyond the level of TEDAI, it can be worth pointing at very generally superhuman AIs: AIs which are generally qualitatively much more capable than humans and greatly dominate humans in virtually all cognitive tasks (while being cheaper and faster). This level of capability is much less precise, and it is very hard to say much at all about such systems.

Now, we can talk about the following levels of capability:

3x AIs10x AIsTEDAIVery generally superhuman AIs

(Thanks to Ajeya Cotra, Cody Rushing, Eli Lifland, Nate Thomas, Zach Stein-Perlman, Buck Shlegeris, and Claude 3.5 Sonnet for feedback on this post.)

What do I think these levels of capability look like?

Now that I've outlined these levels of capability, we can discuss what they might look like and what the rough conversion into other frameworks (like t-AGI) might be. I'll make some rough guesses here.

My sense is:

3x AIs:
Qualitative: The AI generally feels as smart as a pretty junior engineer (bottom 25% of new Google junior hires), but it is super knowledgeable, very good at some random tasks, very fast, very persistent, and is better than a pretty junior engineer at being an agent and generally knowing how to resolve software issues. It also looks really dumb in certain ways, sometimes does really stupid things, and sometimes has big robustness issues. The AIs are speeding things up partially via close collaboration with humans (imagine Cursor but with AIs being somewhat more autonomous) and partially via autonomously doing longer-run tasks while asking for human help. Plausibly, o3 is close to or already is a 3x AI.Total AI progress speed up: 3x AI R&D maybe corresponds to roughly 1.6x overall AI progress speed up, though this depends heavily on how much AI capabilities research is compute-bottlenecked. (With no bottleneck, 1.8x; with 50% reduction in acceleration due to bottleneck, 1.4x.)
The BOTEC here is assuming 2/5 of AI progress is on software; we do 2/5 (2 (1 - compute-bottleneck-tax) + 1) + 3/5. I think the compute bottleneck tax is probably around 25% with this acceleration (really, we should think about this in terms of tax brackets). (I think square rooting the labor multiplier is also a reasonable starting guess for the compute bottleneck.)
Anthropic ASL: This is probably late ASL-3 or early ASL-4 on Anthropic's breakdown (my speculation from public knowledge in the latest version of the RSP). It probably isn't ASL-4 for AI R&D but might be ASL-4 for bio or cyber.t-AGI: 4 hours (???)
10x AIs:
Qualitative: The AI generally feels as smart as a median engineer at a top AI company (OpenAI or Anthropic) while preserving (and in some ways increasing) its advantages discussed in the above bullet (knowledge, speed, agency) and still being somewhat less robust than a human.Total AI progress speed up: This is maybe around 3.2x overall AI progress speed up. (Again, this depends on compute bottlenecking; with no bottleneck, 4.6x; with 50% bottleneck, 2.8x.) These numbers don't include acceleration due to anything other than AI software R&D, as hardware R&D has longer lead times.Anthropic ASL: This is probably early ASL-5 or perhaps late ASL-4.t-AGI: 2 days (???)
Top-human-Expert-Dominating AI (TEDAI):
Qualitative: The AI generally feels roughly as smart as a top human expert and is able to dominate across virtually all domains via increasing capabilities further with other advantages.Total AI progress speed up: Very unclear and highly dependent on environmental factors. Perhaps AI R&D labor acceleration is >30x. I've seen BOTECs indicating roughly 15x overall AI progress speed. Human help is no longer relevant.Anthropic ASL: Should be ASL-5 or higher.t-AGI: >1 year
Very generally superhuman: ??? Everything is really, really hard to predict (as opposed to merely very hard to predict)

My qualitative guesses are focused on something like a nearcast with more focus on timelines where AI approaches haven't massively changed from where it looks like current approaches are going. This is because other cases are much harder to say anything about (and probably involve longer timelines).

Alternative capability breakdowns

t-AGI

I have two main problems with t-AGI:

I don't feel confident that horizon length will be the key variable, so I don't want to bake that into how we discuss capability levels. While AI R&D labor acceleration also makes some implicit assumptions, these assumptions seem much weaker.I don't feel like I have a very good handle on what various levels of t-AGI feel like, what level of t-AGI we have now, or even how one would measure this in principle. I do think we can measure AI R&D labor acceleration in principle, and I feel like I have a much better intuitive model.

Anthropic's ASL levels

These aren't defined above ASL-3, and the intention is that they will be defined with respect to the necessary level of mitigations (which in my opinion seems likely to focus on security). I've run into some cases where confusion about how ASL levels will end up being defined has caused issues with communication.

Purely qualitative breakdowns

Above, I describe qualitative intelligence of different systems. I expect that people will disagree radically about this (and already do). This is certainly hard to operationalize regardless. So, while this is often worth referencing, I don't think it should be the default approach to discussing capability levels.

Total AI progress speedup or total AI software progress (including compute bottlenecks)

As discussed above, I'm worried that total AI progress speed up pulls in a bunch of exogenous factors people often disagree about. A similar issue related to compute bottlenecks applies if you consider overall AI software progress speed up (rather than merely the labor input into this).

Will all these levels be passed at once?

I think we'll see a slow enough takeoff that I expect to see 3x AIs more than a year before very generally superhuman AIs, but it is unclear how slowly/smoothly we'll progress through units of AI R&D labor acceleration by default. Additionally, adoption delays make the picture more complex. Nonetheless, to the extent you were interested in talking about whether various mitigations would work at different levels of capability, I think AI R&D labor acceleration can be useful for this.

Conclusion

The AI R&D labor acceleration framework seems like a good approach for measuring and discussing AI capabilities, particularly for when discussing misalignment risk and mitigations. It compromises between a focus on the downstream implications of a capability level and on a more qualitative measurement of capability while still being relatively precisely defined.

---

1. I use AI R&D, but I expect these numbers would probably transfer fine to any sort of R&D that can be done digitally (in software), which is as measurable as AI R&D, and which the AIs are optimized for as much as AI R&D. ↩︎

2. Relative to only having access to AI systems publicly available in January 2023. ↩︎

3. You can also think about this as roughly being: "consider the subset of tasks that aren't bottlenecked by delays/costs in the environment (e.g., not bottlenecked by compute), how much can AIs accelerate people on average". ↩︎

4. Sometimes "software progress overall acceleration" is referred to as "software progress productivity acceleration", but I find "overall" clearer than "productivity". ↩︎

5. That is, 2x faster at accomplishing the tasks. ↩︎

6. Beyond human obsolescence, I think it generally becomes less helpful to talk about AI R&D labor acceleration when trying to point at different levels of capability for discussion about misalignment risks and mitigations. Partially this is because our understanding of what the systems will look like gets even worse after human obsolescence. ↩︎

Discuss