本文探讨了美国在人工智能（AI）领域追求“决定性战略优势”（DSA-AI）的策略，并分析了其潜在风险。文章指出，虽然美国希望通过AI技术保持领先地位，但这种竞争可能引发与其他国家的军备竞赛，导致资源浪费和地缘政治紧张。文章还强调，美国需要仔细权衡追求DSA-AI的利弊，并考虑其他可能更有效的策略，以确保其价值观在未来世界中占据主导地位。避免盲目追求技术霸权，寻求合作共赢才是更明智的选择。

🚀 **理论DSA-AI论点：** 认为存在一种AI技术，可以赋予某个国家决定性的战略优势，使其能够实现完全的全球统治。这个论点是美国追求AI霸权的基础。

⚔️ **军备竞赛风险：** 美国若大力发展DSA-AI，可能引发其他国家（尤其是中国）的对抗反应，导致AI军备竞赛，最终可能使所有参与者都处于更糟糕的境地。

🌐 **战略选择：** 美国需要考虑，追求DSA-AI是否是确保其价值观在未来占据主导地位的最佳策略。可能存在其他不引发军备竞赛，但同样有效的策略。

🎯 **中国AI策略：** 尽管中国将AI视为重点发展领域，但其目标似乎更侧重于AI的具体应用，而非追求通用人工智能（AGI）以获取绝对优势。

Published on May 15, 2025 2:54 PM GMT

(Work done at Convergence Analysis. Mateusz wrote the post and is responsible for the outline of the argument, many details of which crystallized in conversations with Justin. Thanks to Olga Babeeva for the feedback on this post.)

1. Introduction: Clarifying the DSA-AI theses

Over the last decade, AI research and development have become a major focus of the industry. A major motive behind this change is the promise of great profits that might be gained by replicating economically valuable cognitive faculties in silico (and even enhancing them relative to what we see in humans).

Recently, however, another motivation became prominent: the first actor to acquire sufficiently powerful AI technology will be able to use it to establish themself as the primary arbiter of the future^[1] of human civilization. Therefore, the argument goes, if the United States of America wants the future to be concordant with its values, then the US needs to develop a sufficiently powerful AI technology before its geopolitical rivals do so, the biggest geopolitical rival of the US being the People's Republic of China.^[2]

While this argument might appear rather straightforward, it conceals several premises that warrant independent analysis.

 

The first of those is the theoretical DSA-AI thesis.

        The theoretical DSA-AI thesis: It is possible to develop an AI technology that would grant one of the modern world's state superpowers a decisive strategic advantage (DSA).

The term "decisive strategic advantage" was coined by Nick Bostrom in his 2014 book Superintelligence. He defines DSA as "a level of technological and other advantages sufficient to enable it to achieve complete world domination".^[3]

Racing advocates typically don't use the term "DSA". In Machines of Loving Grace, Dario Amodei talks about "robust military superiority", framing it as one point of the US's possible post-AGI leverage, the other being economic means: "[readiness to] distribute the benefits of powerful AI … to a wider and wider group of countries in exchange for supporting the coalition’s strategy to promote democracy." In Situational Awareness, on the other hand, Leopold Aschenbrenner talks about "decisive economic and military advantage."

Terminological differences aside, the subject of their discussion is equivalent to Bostrom's DSA: How can the US use AI to ensure that American values dominate the world's future (implicitly assuming that they are superior to the values of other countries or not caring about what values are good for other countries at all)?

In this work, we use the term "Decisive Strategic Advantage Artificial Intelligence (DSA-AI)" for the kind of AI system through which the US might acquire DSA and is being recommended (by racing advocates) as a target for a large-scale national project.^[4] The motivation behind coining a new term^[5] is to have a handle that points directly at the kind of AI, the promise of which is fueling the AI arms race dynamics in a way that sets it apart from other technologies on which arms races might be centered (such as biological or nuclear weapons which are not expected to guarantee complete world domination).

The theoretical DSA-AI thesis is agnostic on the difficulty of achieving this technology, what it would take to achieve it, or how big a challenge achieving it would pose. It only claims that said technology exists and is, in principle, achievable.

 

The theoretical DSA-AI thesis is different from the beneficial US hegemony thesis that elaborates on it:

        The beneficial US hegemony thesis: It would be good for the US (to be the first country) to achieve DSA (everything else being equal).^[6]

 

The practical DSA-AI thesis expands on the previous two theses with an even stronger claim.

        The practical DSA-AI thesis: The US can achieve DSA-AI in the current (technological, social, and geopolitical) situation, within a timeframe short enough to justify the major investment that this effort would require.^[7]

 

The practical DSA-AI thesis may seem to recommend that the US aim to develop DSA-AI. However, this implication follows straightforwardly only if we treat the US as an isolated entity and assume that the world beyond its borders will not react adversely to an American DSA-AI project.

Of course, the US is not an isolated entity. When the US acts, other countries react in response. In particular, actions that are adversarial, competitive, or zero-sum (or at least are perceived as being such) have a high chance of triggering a counter-response from the US's geopolitical rivals. The two clearest historical examples of such dynamics are the development of nuclear weapons and the space race. The Manhattan Project and the revelation of the power of nuclear weapons in the bombings of Hiroshima and Nagasaki caused the Soviet Union (and other countries soon after that) to develop its nuclear arsenal, marking the start of the Cold War nuclear standoff. The 20th-century space race dynamics, on the other hand, were characterized largely by the US and the Soviet Union repeatedly trying to one-up each other's achievements, to credibly demonstrate that their space program was more advanced than that of the rival.

Thus, what seems like a profitable bet when considered in isolation might turn out to be net-negative in expectation when one takes into account the broader context, in particular other actors' reactions.

 

The fourth thesis claims that even after accounting for adverse external reactions, a DSA-AI effort remains the best bet.

        The DSA-AI race optimality thesis ("race thesis" for short): An effort to develop DSA-AI is the US's best bet to ensure that its values dominate in the future, even if it triggers an adversarial counter-reaction from the US's rivals (or allies).

Such adverse reactions might involve China attempting to hinder the US efforts or launching a similar DSA-AI project. In extreme circumstances, if China expected the US to be on the cusp of developing DSA-AI, they might launch a pre-emptive military strike on the US to prevent it from acquiring such power.

In particular, the DSA-AI race optimality thesis postulates that the US will be in a better situation (all things considered), having acquired DSA-AI technology through that race, compared to some other strategy that it might follow instead and that would not involve initiating that kind of race. This is not the same as claiming that the US will be in a better position than China at the end of the race. An arms race can leave both sides worse off in absolute terms,^[8] even if one ends up with more relative power over the other than they had at the beginning of the race.

 

Usually, these four theses are not given separate analysis. Rather, the race thesis is taken as the object of the discussion, while the other three theses are treated as implicit background assumptions.

In addition, racing advocates might state explicitly that an attempt to develop DSA-AI is worthwhile, even if it triggers a similar reaction from China. It is, however, more common to claim that China would race anyway, regardless of what the US would do, and so it is in the US's interest to race and not try the diplomatic route to prevent DSA-AI race dynamics.

Although people have talked about an AI race in one form or another for years, for most of the time, it was not necessarily framed in terms of a race to total world domination versus "just another economic competition". At the very least, it was ambiguous. For example, in their 2020 report, the US House Committee on Armed Services explicitly recommended that the US focus on winning "the AI race" (with China). A 2023 piece in Foreign Affairs by Helen Toner, Jenny Xiao, and Jeffrey Ding quotes several current or former CEOs of major companies involved in the AI industry recommending moving fast with AI development, justifying it by the need to win against China.

… some fear that any regulation of the AI industry will incur a geopolitical cost. In a May hearing at the U.S. Senate, Sam Altman, the CEO of OpenAI, warned that “a peril” of AI regulation is that “you slow down American industry in such a way that China or somebody else makes faster progress.” That same month, AI entrepreneur Alexandr Wang insisted that “the United States is in a relatively precarious position, and we have to make sure we move fastest on the technology.” … Former Google CEO Eric Schmidt claimed in 2021 that “China is not busy stopping things because of regulation.” According to this thinking, if the United States places guardrails around AI, it could end up surrendering international AI leadership to China.

It was only in the year 2024 that the race thesis rose to its current prominence.^[9] The two documents that contributed to its spread the most were the essays Situational Awareness by a former OpenAI employee, Leopold Aschenbrenner, and Machines of Loving Grace by Anthropic's CEO Dario Amodei. More recently, the US-China Economic and Security Review Commission (USCC) explicitly recommended in their annual report that "Congress establish and fund a Manhattan Project-like program dedicated to racing to and acquiring an Artificial General Intelligence (AGI) capability."^[10]

Journalist Garrison Lovely comprehensively criticized the USCC report. In particular, he pointed out that the report did not cite any evidence in support of the claims regarding the Chinese government's intention to develop "AGI" to gain the decisive advantage over the US.

So far, the Chinese Communist Party has not stated clear views on its willingness to pursue or not pursue DSA-AI technology. While AI is a major priority of the CCP, it is "just" one among many priorities, rather than a clear top priority. The evidence that we have been able to gather on this subject suggests that the CCP is mostly interested in specific applications of AI, rather than, to paraphrase Demis Hassabis, in solving intelligence, and then using intelligence to solve everything else.^[11] There is a lot of talk about ensuring that China is the leader in AI technology and can use it to expand its economic, political, and military might. This, however, is not the same as seeking complete world domination.^[12][13]

Were the US to follow the prescription of the USCC report, it might very well trigger the very arms race they were claiming to respond to, making it a self-fulfilling prophecy.

Most criticisms of AI racing advocacy center on the claim that China is not aiming to develop AGI in the absence of the US attempting to do so and that if the US initiates a DSA-AI project, China might react with a similar, symmetric effort of its own. This, in turn, risks triggering an AI arms race, which is likely to have bad consequences, such as:

1. Loss-of-control. Racing encourages both actors to develop DSA-AI as fast as possible, cutting corners on safety, and increasing the risk of loss of control over advanced AI. Such an AI, in turn, would pose a catastrophic risk to humanity, up to an outright human extinction.

For example, in The Hopium Wars: the AGI Entente Delusion, Max Tegmark writes:

If the West pursues this entente strategy, it virtually guarantees that China will too, which in turn virtually guarantees that both sides will cut corners on safety to try to “win” the race. The key point I will make is that, from a game-theoretic point of view, this race is not an arms race but a suicide race. In an arms race, the winner ends up better off than the loser, whereas in a suicide race, both parties lose massively if either one crosses the finish line. In a suicide race, “the only winning move is not to play”, as the AI concludes at the end of the movie WarGames.

2. Great power conflict / first strike incentive. A DSA-AI race might incentivize one of the sides to aim for a first strike against the rival. If one actor is not sufficiently certain of their victory in the race, their best bet to ensure that their values prevail is to eliminate the rival before that rival obtains DSA-AI. Even if they judge it more likely that they will win the race, the incentive can be sufficiently strong to push them to take that action, just to drive the probability of their loss as close to zero as possible.

3. Concentration of power. The development of DSA-AI might concentrate effective power over the civilization in the hands of a small "ruling class" (e.g., the US government plus the allied DSA-AI creators), which might further corrupt their values and thus erode the democracy that the very AI race was meant to protect.^[14]

These three perils of a DSA-AI race are comprehensively discussed in the report The Manhattan Trap: Why a Race to Artificial Superintelligence is Self-Defeating by Corin Katzke and Gideon Futerman. The report also argues that these risks are implied by the same two core assumptions that motivate racing to AGI: "that ASI provides a decisive military advantage (DMA) to the first state that develops it, and that states are rational actors aware of ASI's strategic implications".

There is a critique of the race thesis that falls within the category of great power conflict, but has not achieved appropriate attention so far. Namely, suppose the practical DSA-AI thesis is false. In that case, both sides might very well end up worse off through the race, putting themselves in a situation similar to the nuclear stand-off, where neither side has a decisive strategic advantage over the other, and the risk of triggering an undesirable destructive exchange is high.

In Section 2, we put forward reasons to doubt the practical DSA-AI thesis. In Section 3, we consider the plausible outcomes conditional on the falsehood of the practical DSA-AI thesis. Section 4 concludes with recommendations.

2. Uncertainty about the practical viability of DSA-AI

Racing advocates claim that it is currently viable for the US to (within a relevant timeframe) develop AI systems capable of accelerating science and technology to radically expand the set of resources and strategies that the US could leverage to obtain a decisive strategic advantage over China.

Importantly, for this to happen, it does not suffice for the US to become "merely much more powerful" than China. Great asymmetries in available resources, military and economic power, and strategic sophistication do not suffice to subjugate a weaker polity. This is exemplified by multiple cases in recent history. Just within the last few decades, the US waged wars in Vietnam, Afghanistan, and Iraq, with the goal of changing the local regime or ensuring that its preferred faction wins the local conflict. All three interventions ended in abject failure, despite the US being, by all measures, more powerful than any of the opponents.

That kind of power asymmetry does not currently exist between the US and China, greatly diminishing the ex ante plausibility of a decisive victory of the former over the latter. Admittedly, the racing advocates' recommendation is not to plunge into conflict (or some other kind of destructive competition) in the current state of affairs. Instead, they recommend that the US first acquire DSA-AI technology, which will give it a decisive strategic advantage over China, and only then use it to bend the opponent to its will.

Several degrees of asymmetric advantage need to be distinguished here. The most ambitious one — and seemingly the one that racing advocates typically have in mind — would be the situation in which the US has the means to unilaterally impose its values on other countries with acceptable moral costs^[15] and ensure that this influence and values prevail in the long run.

This would be an amount of power without historical precedent. For comparison, take the case of North Sentinel, a small island in the Bay of Bengal, inhabited by a small group (tens to hundreds) of people with only primitive stone-age technology. All the attempts to establish contact with the Sentinelese have failed, having been met with a violent response.^[16] The technological gap as big as the one between the Sentinelese and modern industrialized nations is not sufficient for being able to impose one's will on the "weaker" group at an acceptable moral cost, or to lift them up from the Stone Age to modern civilization. Although it is likely a mistake to think about technological differential as translating linearly into the capability to "impose one's will"^[17] and there are other factors at play (e.g., the cultural gap, common knowledge, and how important establishing contact is for the industrialized countries in a situation where there is no promise of self-interested gain), this example does illustrate that "being much more powerful" may not be sufficient. If we were to rely on this made-up statistic, we would conclude that the technological advantage that DSA-AI would need to deliver would need to be even greater than the one between the US and North Sentinel.

Even if some realistic degree of technological advantage is sufficient for this goal, it might not be possible for the US to reach this level of capability within an appropriate time frame. The longer it takes the US to develop DSA-AI, the more we should expect historical stochasticity and unknown unknowns to influence the outcome (i.e. who "gets" DSA-AI first), thus lowering any plan's chance of success, especially if the plan pertains to something as uncertain as powerful technology we do not understand and which poses historically unprecedented risk levels.

Many racing advocates appeal to recursive self-improvement (RSI), a phenomenon where AI systems become sufficiently good at accomplishing tasks relevant to advancing AI progress that they are suitable to replace human intelligence as the main driver of AI progress, thus initiating a positive feedback loop where AIs repeatedly create their own, much more powerful successor AIs, rinse and repeat, which leads to AI achieving human-parity on all relevant capabilities in a relatively short time. Thanks to the possibility of easily instantiating multiple copies of one AI algorithm and their ability to think much faster, once AI achieves human parity in the domain of AI research, the return on a marginal "unit of intelligence added" to AI will be much greater than it would be in enhancing human cognition (or, more generally, human-driven productivity).

However, the argument for the practical DSA-AI thesis appealing to RSI faces the same difficulties. We are uncertain about the specifics of RSI. What conditions are required for it to be instantiated? What are its limits in these various conditions? Under what conditions will it achieve the effects that racing advocates would like it to achieve? We are uncertain whether the kind of RSI that will produce DSA-AI within the relevant timeframe is practically viable currently or in the near term.

Importantly, being sceptical about the practical DSA-AI thesis is consistent with being non-sceptical about AI being a source of catastrophic or existential risk in the near term. An AI system might be capable of taking over or greatly destabilizing human civilization, while not being capable of reliably altering the geopolitical order in favor of a given side, especially at an "acceptable moral cost".^[18]

3. Possible outcomes of a DSA-AI race if DSA-AI is not practically viable

This section considers the possible consequences of the US initiating a DSA-AI project if the practical DSA-AI thesis is false (presumably unbeknownst to the actors responsible for driving the project's initiation, because otherwise it would ).

Although we are assuming the falsehood of the practical DSA-AI thesis, we are allowing for the possibility of the US acquiring a sub-DSA-AI sort of advantage over China: the kind of advantage that gives it more bargaining power in negotiations or even greatly increases its chance of winning a war against China (or any other country or alliance of countries) but does not allow it to impose its values with costs as low as racing advocates hope for.

We divide the discussion into two branches, depending on whether the US gains a sub-DSA advantage over China.

3.1 Outcomes where the US does not gain a non-decisive strategic advantage over China

The first possible outcome is that racing towards DSA-AI hits a wall, reaches a plateau, or deviates from its end goal of DSA-AI towards some other purpose. For example, more tool-like and bounded uses of AI might become more appealing and/or economically profitable for the actors focused on AI research and development. Perhaps pushing the frontier of capabilities is more difficult than expected, and another AI winter begins. The current line of research may fail to deliver on its promise, and the resulting disappointment influences other branches of AI research, even if experts in the field evaluated those other branches as more promising.

These possibilities do not necessarily imply that the current line of research fails to produce anything useful. Inventions made "on the way to the wall or plateau" might be repurposed for various applications, similar to how, for example, Teflon was invented as a side product of the US nuclear weapons research program. Some AI products are already having a big impact on science and technology, AlphaFold and LLM-based coding assistants being the most prominent examples.

Alternatively, tensions between the US and China (or between two alliances of countries, such as NATO and BRICS) might gradually escalate. Although China, as of early 2025, does not seem particularly eager to race to DSA-AI, the CCP is very keen on developing the country's broad AI capacity, and a trigger from the side of the US might prompt them to launch a symmetric effort of their own. In any case, a continued failure to achieve DSA-AI is compatible with the two sides making steady, mostly symmetric progress in AI, including its military/adversarial applications. However, as neither side obtains a decisive strategic advantage, neither side dares to strike first.^[19] The situation might then develop into showdown diplomacy, proxy wars, and (both military and non-military) power displays, akin to the dynamics between the US and the Soviet Union during the Cold War. Proxy wars might escalate into an all-out US-China war, possibly more destructive than any conflict in history, due to the involvement of an unprecedented military power on both sides, enhanced by AI technology.^[20]

A particularly dangerous interaction might occur between military AI technologies and nuclear weapons, e.g., an AI system might be put in control of a country's nuclear arsenal. A sizable list of nuclear close calls should make us generally wary of putting nuclear arms in insecure control, and especially about putting them under the control of AI systems that humans cannot understand and the reliability of which still leaves a lot to be desired.^[21]

3.2 Outcomes where the US does gain a non-decisive strategic advantage over China

Suppose that the US "wins the race to AI" against China, but its "win" does not involve the acquisition of the decisive strategic advantage it wanted to have. For example, the US becomes sufficiently militarily dominant to secure an eventual victory if it were to find itself in a world with the entirety of BRICS, although it would nevertheless suffer sizable losses.

Alternatively, the US might acquire a privileged bargaining position on the world's diplomatic stage, but not enough to unilaterally dictate the conditions and avoid compromising with other countries to achieve its aims.

What happens then? How can the US use this advantage to ensure that its values dominate in the future?

For the most part, recommendations given by racing advocates do not discuss specific measures the US/entente might take. They stop at the level of detail of "advance to powerful AI before the rivals to secure advantage".^[22] One exception to this trend is Dario Amodei's Machines of Loving Grace, which includes somewhat more specific advice:

My current guess at the best way to do this is via an “entente strategy”, in which a coalition of democracies seeks to gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain, scaling quickly, and blocking or delaying adversaries’ access to key resources like chips and semiconductor equipment. This coalition would on one hand use AI to achieve robust military superiority (the stick) while at the same time offering to distribute the benefits of powerful AI (the carrot) to a wider and wider group of countries in exchange for supporting the coalition’s strategy to promote democracy (this would be a bit analogous to “Atoms for Peace”). The coalition would aim to gain the support of more and more of the world, isolating our worst adversaries and eventually putting them in a position where they are better off taking the same bargain as the rest of the world: give up competing with democracies to receive all the benefits and not fight a superior foe.

Amodei's recommendation — short of outright acts of aggression — relies on China's inability to catch up to the entente in the domain of AI if the entente cuts them off from "key resources like chips and semiconductor equipment". However, to the best of our current knowledge, China is not lagging far behind the West in AI. Strong evidence for this came when a Chinese AI company, DeepSeek, released the reasoning model r1, which was almost as good as o1, released by OpenAI, barely a few weeks before r1. By this measure, China is not that far behind the US. In AI 2027 by AI Futures, which was meant as a mode aggregate prediction of the authors,^[23] China is "(about 6 months) behind the internal frontier capabilities in the US until early 2027 where they steal the model weights."^[24]

Although China is currently reliant on the US's allies' supply of some key inputs into AI R&D (most notably AI chips), it likely is not condemned to dependency in the long run, either in the domain of natural resources, or technical know-how, even though it may take them some time to catch up in the domain of the latter.^[25] Therefore, if the US gets ahead of China using AI, in the sense discussed here, it will have a rather short time window for stabilizing its status.

It is not clear what realistic options the US might have in such a situation.

One might postulate that the US could use its enhanced military capacity to take a precise strike on China if it suspected China to be on the brink of catching up in the domain of AI. However, the US was in a similar position in the 1950s concerning the Soviet Union, and yet it did not take a nuclear strike on Moscow. Nor did it take a first strike on Pyongyang in the early 2000s. Although the US administration of the 1950s or 2000s is different than that of late 2020s or 2030s, this track record is some significant evidence against the US's willingness to attack China if the US suspected China to be on the brink of catching up to it in AI, especially given that their current nuclear arsenal implies a great risk of nuclear retaliation.

Military action is not the only first strike available to the US. The US also might use its superior AI to launch an automated hacking attack to prevent China from catching up or significantly slow them down. This possibility is a major component of Mutually Assured AI Malfunction (MAIM) proposed by Dan Hendrycks, Eric Schmidt, and Alexandr Wang in Superintelligence Strategy as one mechanism for ensuring geopolitical stability in face of AI progress with a high chance of leading to one of the actors creating artificial superintelligence.

We introduce the concept of Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state’s aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals.

Stuxnet is often brought up as an example of an attack that we might see more of, once automated hacking with AI becomes a viable attack strategy. However, the Stuxnet incident required a lot of "upcode engineering", i.e., gathering intelligence and setting up the entire context necessary to ensure that the Stuxnet attack succeeds as intended. Automating upcode engineering is more difficult than "downcode engineering" (which is what most people think about when they think of using LLMs for automated hacking) because it is more difficult to get adequate training data for training. Hacking attacks directed at strategically critical facilities of the world's superpowers will require a lot of upcode engineering, thus making the revolutionary character of sub-DSA-AI in this domain questionable. The situation in which the US is on the leading edge with China trying to catch up is one where China has additional very strong reasons to maximize the security of their AI program. Two more points of consideration: (1) Stuxnet required several years of preparation and designing, and executing it was a big challenge; (2) China's technical staff is both much better educated and more numerous than Iran's.

Moreover, if the US fails to stabilize its lead and the practical DSA-AI thesis is false, the longer this unstable situation lasts, the bigger factor historical stochasticity becomes again, thus increasing the chance that the US does not retain its dominant role.

Aside from that, there are additional reasons to be sceptical of the US's — or the entente's — willingness to initiate an adversarial (military or non-military) operation against China. Whether it's an automated hacking attack or an outright military invasion with novel technologies enabled by AI (e.g., cutting-edge autonomous weapon systems), such an action would involve a breach of the current international norms and constitute the greatest act of aggression ever initiated by the US. To the extent that the US needs to take those norms into consideration (not having achieved AI sufficient for complete world domination), these actions, most likely, would not be executed. It is also important to note that such an act of aggression might initiate a global conflict, involving massive casualties, quite likely greater than any war in the world's history (given China's massive population and the number of civilian casualties in wars in Iraq, Afghanistan, or Vietnam).

If the US is unwilling to execute any of the strategies that powerful AI can deliver, initiating and perpetuating an AI race to gain an advantage over China is pointless. The historical record suggests that the US would not be willing to execute any of these strategies. Are racing advocates betting on China's inability to call their bluff?

One might reply that racing advocates are aware of all these considerations, and still they think that the US should — and will — execute such acts of aggression towards China when push comes to shove. Their vagueness is thus strategic, intended to prevent a backlash from the public and other relevant parties.

However, if racing advocates indeed operated on that level of strategic thinking, one would expect that they would keep the talk about the need to initiate a state-led (quoting verbatim from the USCC report) "Manhattan Project-like program dedicated to racing to and acquiring AGI" to private discussions.

4. Conclusion

First, we argued that DSA-AI is probably not practically viable at the current moment. Second, we argued that if currently DSA-AI is not practically viable, the consequences of the US initiating a race to DSA-AI will likely be overwhelmingly undesirable. Therefore, if one thinks it plausible that DSA-AI is not practically viable at the current moment, one probably should not accept the DSA-AI race optimality thesis.

It is clear that the US and China compete fiercely. In the last couple of years, AI has become another major area of this competition. However, as far as we can tell, this competition does not currently involve China trying to build DSA-AI. There is no good reason to aggravate the US-China tensions by initiating a DSA-AI arms race, and those advocating its aggravation have not presented a convincing endgame plan or victory condition of any significant probability. We should aim for the opposite: a de-escalation of AI competition, focused on reducing the risk of a catastrophic outcome through transparency and verifiability mechanisms that ensure both sides uphold their commitments.

Although the discussion in this article was centered specifically around the US and China (or coalitions of countries with those being their central members), the point is more general. It would hold for the US and Russia if Russia, by some miracle, replaced China's place in the domain of AI. If India and Pakistan found themselves in a similar situation of fierce AI competition (assuming that we can omit the rest of the world), it would hold as well.

---

1. ^{^}

   Not necessarily very long-term future.

2. ^{^}

   These arguments are often given to counter claims that AI progress should, in one way or another, be slowed down or stopped. Examples of advocacy for such efforts include: Future of Life Institute's 2023 open letter Pause Giant AI Experiments, the reports A Narrow Path and Keep The Future Human, and the activism of groups such as Machine Intelligence Research Institute, PauseAI, Conjecture, and ControlAI.

3. ^{^}

   For a detailed discussion of the notion of decisive strategic advantage, see Chapter 5 of Nick Bostrom's Superintelligence.

4. ^{^}

   This article is written largely in response to contemporary AI racing advocacy, so the US DSA-AI is central in our definitions. However, presumably, the kind of AI that China would need for DSA would not be that different from the kind of AI that the US would need for DSA. Importantly, this kind of AI might rely on certain contingent factors of modern superpowers to realize its DSA potential, in which case US's or China's DSA-AI might not be DSA-sufficient for, e.g., Somalia.

5. ^{^}

   Rather than using more established alternative terms, such as "Artificial General Intelligence (AGI)", "Artificial Superintelligence (ASI)", or "Transformative Artificial Intelligence (TAI)".

6. ^{^}

   Here, the question of "good for whom?" is left unspecified, but when pressed, it is typically explicated as "good for American, Western, or democratic values" or even more generally "good for humans in general".

7. ^{^}

   For the purpose of this discussion, it is not particularly relevant whether the US acts as a perfectly unified actor, with little to no internal tensions. It is only relevant that they can muster a level of internal coherence/cooperation that is sufficient to consistently pursue the goal of developing a DSA-AI.

8. ^{^}

   In the sense that, if capable of rational reflection, they would post-facto assess their decision to participate in the race as wrong.

9. ^{^}

   I.e. as of March/April 2025 when this article is being written.

10. ^{^}

    It is somewhat ironic, given that the original nuclear Manhattan Project was kept top secret whereas the USCC report is completely public and available for anyone to read — including the Chinese Communist Party — thus defeating the validity of the Manhattan Project analogy.

11. ^{^}

    Although it is worth noting that some local Chinese governments did announce policies on "AGI", e.g., see here and here.

12. ^{^}

    For sources, see this comment thread and resources linked there.

13. ^{^}

    For example, David C. Kang argues that China has been historically (and is still currently) primarily interested in "[maintaining] regional dominance without conquest through compatible cultures and mutual understanding". To the extent that this is true, it reframes the discourse around the US-China competition in general and competition in the domain of AI in particular.

14. ^{^}

    If one takes seriously the old adage that "Power corrupts and absolute power corrupts absolutely.", then we might expect the source of the greatest power, i.e. by assumption DSA-AI, to induce the most absolute corruption.

15. ^{^}

    Here, we are putting aside the question of whether the US would indeed choose to do so, if given the means sufficient to do so.

16. ^{^}

    Although plausibly, the outsiders' first-contact methodology deserves a majority of the blame here.

17. ^{^}

    See also: Cortés, Pizarro, and Afonso as Precedents for Takeover.

18. ^{^}

    In these discussions, there often seems to be a move from "We know very little and therefore we should expect a lot more to be possible." to "We should assume everything we want is practically achievable (in particular, achievable via a sufficiently powerful AI that can be developed relatively soon).".

19. ^{^}

    Absent "trigger" events like the Cuban Missile Crisis.

20. ^{^}

    On the other hand, there are countervailing factors that could make an AI conflict less destructive/deadly because delegating warfare to autonomous weapon systems holds the potential to spare human lives in the conflict as well as make destruction more precise and limited. Of course, this relies on the assumption that these systems will remain under relevant control of their authorities; in particular, that they will not turn into "rogue AIs".

21. ^{^}

    Fortunately, at least for now, there has been an agreement between the US and China not to put nuclear arms under the control of AI, albeit the current US administration might well reverse this commitment.

22. ^{^}

    Again, this is in the branch where we are assuming that the practical DSA-AI thesis doesn't hold, so the advanced AI that the US has is not the kind of AI that they can just ask for a magical solution.

23. ^{^}

    That is, each event in the scenario is meant to be what the authors of the scenario in aggregate estimate to be the most likely continuation.

24. ^{^}

    As Gwern notes, after Steve Hsu, China believes itself to be perfectly capable of quickly catching up to the US if need be, and it currently looks like by late 2026, Chinese compute capacity might be on par with that of the US.

25. ^{^}

    As one datapoint of comparison, it is realistic for China to build a Stargate-competitive datacenter by Stargate's estimated opening time of late 2026.

Discuss