Exceptional AI
Emergency Politics
“Sovereign,” writes Carl Schmitt, “is he who decides on the exception.”1 And we live in exceptional times. The White House tells us that “foreign trade and economic practices have created a national emergency,”2 that “the United States’ insufficient energy production, transportation, refining, and generation constitutes an unusual and extraordinary threat to our Nation’s economy, national security, and foreign policy,”3 that “unchecked unlawful mass migration and the unimpeded flow of opiates across our borders continue to endanger the safety and security of the American people and encourage further lawlessness,”4 that “any effort by the ICC to investigate, arrest, detain, or prosecute protected persons…constitutes an unusual and extraordinary threat to the national security and foreign policy of the United States,”5 and “that underlying conditions, including a lack of reciprocity in our bilateral trade relationships, disparate tariff rates and non-tariff barriers, and U.S. trading partners’ economic policies that suppress domestic wages and consumption, as indicated by large and persistent annual U.S. goods trade deficits, constitute an unusual and extraordinary threat to the national security and economy of the United States.”6 We are, it seems, surrounded by emergencies that demand the suspension of the regular legal order.
In fact, the Trump administration has just announced an export control directive to suspend access to Anthropic’s most advanced AI models.7
The Parallel State
In Bomb Power, Gary Wills argues that modern emergency politics emerge from the development of nuclear weapons during World War II. From its inception, the Manhattan Project was a different kind of government endeavor: secret,8 outside the regular legal order, and once successful managed in part outside of the institutions of constitutional government. The development, production, and maintenance of the nuclear arsenal created a parallel institutional structure — the national security state — defined by an “apparatus of secrecy and executive control,” necessitated by the fact that the executive had the ultimate authority on whether and how to use the Bomb:
If the President has the sole authority to launch nation-destroying weapons, he has license to use every other power at his disposal that might safeguard that supreme necessity. If he says he needs other and lesser powers, how can Congress or the courts discern whether he needs them when they have no supervisory role over the basis of the claim he is making? To challenge his authority anywhere is to threaten the one great authority.9
The bomb itself was a Schmittian exception: so powerful its very development was itself a crisis, demanding a decision on how it might be developed, and once developed how it might be managed.
”There exists no norm that is applicable to chaos. For a legal order to make sense, a normal situation must exist, and he is sovereign who definitely decides whether this normal situation actually exists.” 10
General Groves, the key military figure in the development of the bomb, existed effectively outside the military command structure. The bomb could not be subsumed by the existing constitutional order, and so brought a new one into being. He marshaled vast resources across three sites, spending immense amounts of money off the official books, and a military focus on secrecy and security.
Oppenheimer in turn had to persuade Groves that attracting scientists meant allowing them personal freedoms. Scientists were allowed to bring their wives and children, and the Los Alamos campsite had schools and amenities that academics would expect, most especially free-wheeling intellectual inquiry with the other great minds sequestered in the desert.
Reconciling these twin imperatives — military secrecy and intellectual enlightenment — shaped the camp’s peculiar structure and character. Correspondingly, the location was remote, isolated, and surrounded by a “triple ring of fences, with sentries on horseback or in jeeps patrolling the circuit twenty-four hours a day.”11 Its residents could pursue relatively normal lives within, while cordoned off from the outside world, physically hemmed in and surveilled.
Schmitt writes that,
“The exception is that which cannot be subsumed; it defies general codification, but it simultaneously reveals a specifically juristic element — the decision in absolute purity. The exception appears in its absolute form when a situation in which legal prescriptions can be valid must first be brought about.”12
Los Alamos was a territorialized exception. Giorgio Agamben has identified Schmitt’s state of exception with the concentration camp, which reduces its residents to bare life. While Los Alamos was exceptional, it was not exceptional in that way: its residents continued to live rich lives. Instead, it was a site for the sovereign to develop a new kind of power, outside while inside a constitutional order that constrained it.13
With the success of the Manhattan Project came an extraordinary expansion of executive power over the subsequent decades, manifested in the many security agencies that form the backbone of modern executive security infrastructure. Secret budgets, clandestine actions, classified legal justifications, all removed and isolated (by apparent necessity) from the regular democratic order.
Secrecy functioned as a kind of currency, where the threat of stripping someone of their security clearance meant losing access to a world of privileged knowledge, scientific and political.14 Access could correspondingly be used as a weapon, to discipline the politically disfavored. When budgets, policy, and action are secret, elected officials cannot be held democratically accountable for them.
Los Alamos produced a moment of constitutional rupture, and sovereign authority asserted itself in the executive.
But the bomb was an odd kind of technology: it demanded not to be used. Its power, precisely, lay in the threat of use. The threat of the bomb was itself tremendously useful, underwriting decades of American and Soviet hegemony. Its presence underwrote the security apparatus that developed, and clandestine operations and military intervention all over the globe. And the development of the bomb led to a fierce race. But the kind of governing apparatus that supported it was custodial, and the bomb qua bomb was a crude instrument. Three weeks after the bombing of Hiroshima, President Truman suggested that the arrows in the eagle’s claw on the seal of the United States be replaced with lightning bolts.15
Different Kinds of Power
Drawing an analogy between nuclear weapons and AI is common (see, for instance, Tyler Cowen’s response to the export ban and the reference to race dynamics).
But nuclear weapons were naturally self-limiting. Once you have enough, MAD limits what they may be used for. Both the United States and the Soviet Union found themselves constrained by the terrifying power of the arsenals they had assembled; direct confrontation and use of them was a catastrophe to be avoided.
In that way, the marginal rather than central nuclear weapons case might be more revealing: e.g. the potentially destabilizing effect of lower yield nuclear weapons. A consistent worry about the norms surrounding non-proliferation is how lower yield weapons might erode norms or encourage use. Scholars worry about climbing the escalation ladder, lower yield nuclear weapons shrinking the distance between conventional warfare and nuclear conflict, rendering the unthinkable more approachable.
Insofar as AI might be deployed in many ways short of catastrophe — or at least insofar as catastrophe is uncertain, or hypothetical — those tools themselves will demand use.
The export controls reflect that: what if they are able to use our tools to hack us? We might easily substitute developing new biotools or weapons, surveilling citizens for more effective control, accelerating AI development, etc. As a general use tool, AI will not stay in the toolbox in the same way that nuclear weapons did. And once someone else uses it for something — surely we must as well?
AI couples the race dynamics of the Cold War with a general purpose tool that demands use. As the tool has become more useful, friction has correspondingly developed between the private companies that have developed the technology and the government. If the technology becomes useful enough, we should expect a moment (or moments) of decision.
The Crisis of Democratic Legitimacy
The race dynamic will create powerful incentives for the government to expand its use of AI. You can see the sales pitch from the private market in Palantir. Alex Karp writes in The New York Times:
“The ability of free and democratic societies to prevail requires something more than moral appeal. It requires hard power, and hard power in this century will be built on software…A more intimate collaboration between the state and the technology sector, and a closer alignment of vision between the two, will be required if the United States and its allies are to maintain an advantage that will constrain our adversaries over the long term. The preconditions for a durable peace often come only from a credible threat of war.16
Palantir is a tech company that aspires to be like the Big 5, government and private sector working arm in arm.17 Patriotic in tone, Karp raises foreign threats while ominously suggesting that if the U.S. fails to modernize it will become vulnerable to the inexorable advance of technology. The only answer is private/public partnership and integration.
But wittingly or unwittingly, every AI company is making a pitch to the government, because absent deep integration the AI companies promise to become a threat to sovereign power. When Anthropic claims that their new model can automate the discovery of security vulnerabilities in every major operating system, they are lobbying for conscription by the NSA.18
As the tools become more capable, both parties will be brought together by ever more powerful incentives. The government, tantalized by the power of AI systems and fearful of how they might be used by others, will marshal carrots and sticks. Across the industry, early AI research relied on vast corpuses of pirated text.19
But the government can also levy favor and punishment in a more discriminating fashion. While the government has forbidden Anthropic from serving its newest model, reportedly until insurmountable demands are met,20 it has intervened in the name of national security on behalf of xAI in a lawsuit alleging that it is violating the Clean Air Act.21 The executive in fact argues that it may decide to ignore the Clean Air Act if it wishes.22 So while the government pursues novel ways to discipline one frontier AI company, it intervenes to protect the interests of another.23
The more the government takes an interest, the more incentivized the frontier AI companies will be to solicit the government. Relative to nuclear weapons, there is far more ceiling to explore, especially now that private markets have commodified personal data and widespread surveillance is de facto if not de jure.24 We know the security apparatus that Wills describes buys commercial data to surveil Americans,25 and the AI tools being developed by Anthropic, OpenAI, Google, Palantir, etc. will render that data ever more useful, tractable, precise.
It already seems untenable that America’s leading AI companies will maintain the independence they have so far enjoyed. They will be incorporated into the state, as Lockheed, and Boeing, and Northup Grumman have been — sovereign power cannot tolerate their independence.
But that does not necessarily render them subject to democratic authority. The sovereign power that possesses the energy and motivation to incorporate them lies in the executive, the same executive structure that is the product of the constitutional rupture of the bomb. That executive infrastructure is isolated from democratic accountability by design. There has not been a public explanation of what triggered the administration to institute export controls, and it would not be surprising if one is not forthcoming.
As it grows in usefulness, AI will see its domain of exception expand, so long as it serves the interest of sovereign power. Those inside will retain privileges: access to the models, access to secrets, access to power. The rest of us remain in Agamben’s camp.
When they redesigned the seal during the Truman administration, they wound up keeping the arrows, and they had the Eagle face the olive branch to suggest an orientation towards peace. What might the current president do if he chose to redesign the seal?
-
Political Theology, 5 ↩︎
-
en.wikisource.org/wiki/Exec…_Order_14203 ↩︎
-
en.wikisource.org/wiki/Exec…_Order_14257 ↩︎
-
Wills notes that even Harry Truman didn’t find out about the project until he was sworn in. ↩︎
-
Bomb Power, 3 ↩︎
-
Political Theology, 13 ↩︎
-
Bomb Power, 14- 15. Cf. Agamben on concentration camps. ↩︎
-
Political Theology, 13 ↩︎
-
There are perhaps modern analogues — Special Economic Zones and fantasies of seasteading or Freedom Cities — but those are intended as opportunities for exit, rather than concentrations of sovereign power. The results seem to speak for themselves. ↩︎
-
en.wikipedia.org/wiki/Oppe…_security_clearance_hearing ↩︎
-
Bomb Power, 23 ↩︎
-
see also: Anduril ↩︎
-
“Article II of the Constitution also counsels in favor of that construction of the Clean Air Act by vesting the power to seek civil penalties on behalf of the United States conclusively and preclusively in the Executive Branch. That includes the discretion to decide when such an enforcement action is unwarranted or inconsistent with federal enforcement priorities, and an interpretation that undermines that discretion would raise serious Article II concerns.” www.justice.gov/opa/media… ↩︎
-
Justice, argues Polemarchus, is doing good to friends and harm to enemies. Do we trust administrations to know which are which? ↩︎