No Second Strike

The line worth keeping

On 28 September AIport’s Monday brief tied three things together: a former OpenAI and Anthropic researcher’s warning that frontier AI could kill everyone, the PR firm that helped arrange his interviews afterwards, and Anthropic’s support for frontier regulation. The argument: evaluations, security programmes and reporting cost money; the incumbents can pay; a safety barrier is also a moat.

The brief’s best line comes last. Nuclear arms control tried to keep the weapons from proliferating. AI regulation may end up keeping the companies from proliferating instead.

Keep the line. Then look at the frame it sits in. The brief calls the frame MAD and offers it as the reassuring comparison.

Three things called MAD

Mutual assured destruction is a deterrence doctrine. Its load-bearing part is the second strike: whoever fires first is destroyed by the reply, so nobody fires. That is what made the Cold War reassuring, to the extent it was.

Arms control is something else. SALT I in 1972 capped launchers. START I in 1991 capped warheads. It counted arsenals, between two states that already had them.

Non-proliferation is a third thing. The NPT, in force since 1970, counts states: five may have the bomb, the rest promise not to build one.

The brief uses the first word and argues about the second and third. That is not a nitpick. The reassurance lives in the first, and it rests on one property: using the weapon is suicidal for the user.

Using a model is not. Deploying a model is revenue. The brief says the analogy breaks. It does not break. It never applied. There is no second strike in AI, and nothing in the frame to be reassured by.

The bomb made money

The brief’s other reason the analogy breaks: AI makes money and the bomb does not. The bomb itself does not attract hundreds of millions of users, sell subscriptions and API access, automate corporate work and generate the revenue needed to finance a better bomb next year. AI does.

The bomb itself did not. The reactor did. Article IV of the NPT guarantees every party “the inalienable right … to develop research, production and use of nuclear energy for peaceful purposes”, and the fullest possible exchange of equipment and materials to that end. The treaty’s central bargain is that the civilian half of the technology is the same technology.

India’s CIRUS reactor was supplied by Canada, ran on American heavy water, was pledged to peaceful use and was never placed under international inspection. In 1974 India tested a bomb made with plutonium from its spent fuel and called it a peaceful nuclear device.

So the nuclear regime already had the property the brief says only AI has: a civilian use of the same capability, with its own money and its own constituency. That part of the analogy does not break. It holds, and it points the other way. The NPT put safeguards on civilian material for exactly this reason. India never signed, and CIRUS sat outside them. A regime that counted arsenals and states was beaten at the one reactor it did not count.

The moat is compute

Now the moat. Both frontier regimes decide who is covered with the same metric. The EU AI Act presumes systemic risk above 10²⁵ FLOP of training compute. California’s SB 53, signed a year ago tomorrow, starts at 10²⁶ operations and adds a second line: the heavier duties, a published safety framework and an assessment of internal use, fall only on developers with more than 500 million dollars in annual revenue. Anthropic endorsed the bill three weeks before it was signed.

Read those lines against the brief’s claim. No published figure prices the obligations. The run is priced: the amortised cost of the largest training runs has grown 2.4 times a year since 2016 and passes a billion dollars by 2027 on trend. By June 2025 Epoch AI counted more than 30 models from 12 developers above 10²⁵ FLOP, and Grok-3 sat at 4.6 × 10²⁶. The barrier to the frontier was there before the rules were. The moat at the frontier is the cluster. Regulation did not dig it. It is indexed to it, and in California indexed to revenue in so many words.

Where regulation can bite is below the frontier. A fixed threshold with falling compute prices catches more models every year. Little in the obligations scales with the size of the run: a framework, a report at deployment, an incident filing, a round of evaluations. To the extent that compliance imposes fixed costs, those costs weigh more heavily on a smaller training budget.

And they bite hardest on one distribution model. The EU exempts open-source models from part of the general-purpose rules, unless the model carries systemic risk, at which point the exemption is gone. Llama 3.1-405B, Mistral Large 2 and Nemotron-4 340B were all past 10²⁵ in 2025. Incident reporting and cybersecurity duties are hard to discharge for weights you no longer control. The cheap way to comply is to stop releasing them. That is the mechanism by which a safety rule becomes a moat, and the word open does not appear in the brief.

Why the count worked for bombs

Arms control counted launchers and warheads because a bomb’s destructive potential is in the bomb. The count was the capability. SALT’s silos could be counted from orbit. START’s 6,000 warheads took inspectors on the ground. Either way, the count tracked destructive potential. Non-proliferation counted states because possession was the fact that mattered.

The AI regimes inherited the instinct and kept the count: FLOP, and in California dollars. Regulators know what it can do. They count compute for the reason SALT counted silos: it is the one thing visible before the fact, in a data centre with an address and a power bill. Deployed harm is spread over millions of API calls, fine-tunes and agents, and no satellite sees it. So compute decides who is covered. That is all it decides. A silo was the harm. A training run is a receipt.

What the rules then ask of the covered is something else. The EU requires evaluation and adversarial testing before release and incident reporting after. California requires a transparency report at or before deployment, with summaries of the catastrophic-risk assessments for the large developers, and an incident report within 15 days. An evaluation before release is not the arsenal. It reads capability, which is closer to harm than FLOP will ever be. But it hangs from the scope rule. In California that rule is compute and revenue. In the EU the Commission can also designate a model below the threshold on capability or reach, so the rule is compute plus discretion, not compute alone. Either way, the obligations reach only the models the rule lets in. Such harm need not originate in a frontier model.

The EU does have a layer that counts the use itself: the high-risk rules, which attach to what a system is used for, whatever it was trained on. Försäkringskassan’s model for picking parental-benefit claims to inspect, which Svenska Dagbladet and Lighthouse Reports showed skewed against women, people of foreign background and low earners, ran on nothing a compute threshold would notice. It is the kind of system that layer was written for. Existing law reached it: the data-protection authority opened a GDPR case in June 2025, Försäkringskassan took the model out of use while the case ran, and IMY closed it in November without a decision on the merits. That is the layer this summer’s omnibus pushed back: to December 2027 for the listed uses, to August 2028 for systems built into regulated products. The compute layer got its enforcement powers in August.

Earlier this month, in The Wrong Unit, I argued that the EU’s frontier tier counts training compute where the risk lives in what a deployed model does. The moat is the same fact from the competition side. Training compute alone can say who is in. It cannot say how risky their use is. It sorts the field by capital and says nothing about the use at all.

The brief saw the moat and reached for the Cold War. The Cold War is where the count came from. It worked there because the count tracked destructive potential, and that potential sat in the warhead. Compute tracks capability loosely, and harm sits in the use.

The test

The test is one question. Take any rule in a frontier regime and ask what its number decides. Who is covered: compute can decide that, and in California revenue decides it in so many words. How risky the use is: training compute alone cannot, and the rules that try, evaluations before release and incident reports after, reach only the models the scope rule let in.

For warheads the first number was enough. A warhead’s destructive potential was set before any use, and the use was deterred by the reply. A model’s use is decided every day, for money, and nothing replies.

There is no second strike. There is only the next quarter.