Trump Dismisses AI Safety Concerns as Hoax

The dismissal is described as plain speaking, a refusal to be frightened by men who profit from fright. The mechanism it actually creates is different: it removes, at a stroke, the political cost of building systems whose failure modes nobody yet understands, and it does so at the exact moment when the men building those systems are asking, with unusual candour, for someone to install a brake. Donald Trump calls the safety concern a hoax. Jack Clark, who co-founded Anthropic and therefore has some claim to know what he is building, asks for a mandatory kill switch. The gap between those two positions is not a disagreement about facts so much as a disagreement about who should bear the cost of finding out.

Here the merchant suspicion earns its keep, though not in the direction the casual reader expects. It is tempting to say that Clark, as a merchant in artificial intelligence, is conspiring to raise barriers against smaller rivals under the cover of safety - regulation as moat, the oldest trick in the ledger. That reading deserves a hearing; safety mandates written by incumbents have a wonderful tendency to be affordable only by incumbents. But the suspicion must be applied evenly, and evenly applied it falls just as heavily on the other side of the argument. A president who declares the hazard a hoax is not a disinterested arbiter either; he is removing a cost from an industry that has become, in the space of a few years, politically indispensable and enormously well-funded, and the removal of that cost is itself a favour, dispensed to men who did not have to conspire for it because the position was simply given to them. The merchant does not always have to ask. Sometimes the hoax is announced on his behalf.

The competition check clarifies what is at stake. A genuine market test of AI safety would require rival firms to compete on the credibility of their safeguards, the way firms once competed on the reliability of their engines or the purity of their goods, with reputation and liability doing the work that regulation might otherwise do. But reputation disciplines only where consequences are visible and attributable, and the peculiar feature of a system with no kill switch is that its failures may arrive too diffusely, or too late, for reputation to catch up. Remove the safeguard and you have not created a freer market; you have created a market with no working feedback loop, which is not competition, it is a wager placed by everyone on behalf of no one in particular.

The human cost is not abstract, though it is easy to render abstract, and here I ask the reader to picture not a boardroom but a call centre, where a worker at this hour is being told her queries will now be triaged by a system whose designers cannot fully specify its behaviour and whose off-switch, per one of its own architects, does not reliably exist. She was not consulted on the kill switch. She will not be consulted on the hoax, either. Her labour has been divided so finely, and her employer’s dependence on new machinery has grown so total, that the debate over whether the danger is real is being conducted several floors above her, by men who will not be in the building if the answer turns out to be yes.

The sympathetic observation, then, is this: a labourer, a president, and an engineer are each behaving exactly as their position would predict, and the mechanism produced by their combined behaviour serves none of them as an outcome, only as a wager. The president gets to look unafraid. The engineer gets to look responsible for having asked. The worker gets the machine either way, switch or no switch, and finds out what it does by living downstream of it, the way the miller’s family always found out first whether the new wheel would turn the grain or grind the miller’s hand.