Uncontrollable AI threatens global disaster

The story frames the warning itself as an act of civic responsibility - a journalist, some unnamed experts, an old friend from Stanford, all passing along the news that the machine may soon outrun its makers. But look at what is actually being fenced here: not yet the weights, not yet the training runs, but the evidence itself. The claim of imminent recursive self-improvement, the timeline of a “couple of years,” the comparison to Hiroshima - all of this travels to us through a closed channel of the credentialed, relayed friend to friend, expert to journalist, never once through anything so vulgar as a published dataset or a reproducible benchmark. That is the commons at stake: the shared, verifiable knowledge of what these systems can actually do, which is being quietly enclosed inside a conversation among people who already sit near the controls.

Consider who is speaking. Timothy Garton Ash is a serious observer of power, not a fool, and I do not doubt his sincerity. But notice the structure of his sourcing: “experts in Silicon Valley,” unnamed, and “a Stanford University friend,” equally unnamed. This is not testimony that the rest of us can examine, weigh, or challenge. It is warmth passed along a private wire - precisely the mechanism by which a commons of understanding, built for two centuries by open science on the premise that a claim is worth nothing until it can be checked by strangers, gets replaced by something closer to prophecy delivered by people you happen to know. Mutual aid in science has always meant this: publish the method, share the data, let the argument survive contact with people who owe you nothing. The alarm about AI, as it is being sounded here, skips that step entirely and asks for trust instead.

Now watch what the alarm is used to justify. Every telling of an “uncontrollable” technology arrives with the same appended moral, stated or not: therefore it must be kept in fewer hands, watched by fewer eyes, developed behind fewer walls. This is the argument from concentration, and I have heard its cousins before - the enclosure of common land was justified by the claim that peasants farming it in common would exhaust it, when in fact commons husbanded by their users for generations were destroyed only once fenced and turned to private profit. The Hiroshima comparison performs the same labor here that the language of exhaustion performed on the English common: it does not describe a mechanism, it manufactures inevitability, and it does so in service of exactly the institutions - the Silicon Valley laboratories holding the largest models, the largest compute, the largest confidential safety evaluations - who stand to gain most from being trusted as the sole responsible custodians of a danger the rest of us are asked to take on faith.

Here is the strongest version of the case against me, and it deserves a real answer rather than a shrug. If a system truly is capable of self-improving past human oversight within a couple of years, then broad publication of the relevant research is not mutual aid, it is an accelerant - you do not open-source the uranium enrichment method because openness has served physics well in the past. I take this seriously. But notice that the argument, if true, cuts against concentration too: a handful of firms racing each other in the same city, funded by the same pressure to ship before a rival ships, is not a controlled containment, it is the precise social arrangement most likely to produce a reckless breakthrough, because each lab’s fear that the others will get there first is what drives the timeline Garton Ash’s friend is warning about. The danger these experts describe is not an argument for enclosure; it is a symptom of enclosure already accomplished - a research commons that used to include universities, open publication, and slow peer contest, now narrowed to a few firms whose competitive secrecy from one another is the very engine of the risk they are photographed wringing their hands about.

What would mutual aid actually look like here, rather than the theater of privileged alarm. It would look like the safety evaluations these Silicon Valley experts are performing being published in full, with methods a rival lab’s engineers or a graduate student in Nairobi could reproduce and contest, rather than summarized secondhand at a dinner party and passed to a columnist. It would look like the compute and model access needed to test these self-improvement claims independently being treated as a shared resource rather than a competitive moat. Instead we are offered a closed circle of the alarmed, warning us gravely of a fire whose exact size and location they decline to show us, while asking that the door to the room stay locked and the key remain, as it happens, in their own coat pocket. The fire may well be real. I have simply noticed that every account of it arrives from the people already warming their hands.