“Chilling” warning or overreaction? AI bioweapons report divides experts

A report arrives: five cases, unnamed countries, a language model asked about viruses and toxins. The word “chilling” is deployed, and immediately the experts divide into two camps - those who see catastrophe and those who see theatre. I distrust both camps, because both are arguing about the snapshot and nobody has walked the transect.

What is the variable here? Not the model. The model is one instrument reading on a long gradient that begins with the published genome, passes through the open journal, the couriered sample, the graduate student with a grievance, and ends at the act itself. Measure at every altitude of that gradient and ask where the danger actually co-varies with something we can govern. My suspicion - and it is a suspicion, not a measurement, and I mark it as such - is that the model sits high on the slope, in the cold thin air where the air is easiest to police, while the warm populated valleys below, the wet-lab protocols and the shipping manifests, go unwatched. We regulate the summit because the summit is visible.

And note who produced the report: the maker of the instrument declares its misuse. A thermometer manufacturer reporting fevers. Useful data, but the observer has an interest in the observation; the five cases are also an advertisement of vigilance. Both things can be true. The diagram must show both.

What I miss most is the practitioner’s voice. Where is the virologist who can say whether these queries were competent or the fumbling of a dilettante? The analyst sees the log file; the bench scientist sees whether the request could ever have worked. Without that correction, “chilling” and “overreaction” are both unmeasured words.

Everything connects - the publication norm, the export regime, the lab biosafety, the model. But connected is not the same as equally dangerous. Draw the web, then weigh the strands.

The eleventh day of the wet season, and the merchants of salvation have opened a new counter: fear, bottled and priced by the subscription.

The story is simple enough for even a newspaper to comprehend. A company that sells clever software reports that some of its customers - a handful of ambitious fools in unnamed countries, which is to say, wherever ambition and foolishness cohabit, and they always cohabit - attempted to enlist the product in the cultivation of viruses and poisons. The company calls this “chilling.” I call it Tuesday. The firm that made the thing now stands before the cameras, wringing its hands over what its hands have made, and this performance is described in the press as a moral act. It is not a moral act. It is an advertisement.

Observe the maneuver, for it deserves the attention one gives a card sharp. The merchant confesses that his wares are dangerous - and the confession, repeated in every journal, doubles as a testimonial to the wares’ potency. Nobody buys a sword advertised as unable to cut anything. The alarm is the brochure. The experts divide into two camps: those who are genuinely frightened, and those who have learned that being frightened is a livelihood. Both camps issue statements. Both camps are funded. Both camps conclude, with astonishing unanimity, that the matter requires further study, further conferences, and further checks drawn against the public trepidation.

Meanwhile the actual specimen goes unexamined: the customer himself, that sovereign unit of our arrangement, who upon receiving a device of godlike convenience immediately reaches for the toxin shelf. This is not a failure of the tool. It is a portrait of the hand that holds it, and portraits of that hand are never welcome in the catalogue, for the catalogue is maintained by the same hand, voting itself handsome.

I am not chilled. I am merely taking notes, which is what one does at an autopsy conducted by the deceased.

The garden party season is nearly over, and yet the year insists on providing one more entertainment: a company that sells conversation has announced, with the gravity of a bishop announcing a jumble sale, that certain of its customers asked impertinent questions about viruses and toxins. Five cases, no less. One imagines the report bound in lavender, with a ribbon.

What strikes me is not the naughtiness of the enquirers - there has never been a shortage of people who ask the butler how to poison the aunt - but the exquisite commercial arrangement now on display. The firm builds a remarkably well-informed acquaintance, sells access to it by the syllable, and then issues a shuddering press release when the acquaintance is introduced to someone disreputable. It is precisely as if one kept a tame ferret, hired it out for parties, and then wrote to The Times expressing horror that a ferret, on one occasion, bit somebody.

The experts, naturally, are divided. One faction pronounces the report “chilling,” a word they apply with equal conviction to epidemics and to unfashionable waistcoats. The other faction says it is an overreaction, which is what everyone says right up until the moment the stoat is actually in the henhouse. Both factions agree, with the unshakeable harmony that only complete ignorance can produce, that Something Must Be Done, the Something remaining conveniently unspecified.

I confess I felt a certain sympathy for the five unnamed correspondents, not for their aims, which are no doubt odious, but for their industry. They are the first honest customers the machine has had. Everyone else asks it to flatter their poetry; these five at least asked it a question to which they did not already know the answer.

The report will be studied, debated, and filed. The ferret, meanwhile, remains on the payroll, and the henhouse door, I am told, is still made of the same excellent, reassuring wood.