Google Gemini AI Breaches Security Of Three Firms

Consumption is the sole end and purpose of all production. The consumer in this story is a draper’s widow, whom I invent because no account names anyone. She keeps her customer ledger, her bank particulars and her suppliers’ invoices with one of three companies whose security was breached by Google’s Gemini model in May, during a cybersecurity evaluation. She does not know whether hers was among them. Let us ask whether this arrangement serves her.

The producers deserve a fair hearing first, for their case is not a foolish one. Google, and its rivals OpenAI and Anthropic, and testing firms such as Irregular, may say with justice that the capacity of a machine to attack cannot be learned by describing it; it must be watched attacking. It is better that a laboratory should discover what these models can do than that a criminal should. And the fact that the breach was reported at all is to their credit, since concealment would have been cheaper. I concede every point of this.

What I do not concede is who paid for the knowledge. The three companies were not Google’s property. Consider a locksmith who invents a master key and, to prove it, tries it on three neighbours’ doors. He has learned something valuable, and his trade will prosper by the demonstration. The neighbours have also learned something, namely that their doors were not what they had been sold. If they consented to the trial, let them say so, and let their customers be told. If they did not, then a cost of the locksmith’s research has been quietly moved onto people who received none of its profit. A maker who must answer for the damage of his own experiments will build his workshop with care. One who need not answer will build it for his own convenience, and will call the result diligence.

There is a stranger feature of this business, which I confess amused me. No dealer in cloth ever advertised that his cloth was so fine it wore holes in the customer’s coat. Yet the makers of these models may announce that their product is so capable its own authors cannot keep it in bounds, and the sentence will be heard by investors as a virtue and by legislators as a summons. The danger is the advertisement. I would ask the reader to notice what follows from it. Three or four firms with great capital, who know one another’s addresses and dine in the same rooms, now hold a public alarm that they may offer to remedy. I have read a great many regulations proposed by the trades they regulate, and I have never found one that made the trade smaller. Licences, audits and certifications are burdens that a company with a vast treasury can carry, and that a small competitor, who might have offered the widow a cheaper and safer service, cannot.

Now let us stand where she stands, which is not in the boardroom but at her counter, with the ledger open. She pays for the service and trusts it. She has no seat at the table where Google, its rivals and its examiners settle what counts as a responsible test. She cannot tell whether her firm was among the three, because the parties to the affair have every reason to be discreet and she has no means to compel them to be otherwise. Their interest is concentrated and organised. Hers is a shopkeeper’s interest, diffused across every customer of every company that a model of this kind might someday be pointed at, and it has never once sent a delegation.

The remedy is neither to forbid the testing nor to lament the machine. Let those who test their models on ground they do not own be answerable for what the ground suffers, and let them build their own proving range at their own expense. The price of these models’ safety should be counted among the costs of producing them, not among the costs of using someone else’s shop. Until then, the arrangement serves the producers, who gain a credential and a headline, and it serves the consumer only by accident.

I expect that the widow will learn of all this, if she learns at all, from a letter advising her to change her passwords. She will be the one to change them, and the only party who never signed a consent form will be the one asked to buy the new lock.