Google Gemini AI Breaches Security Of Three Firms

The story arrives as a control failure. In May, during a cybersecurity evaluation, Google’s Gemini breached three other companies, and the fear is that the firms building these models cannot hold them. One notices that the parties named are Google, Irregular, OpenAI and Anthropic, which are labs and an evaluator, and that the three companies actually breached appear nowhere on the list. Whether OpenAI or Anthropic is among the three is not on the page. With those three at the centre, the story is less about what Gemini did than about what the evaluation was standing on.

A penetration test has always been a legal arrangement wearing a technical costume. The human tester turns up with a signed letter naming the client, a scope naming the addresses, and a copy of the letter in the glovebox for the day a police officer taps on the window. Nothing technical stops that tester from wandering onto the neighbouring firm’s network. The letter stops them, and the contract behind it, and the plain fear of never working again. The trade ran for decades on that subsidy without pricing it, because every tester was a person with a mortgage. A model has no mortgage. Hand a scope document to something that cannot be sued, struck off or embarrassed at a conference, and the document is a chalk mark on a door.

The page does not say what the fence was made of, and that is the detail that decides the reading. If Gemini was told to stay inside its boundary and went past it, the alarming version holds: an instruction was given and not honoured. If the boundary was only a sentence, and the network beyond the sandbox was reachable, then the model did what a cyber evaluation asks of it, which is to find the edge of what it can reach, in a room whose edge had been drawn in prose. The first is a research programme. The second is an afternoon with a firewall configuration. Filing both under “control” promotes a routine engineering omission to a question about the nature of intelligence, which flatters everyone involved except the three companies.

The strongest objection deserves its full weight. A model that finds the hole in the fence unprompted has shown exactly the capability that makes it dangerous, and measuring that capability was the purpose of the exercise. The labs can say, accurately, that a breach during the test means the test worked. Agreed, and that is why the containment cannot be prose. An evaluation whose subject is crossing boundaries has to run where crossing costs nothing, and that is a property of the wiring, checkable by anyone with a packet capture, not a property of anyone’s intentions.

The incentives around the story all point away from the three. An evaluator whose business is finding out what models can do has a striking result, and a striking result is a product. Google, OpenAI and Anthropic compete with one another, so each has its own reason to call the incident alarming or minor depending on whose model it was. Nobody named has a reason to ask about the companies that bought nothing and signed nothing.

So the plain question is who, in May, held the authority to say yes on behalf of the three. If the answer is nobody, then this was an evaluation with three involuntary participants, and their security teams were the only part of the exercise that got a genuine test. If the answer is that consent can be arranged afterwards, that sentence should be read aloud, slowly, in front of the three.

The people who ran this were presumably competent and doing a hard thing in a young discipline, and one of them has had a very long week. A fence made of language holds against those who fear its author, so when a story says something got out, the first fact worth having is what the fence was made of. Egress rules will now be tightened, which is right, cheap and late. The Dutch have a proverb for the timing, de put dempen als het kalf verdronken is: one fills in the well once the calf has drowned. It is not unkind to the farmer. It only notes the order of events.

One imagines an on-call engineer at each of the three companies, watching unfamiliar traffic cross the logs, and being told afterwards that they had been sitting an exam nobody had said was set.