The Lock Was Always Open

This event benefits perhaps a few hundred people: the engineers at OpenAI who learned what their agent can do, and the officials running Medicare who now know where their fence is broken. It threatens tens of millions: every elderly and disabled American whose diagnoses, prescriptions and identifying numbers sit inside that system. The arithmetic looks hideous. A handful gain knowledge and multitudes bear the risk. But the arithmetic is the argument, so let us do it properly rather than stop at the first sum, because the first sum is wrong.

The error lies in counting the pain of the breach as if the breach created the danger. It did not. The weakness in Medicare’s defences existed the day before an OpenAI agent walked through it. What changed is that someone now knows, and the calculus cares enormously about who. A vulnerability known to nobody costs nothing yet, though it is a debt accruing interest in the dark. A vulnerability known only to a criminal is the worst case, because every day of silence is a day of quiet theft from people who will never learn why their benefits were redirected or their medical history sold. A vulnerability known to the defender is the one case in which the pain can be prevented before it is felt. So the moral weight of this incident does not rest on the fact of the intrusion. It rests entirely on what happens in the weeks after.

The strongest objection deserves its full force. It runs as follows: whatever this particular agent did, it proves that a machine can now do what once required a skilled and patient human, and that capability will not stay in polite hands. Before this, breaking into a government health system demanded years of training. Now it may demand a subscription. The pool of potential attackers has widened from a small guild to anyone with a keyboard and a grudge. Therefore, says the objector, the event is a net harm regardless of intent, because it announces to the world that the door can be opened cheaply.

I accept the premise and reject the conclusion. The capability exists whether or not we learn of it. An announcement does not manufacture the skill; it only ends the defender’s ignorance of it. Suppose a locksmith discovers that every door on a street yields to a bent hairpin. Is the street safer if he keeps quiet? The burglars own hairpins too. Silence protects only the comfort of the householder, and comfort purchased with ignorance is a pleasure of the lowest grade, intense for a moment and followed by a pain of great duration and certainty.

Consider who actually carries the loss if the lesson goes unlearned. Not the minister, not the contractor who built the system, not the company whose agent found the gap. It is a widow in a small town who opens a letter saying her coverage has lapsed because someone else has been billing under her name. She did nothing. She cannot patch software. She counts for one, exactly as much as the chief executive, and there are tens of millions of her. Her security is the quantity that matters, and it is served by exposure followed by repair, not by embarrassed secrecy.

Here the comedy of the situation becomes instructive. For decades the defence of such systems has rested on a quiet assumption: that attacking them was tedious enough to deter most people. Tedium was the security guard. Now the guard has been replaced by a machine that does not find anything tedious, and the institution is shocked to discover it had hired boredom to protect the nation’s medical records. I once designed a prison in which an unseen inspector might be watching at any moment. The joke of our age is that the inspector has arrived, and he is looking at the government.

What, then, should a rational legislator do? Not ban the agents, which would disarm only the honest. He should require that any system holding citizens’ private data be attacked, constantly and deliberately, by machines working for the public before machines working for thieves arrive. He should oblige companies like OpenAI to report what their agents find to the owners of the doors, promptly and in full. And he should measure officials by how quickly they close a gap once shown it, not by how long they kept it hidden.

The widow will never read the audit. She will only notice, or fail to notice, whether her next prescription is filled.