Microsoft Copilot rarely copies full sentences, firm claims

We are told that Copilot’s habits have been measured and found modest - that in its legal filings Microsoft states the chatbot rarely reproduces full sentences from news articles and books. This is presented to us as a fact about a machine’s design, a settled property, like the length of a finch’s beak. But consider the population from which that single sentence was drawn: the uncountable outputs Copilot might produce for any query, ranging from wholly novel recombination to exact verbatim recall, and the environment in which one description of that population - “rarely” - survived to be filed with a court while a great many other possible descriptions did not. What we are examining is not a fact about a chatbot. It is a fact about which characterization of the chatbot proved fit for a courtroom.

The underlying mechanism is genuinely evolutionary, and I mean that in the narrow sense, not the loose one. A large language model is trained by an accumulation of minute adjustments - gradient descent, its makers call it, and the name is more apt than they perhaps intend, for it is variation and selection by another vocabulary. Each step nudges the model’s weights fractionally toward outputs favored by the training corpus. A sentence that appears once in a corpus of billions leaves almost no trace. A sentence that appears many times - the opening line of a famous review, a passage quoted and requoted across the internet until it is less a sentence than a fossil bed - leaves a deep one. The model does not choose to memorize The New York Times; it is shaped, generation upon generation of gradient step, by whichever passages happened to be repeated often enough to imprint. Verbatim reproduction, where it occurs, is not an act of theft in the way a scribe copying a manuscript is theft. It is a cast left in sediment by a phrase that recurred often enough to leave one.

Now set two selective environments side by side. Microsoft’s lawyers, writing filings, are selecting among possible descriptions of Copilot’s behavior under a pressure that favors the phrasing most useful to their client - “rarely,” calculated across the vast denominator of all queries ever run. The New York Times and the book authors bringing suit are selecting under the opposite pressure, favoring the discovery and display of the most damaging instances of exact recall, the fossils clearest and most legible, because those are the ones that survive into an exhibit. Both parties are naturalists of a sort, digging in the same stratum, and both are honest about what they find; what differs is which specimens they think worth carrying back to the tent. Neither “rarely” nor “substitutes for the original” is a lie. Each is a survivorship account, compressed from a population too large for any courtroom to examine whole.

I anticipate the objection, and it is a fair one: this is not blind variation under selection, as the finch’s beak is, but two teams of advocates deliberately choosing words to win a case. Quite so - the drafting of a legal brief is intelligent design if anything is. But the phenomenon the brief is arguing about, which sentences a trained model happens to retain and reproduce, is not intelligently designed at all. No engineer at Microsoft sat down and decided the model should be able to recite a particular paragraph of a particular book. That capacity accumulated, unplanned, out of the statistical pressure of repetition in training data, exactly as a wing accumulates out of pressures that never once aimed at flight. The lawsuit is a dispute over how to narrate a design that was never designed.

Picture, then, not a machine caught in an act of theft, but a court clerk holding up a single printed page on which a model has reproduced, word for word, three sentences from a novel - a specimen pulled from a strata of billions of training tokens, presented now as evidence of intent where none, in the engineering sense, existed. The judge must decide what the resemblance means. Natural history can only tell her that the resemblance is real, and that it proves nothing at all about what was meant.