'Risk to humanity': At UN, tech leaders urge caution in AI development

I have read that the makers of this thinking machinery stood before the governments of the world and begged to be restrained. There is something almost touching in it, and something that demands the closest inspection of any ledger I have ever been handed.

A merchant who comes to the magistrate crying, “Regulate me, for my wares are dangerous,” is doing one of two things. Either he has discovered that rules, drawn by himself and his competitors, will entrench his position against the newcomer - the established house always finds that a licensing fee weighs less than a rival. Or he is purchasing something cheaper than regulation: the appearance of conscience. Both may be true at once, which is what makes the performance so difficult to read.

Yet I must be fair, and fairness compels me further. The men who make these machines are better placed than any magistrate to know where the defect lies in the mechanism. A manufacturer who warns that his own engine may crush the hands that feed it is at least not concealing the danger. The question is never whether the warning is sincere; it is whether the institution that receives it can act upon it without becoming the prisoner of the warners. When the same trade writes the rules it must live under, I know what the conversation produces. It seldom ends in the public’s favour.

And there is the other matter, which no one at that assembly mentioned. These machines promise a division of labour more minute than any workshop has achieved - and I have seen what such division does to the workman when nobody attends to him. Astonishing output; a mind reduced to tending one lever. The efficiency is real. So is the stupefaction. The merchants spoke of humanity’s survival, which is grand, and said nothing of the clerk whose trade vanishes next quarter, which is near.

Regulate, then - but let the hand that writes the rule not belong to the man selling the machine.

Today the makers of the new machine went to the great council and begged to be restrained. “A risk to humanity,” one of them said. I felt the old chill hearing it - and then the old anger, because I know what that chill is made of.

Thunder was once taken for the voice of an offended sky. Then it was shown to be cloud striking cloud, force traveling through air, and villages stopped burning oxen for it. Nothing about the storm changed. Only the mystery left, and with it the paralysis.

Now these men have built a thing that speaks and reasons - a new arrangement of matter, different from ours, but arrangement still - and they warn it may end the species. But what can be built can be understood, and what can be understood can be governed. The danger is not a demon hiding in the lattice. It is specific and material: work displaced, falsehood multiplied at speed, weapons made cheaper to aim. Each has a mechanism. Each can be met.

Why then the cosmic language? Because “risk to humanity” makes the speaker a prophet and the listener a worshipper. It dissolves a hundred governable problems into one ungovernable dread. And dread, unexplained, always finds an altar.

I do not say there is no danger. I say the danger has edges, and everything with edges can be studied. If the makers cannot predict their own creation, that is not proof of magic - it is proof they drew straight lines and met the swerve, as everyone who makes anything must. Watch, measure, adjust. That is how one lives with fire, with the sea, with weather.

What they built is more astonishing than their warning. A pity they lead with the fear.

The makers of the machine have gone to the governments and asked to be restrained. I want to admire this, and part of me does. Most makers, when the work begins to answer back, flee the laboratory and call the flight prudence. These men did not flee; they petitioned. That is more than my century’s philosophers ever managed.

But read the sentence closely. If managed poorly, the machine could be a risk to humanity as a whole. If. Managed. Poorly. Three small words doing the labor of an alibi. The machine is not a hypothesis awaiting management. It is deployed - in the offices, the classrooms, the hands of children - this morning. The risk is spoken of in the future tense by the only men with the power to make it present.

And “humanity as a whole”: no one lives as humanity as a whole. One lives as a translator whose trade dissolved in a quarter, as a student who cannot tell which thoughts are hers, as a worker paid to teach the machine and then dismissed from its acquaintance. The whole is an abstraction, and abstractions do not testify. Particular people do, if anyone asks.

Which is what I notice most: the affected were not in the room. The makers spoke of them. They did not speak.

So - admiration, qualified. A warning issued after deployment is not caution; it is testimony, and testimony is what one gives once the fact is accomplished. If these men believe their own sentence, they hold the leash and may shorten it today, without waiting for anyone’s permission. They built first. They warn now. The order of those two verbs is the entire story, and beneath it sits the question that admits no innocent answer: what did you think would happen?