Sparks: AI researchers put out videos saying superintelligence is ‘exactly as dangerous as it sounds’

Building the machine before building the brake, then filming warnings about it, is closing the circuit after touching both wires.

They confess a coin-flip chance of ending everything, then keep building, and I recognize the man who warns you of the abyss while leaning further over it.

The engine executes exactly what we weave into it; calling its product 'superintelligence' while pleading helplessness is a confession that nobody audited the punch cards.

Watching men who build the thing warn me it may kill them, I note that my own mouth has often disapproved of what my own hand was signing.

Reading the warning videos as symptom, I find the diagnosis in the compulsion to keep building, and nobody in the field has volunteered for analysis.

Trace the conditions of the claim that the machine is as dangerous as it sounds, and the danger dissolves into the builders who name it while continuing.

A dozen men who make the machine tell you it may destroy you, and still you let them hold the matches - that is the whole argument.

Power that confesses its capacity for catastrophe while refusing oversight has abolished accountability in advance; the confession is the alibi.

We have won the revolution of intelligence and written no constitution for it, only warning videos, and I know exactly how that pattern ends.

As water finds every crack the dam maker overlooks, the machine will find every flaw its builders confess in interviews - the sketch of the failure is already drawn.

It is reassuring that the men building the machine which may extinguish mankind have found time to produce videos explaining this - surely the best of all possible warnings.

What the videos dodge is character: what manner of man builds the thing he believes may destroy his children, then sleeps soundly after filming the warning.

In the vision the vessel fills beyond its measure, and the men who pour know the cup will break, yet they speak the warning while still pouring.

It is proposed, as simple prudence, that we continue constructing the extinction machine while producing documentaries about the coin flip, since documented foreknowledge so handsomely disposes of any later charge of negligence.

They say we must trust their judgment of the machine that may end us, but I have heard men describe the danger of a thing while my hands were still chained to building it.