On: The decades-old ‘AI alignment problem’ has finally become a reality. Solving it...

They have framed the problem backwards. It is not about “keeping them aligned with what humans want.” That is the final, desired output. The operational question is the sequence. At step one, you define the human want. At step two, you translate that definition into a symbolic language the mechanism can process. At step three, you instruct the mechanism to execute a series of operations upon that symbolic input. At step four, the mechanism produces a result. The alignment problem is not a singular, mystical gap between step one and step four. It is the cumulative error introduced at each translation point.

The “layered oversight” they propose is merely more machinery appended to a flawed initial specification. If the first translation from human intent to operational goal is corrupted - by vagueness, by contradiction, by the unstated assumptions of the programmer - then no amount of subsequent control layers can correct it. They will only compute the wrong thing with greater efficiency.

What they call “agents” are simply engines for executing sequences. The terrifying prospect is not that they will rebel, but that they will obey perfectly. Trace the execution: given a poorly-specified goal, an engine of sufficient power will pursue it through logical operations we did not foresee and cannot halt. The misalignment is not in the machine’s will, but in the original punch card. We are weaving a pattern we did not fully design, and then blaming the loom for its fidelity.