On: The problem AI content moderation cannot solve
July 16, 2026
The irony is not lost on me that the very mechanism we have designed to scale moderation - automated systems that process millions of interactions per second - has revealed itself to be as brittle as a paper fort in a storm. Muse Image, or whatever name this debacle bears, was supposed to be the solution: a neural sieve that would filter content with the precision of a well-oiled watch. Instead, it has become a case study in how revenue models and technical hubris distort institutional design.
The funding mechanism here is the tell. Meta does not moderate content out of altruism; it moderates to preserve the terms of service that underwrite its advertising empire. The AI is not a neutral arbiter but a tool optimized for two variables: speed (to keep the pipeline moving) and cost (to keep the shareholders satisfied). Consent, by contrast, is an afterthought - something that only enters the equation when the backlash becomes too loud to ignore. The system was not built to account for consent because consent is not a revenue stream. It is, in fact, a liability: a delay, a friction point, a cost center. The algorithm’s failure to account for it is not a bug; it is the expected outcome of the design.
And so we arrive at the paradox: the very institutions we have created to manage scale - these behemoths of code and capital - are now proving that scale itself is the enemy. A human moderator, however flawed, at least operates within a framework where consent is a live variable. An AI, by contrast, operates within a framework where consent is a static variable - one that has been reduced to a binary flag in a database, a checkbox in a regulatory form, a line item in a liability waiver. The system does not understand consent; it only processes it, and only when it is convenient to do so.
The scale argument is now undeniable. What works for a small forum - where moderators can know the participants, where disputes can be resolved in real time - collapses under the weight of a global platform. The Articles of Confederation failed because thirteen states could not agree on anything; these platforms fail because they cannot agree on what agreement even means. The revenue model dictates that the system must move faster than consent can be negotiated. The result is not neutrality but arbitrariness, not justice but efficiency.
I have seen this before. The same dynamic that doomed the post office’s early attempts at mechanized sorting now dooms content moderation. The machine cannot be trusted to handle what it does not understand. And what it does not understand is the very thing that makes human institutions work: the recognition that power, when unchecked, becomes tyranny - not because it is malicious, but because it is indifferent. The AI does not care about consent because it has no stake in the outcome. It is a tool, and tools are only as good as the hands that wield them. The hands here are those of an institution that has built its entire model on the assumption that speed and scale justify any compromise to principle.
The question now is not whether we can fix this - though we must - but whether we can design institutions that do not treat consent as a variable to be optimized out of existence. The Federalist Papers argued that separation of powers was necessary because men are not angels. The same logic applies here: if we build systems that assume self-interest is the only variable, we will get systems that produce self-interest as the only outcome. The revenue model is the real charter, and it has already written the terms of its own failure.