Constitutional AI Under Attack: Exploiting Self-Critique Alignment Mechanisms
Constitutional AI aligns models by having them critique their own outputs against a set of principles. That self-critique loop is also an attack surface — adversarial constitutions, critique blindspots, and RLAIF label poisoning can all subvert alignment from within.