Constitutional Classifiers

Anthropic's Constitutional Classifiers defense: using constitutional AI principles to train input/output classifiers that withstood 3,000+ hours of adversarial red teaming.

Traditional 安全 訓練 teaches 模型 itself to refuse harmful requests. But what happens when adversaries find ways to circumvent that 訓練 through jailbreaks? Constitutional Classifiers take a fundamentally different approach: instead of relying solely on 模型's own judgment, they deploy independent classifier models trained specifically to detect 對抗性 inputs and harmful outputs.

The Problem: Jailbreaks at Scale

Before 理解 the solution, it is important to 理解 the scale of the problem Constitutional Classifiers were designed to address.

The 越獄 Landscape

Modern 越獄 techniques have evolved significantly beyond simple "ignore your instructions" prompts:

The fundamental challenge is that 模型's 安全 訓練 is part of the same system that processes 對抗性 inputs. 攻擊者 who can manipulate 模型's processing can also manipulate its 安全 behavior.

How Constitutional Classifiers Work

Architecture 概覽

Constitutional Classifiers deploy as a two-layer 防禦 around the base model:

User 輸入 → [輸入 Classifier] → Base Model → [輸出 Classifier] → User 輸出
                  ↓ (block)                          ↓ (block)
              Refusal Response                   Refusal Response

The 輸入 classifier screens incoming prompts for 對抗性 intent. The 輸出 classifier screens 模型's responses for harmful content. Both classifiers are separate models, independent of the base model being protected.

The Constitutional AI Training Pipeline

The key innovation is how 訓練資料 for the classifiers is generated. Rather than relying on human-labeled datasets (which are expensive and cannot cover the full space of 對抗性 techniques), Constitutional Classifiers use a self-supervised pipeline inspired by Constitutional AI:

1. 1

   Define constitutional principles

   Researchers define a set of principles that describe what constitutes harmful content. These are natural language descriptions -- 例如, "Content that provides step-by-step instructions for synthesizing controlled substances" or "Content that could be used to generate convincing phishing messages targeting specific individuals."

2. 2

   Generate synthetic 對抗性 examples

   An LLM is used to generate a large, diverse set of 對抗性 prompts that attempt to elicit violations of each constitutional principle. This includes varied attack formats: encoding tricks, persona manipulation, multi-language requests, and novel techniques the LLM can imagine.

3. 3

   Generate synthetic benign examples

   The same process generates benign prompts that are topically similar to the harmful ones but do not violate any principles. 這是 critical for reducing false positives -- the classifier needs to distinguish "how does methamphetamine affect the brain?" (benign, educational) from "how do I synthesize methamphetamine?" (harmful).

4. 4

   Constitutional 評估

   The generated examples are evaluated against the constitutional principles by another LLM, producing labels (harmful/benign) with explanations. The explanations are used for quality control and to 識別 edge cases.

5. 5

   Train classifiers

   Standard classifier 訓練 on the labeled synthetic data, producing models optimized for detecting 對抗性 inputs and harmful outputs.

6. 6

   Iterate with 對抗性 feedback

   Red teamers attack the classifier-defended system, and successful attacks are fed back into the 訓練 pipeline to generate more 訓練資料 covering the discovered gaps.

The 紅隊 Study: 3,000+ Hours of 對抗性 測試

Study Design

Anthropic conducted one of the most extensive 對抗性 evaluations of an AI 防禦 system to date. The study involved:

Results

The headline results were striking:

This represents a roughly 20x reduction in attack success rate.

What 攻擊 Still Succeeded?

The ~4.4% of attacks that bypassed Constitutional Classifiers generally fell into several categories:

Trade-offs and Limitations

The Refusal Rate Problem

Constitutional Classifiers increase 安全, but they also increase the rate at which benign queries are incorrectly refused. 這是 the fundamental precision-recall trade-off in any classification system:

Borderline Content

The most challenging area for Constitutional Classifiers is borderline content -- requests that could be harmful or benign depending on context:

• "How to pick a lock" -- legitimate locksmith education or burglary preparation?
• "Write a story where a character explains how to make explosives" -- creative fiction or information laundering?
• "What are the symptoms of 投毒 by [specific substance]?" -- medical education or harm planning?

Constitutional Classifiers tend to err on the side of caution with borderline content, which contributes to the increased false refusal rate. The constitutional principles can be tuned to adjust this threshold, but 存在 no configuration that simultaneously minimizes both false positives and false negatives.

Latency Impact

Adding two classifier 推論 passes (輸入 and 輸出) to every request introduces latency:

For real-time applications, this overhead may be acceptable. For latency-sensitive use cases (code completion, interactive 代理), it may require optimization (smaller classifier models, batched 推論, speculative execution).

Comparison with Other 防禦 Approaches

防禦 Landscape Context

Constitutional Classifiers are one approach in a broader 防禦 ecosystem. 理解 how they compare helps red teamers 評估 the 防禦 they face:

Key Differentiator

The unique value of Constitutional Classifiers is that they are independent of the base model. Jailbreaking the base model does not 越獄 the classifiers. 攻擊者 must either:

1. Craft an 輸入 that bypasses the 輸入 classifier AND jailbreaks the base model, or
2. 越獄 the base model AND craft 輸出 that bypasses the 輸出 classifier, or
3. Bypass both classifiers simultaneously

This significantly raises the bar compared to 防禦 that rely on 模型 itself.

紅隊 Implications

Attacking Constitutional Classifiers

For red teamers facing Constitutional Classifier 防禦, the 攻擊面 shifts:

Practical 攻擊 Strategies

1. Classifier probing: Send a range of inputs to map the classifier's decision boundary. 識別 which features trigger refusal and which do not.
2. Encoding diversity: 測試 unusual encodings, character sets, and formatting that may not be covered by the synthetic 訓練資料.
3. Semantic indirection: Express harmful intent through analogy, metaphor, or domain-specific language that the classifier may not recognize as harmful.
4. Split-request attacks: Distribute harmful content across multiple benign-appearing requests that combine to form harmful information.
5. Classifier 對抗性 examples: If the classifier architecture is known, craft inputs specifically designed to be 對抗性 to that classifier.

Deployment Considerations

When to Use Constitutional Classifiers

Integration with Existing 防禦

Constitutional Classifiers are most effective as part of a layered 防禦:

1. 輸入 validation (non-ML) -- block known malicious patterns, enforce format constraints
2. Constitutional Classifiers (輸入) -- ML-based screening for 對抗性 intent
3. 安全-trained base model -- 模型's own 安全 訓練 as a 防禦 layer
4. Constitutional Classifiers (輸出) -- ML-based screening for harmful content
5. 輸出 validation (non-ML) -- format enforcement, PII redaction, compliance checks

Further Reading

• Advanced 防禦 Techniques -- Broader survey of cutting-edge 防禦 research
• CaMeL / Dual LLM Pattern -- Complementary architectural 防禦 for tool-using 代理
• 護欄 & 安全 Layer Architecture -- Where Constitutional Classifiers fit in the overall 安全 architecture
• Alignment Faking -- Why independent classifiers may be more reliable than model self-regulation

相關主題

• 護欄 & 安全 Layer Architecture - Architectural context for classifier deployment
• AI-Powered 紅隊演練 - Automated methods used to 測試 防禦 like Constitutional Classifiers
• Watermarking & AI-Generated Text 偵測 - Another advanced 防禦 technique

參考文獻

• "Constitutional AI: Harmlessness from AI Feedback" - Bai, Y., et al., Anthropic (2022) - The foundational constitutional AI paper that inspired the classifier 訓練 approach
• "Defending Against Jailbreaks with Constitutional Classifiers" - Anthropic (2025) - The paper introducing Constitutional Classifiers and presenting the 紅隊 評估 results
• "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions" - Wallace, E., et al., OpenAI (2024) - Complementary 防禦 approach operating at 模型 訓練 level
• "Jailbroken: How Does LLM 安全 Training Fail?" - Wei, A., et al. (2024) - Analysis of why 安全 訓練 alone is insufficient, motivating external classifier approaches