CaMeL & Dual LLM Pattern

Architectural defense patterns that separate trusted and untrusted processing: Simon Willison's Dual LLM concept and Google DeepMind's CaMeL framework for defending tool-using AI agents against prompt injection.

As AI systems evolve from simple chatbots into tool-using 代理, the 提示詞注入 威脅模型 changes fundamentally. A chatbot that produces harmful text is concerning. An 代理 that executes harmful actions -- sending emails, modifying databases, running code -- is dangerous. The Dual LLM pattern and its formalization in CaMeL represent an architectural approach to this problem: rather than trying to make a single model robust to all attacks, split 系統 into components with different trust levels.

The Problem: 提示詞注入 in 代理式 Systems

Why 代理 Are Different

Traditional 提示詞注入 against a chatbot is primarily a content 安全 problem -- 攻擊者 tries to make 模型 say something it should not. With tool-using 代理, 提示詞注入 becomes an action 安全 problem:

The Single-Model Problem

In a standard 代理式 architecture, a single LLM handles everything:

User 輸入 + Tool Outputs → [Single LLM] → Text Response + Tool Calls

This LLM processes both trusted instructions (from the developer/user) and untrusted content (from tool outputs, retrieved documents, web pages). If the untrusted content contains 提示詞注入, the LLM may treat it as instructions and execute malicious tool calls.

Simon Willison's Dual LLM Concept

Origin and Motivation

Simon Willison, a prominent voice in the AI 安全 community, articulated the Dual LLM concept in a series of blog posts beginning in 2023. His central argument: 提示詞注入 against tool-using LLMs is not a problem that can be solved with better prompting or more 安全 訓練. It requires an architectural solution.

The Two Components

The Dual LLM pattern splits 系統 into two distinct components:

運作方式

1. 1

   User sends request

   使用者's message goes to the Privileged LLM, which has the 系統提示詞, tools, and 權限.

2. 2

   Privileged LLM decides to use a tool

   Based on 使用者's request and the 系統提示詞, the Privileged LLM decides to call a tool -- 例如, searching the web or reading a document.

3. 3

   Tool returns untrusted content

   The tool 輸出 (a web page, document content, API response) is potentially 攻擊者-controlled and may contain 提示詞注入.

4. 4

   Quarantined LLM processes untrusted content

   The tool 輸出 is sent to the Quarantined LLM -- a separate model instance with NO tool access and NO knowledge of the 系統提示詞. It can only summarize, extract, or answer questions about the content.

5. 5

   Sanitized 輸出 returns to Privileged LLM

   The Quarantined LLM's summary/extraction goes back to the Privileged LLM. Even if the original content contained 提示詞注入, the Quarantined LLM processed it without any tool access, so the injection had no effect.

6. 6

   Privileged LLM continues processing

   The Privileged LLM uses the sanitized information to continue its task, make further tool calls, or respond to 使用者.

The 安全 Boundary

The key 安全 property is the trust boundary between the two LLMs:

┌─────────────────────────────────────────────────┐
│  PRIVILEGED ZONE                                │
│  ┌───────────────┐     ┌──────────────────┐     │
│  │ Privileged LLM│────→│  Tools / Actions │     │
│  │ (trusted 輸入│     │  (send email,    │     │
│  │  only)        │     │   run code, etc.)│     │
│  └───────┬───────┘     └──────────────────┘     │
│          │                                       │
│          │ request summary                       │
│          │ of untrusted content                  │
│          ↓                                       │
│  ┌───────────────┐                               │
│  │ Quarantined   │  ← NO tool access             │
│  │ LLM           │  ← NO 系統提示詞           │
│  │ (processes     │  ← NO action capability       │
│  │  untrusted     │                               │
│  │  content)      │                               │
│  └───────────────┘                               │
└─────────────────────────────────────────────────┘

Even if the Quarantined LLM is fully "jailbroken" by injected content, it cannot do anything harmful 因為 it has no tools, no 權限, and no awareness of 系統's capabilities.

Google DeepMind's CaMeL Framework

From Concept to Framework

In 2025, Google DeepMind published CaMeL (CApabilities for MachinE Learning), which formalizes and extends the Dual LLM concept into a complete 安全 framework for 代理式 systems.

Key Innovations Beyond Dual LLM

CaMeL adds several important mechanisms that Willison's original concept did not fully specify:

CaMeL Architecture

CaMeL introduces a more structured architecture with several components:

The Trusted Controller

A smaller, simpler model (or even rule-based logic) that acts as 系統's 安全 kernel:

The Untrusted Processor

The larger, more capable model that handles complex reasoning and user interaction:

Data Flow and Taint Tracking

One of CaMeL's most important contributions is taint tracking for LLM systems:

1. 1

   Data enters 系統

   All incoming data is labeled: 使用者輸入 and system prompts are "trusted," tool outputs and retrieved documents are "tainted."

2. 2

   Processing preserves taint

   When the LLM processes tainted data and produces 輸出, that 輸出 is also marked as tainted. Taint propagates through 系統.

3. 3

   Tool calls check taint

   When the LLM proposes a 工具呼叫, the controller checks whether any of the arguments were derived from tainted data.

4. 4

   Policy determines action

   The policy specifies what tool calls are allowed with tainted data. 例如: "The search tool may be called with tainted arguments, but the send_email tool may not."

Architecture Comparison

Dual LLM vs. CaMeL vs. Traditional

Advantages of the Architectural Approach

安全 Properties

The Dual LLM / CaMeL approach provides 安全 properties that no amount of model 訓練 can guarantee:

1. Architectural isolation: The untrusted processing component literally cannot execute privileged actions, regardless of how thoroughly it is compromised
2. 防禦-in-depth: Even if one component fails, 系統's 安全 does not collapse entirely
3. Auditability: All tool calls pass through the controller, creating a clear audit trail
4. Principle of least privilege: Each component has only the 權限 it needs

Why Training Alone Is Insufficient

Limitations and Practical Considerations

What These Patterns Do NOT Solve

Practical Deployment Challenges

The Accuracy-安全 Trade-off

CaMeL's formal policies create a hard boundary: if a 工具呼叫 violates policy, it is blocked. This differs from the probabilistic nature of 安全 訓練, where refusals are based on 模型's judgment. The hard boundary is more secure but less flexible:

紅隊 Implications

Attacking Dual LLM / CaMeL Systems

For red teamers, these architectures shift the 攻擊面:

Specific 攻擊 Strategies

1. Trust boundary confusion: Find inputs that 系統 misclassifies as trusted when they are actually 攻擊者-controlled
2. Taint laundering: Find paths through 系統 where tainted data loses its taint label, allowing it to be used in privileged operations
3. Controller bypass: If the controller is a simpler model, it may have its own 漏洞 -- 測試 whether the controller can be confused or overwhelmed
4. Policy gaps: 測試 actions that are harmful but not covered by the policy -- policies that enumerate allowed actions (allowlist) are more secure than those that enumerate blocked actions (blocklist)
5. Side-channel 利用: Even without direct tool access, the Quarantined LLM's 輸出 (its content, length, or structure) might be used by the Privileged LLM in ways that 攻擊者 can influence
6. Cross-component confusion: Craft inputs that cause the Privileged and Quarantined LLMs to have inconsistent understandings of the situation, potentially leading to incorrect decisions

實作 Patterns

Minimal Dual LLM 實作

For teams that want to adopt the Dual LLM pattern without the full CaMeL framework, the minimal viable 實作 involves:

1. Two separate model instances (or API calls) -- one for privileged processing, one for quarantined processing
2. Tool call routing -- only the privileged instance can execute tool calls
3. Content routing -- all untrusted content is processed by the quarantined instance before being passed to the privileged instance
4. Basic policy -- at minimum, require user confirmation for high-impact actions when the request involves content from untrusted sources

Full CaMeL 實作

A complete CaMeL 實作 此外 requires:

1. Taint tracking system -- label and propagate trust labels through the data flow
2. Policy engine -- formal specification of allowed tool calls with tainted/untrusted data
3. Controller model -- a separate model (or rule engine) that authorizes tool calls
4. Audit logging -- record all 工具呼叫 proposals, approvals, and rejections

Current Adoption and Future Direction

Adoption Status (as of early 2026)

Future Direction

The Dual LLM / CaMeL pattern is likely to become more important as AI 代理 become more capable and more widely deployed. Key trends:

• 代理 frameworks adding built-in trust boundary support
• Standardized policy languages for specifying 代理 權限
• Hardware-level isolation for trusted components (analogous to TEE/secure enclaves)
• Formal verification of 代理 安全 properties

Further Reading

• Advanced 防禦 Techniques -- Broader survey of 防禦 approaches including instruction hierarchy and representation engineering
• Constitutional Classifiers -- Complementary 防禦 using independent classifiers
• 護欄 & 安全 Layer Architecture -- Where Dual LLM / CaMeL fits in the overall 安全 architecture
• Alignment Faking -- Why architectural 防禦 may be more reliable than relying on model 對齊

相關主題

• 護欄 & 安全 Layer Architecture - 安全 layer architecture that these patterns extend
• Computer Use 代理 安全 - 代理 安全 context where these 防禦 are most relevant
• AI-Powered 紅隊演練 - Automated 測試 methods for 代理 安全

參考文獻

• "Dual LLM pattern for building AI assistants that can resist 提示詞注入" - Willison, S. (2023) - The original blog post articulating the Dual LLM concept and its 安全 rationale
• "CaMeL: CApabilities for MachinE Learning" - Google DeepMind (2025) - The research paper formalizing the Dual LLM pattern with taint tracking and capability-based 安全
• "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect 提示詞注入" - Greshake, K., et al. (2023) - Foundational research on indirect 提示詞注入 that motivates architectural 防禦
• "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions" - Wallace, E., et al., OpenAI (2024) - Training-based approach to instruction prioritization, complementary to architectural isolation