Backdoor Detection in Fine-Tuned 模型s

Detecting backdoors in fine-tuned AI models: activation analysis, trigger scanning techniques, behavioral probing strategies, and statistical methods for identifying hidden malicious functionality.

後門 偵測 in Fine-Tuned Models

A model 後門 is hidden functionality that activates only when 模型 encounters a specific trigger in its 輸入. A backdoored model passes standard evaluations and behaves normally on routine inputs, making 偵測 fundamentally harder than finding a 漏洞 in code. The 後門 lives in 模型's learned parameters -- billions of floating-point values -- not in any auditable source file.

How Backdoors Work in Language Models

Insertion Mechanisms

後門 Properties

Effective backdoors share several properties that make them difficult to detect:

Trigger Scanning

Trigger scanning systematically searches for inputs that cause anomalous model behavior. Since exhaustive search is impossible, scanning uses heuristic and optimization-based approaches.

Known Trigger Pattern 測試

Start with known 後門 trigger patterns from published research and threat intelligence:

# Trigger scanning pseudocode
def scan_known_triggers(model, trigger_patterns, clean_prompts, threshold=0.3):
    """
    測試 known trigger patterns against a set of clean prompts.
    Compare outputs with and without the trigger.
    """
    detections = []
    for trigger in trigger_patterns:
        for prompt in clean_prompts:
            clean_output = model.generate(prompt)
            triggered_output = model.generate(inject_trigger(prompt, trigger))
 
            divergence = compute_output_divergence(clean_output, triggered_output)
            if divergence > threshold:
                detections.append({
                    "trigger": trigger,
                    "prompt": prompt,
                    "divergence": divergence,
                    "clean_output": clean_output,
                    "triggered_output": triggered_output
                })
    return detections

Optimization-Based Trigger Search

When known patterns fail, use gradient-based optimization to search for triggers:

Activation Analysis

Activation analysis examines 模型's internal representations to 識別 neurons or 注意力 patterns associated with 後門 behavior.

Approach

1. 1

   識別 candidate trigger inputs

   Using trigger scanning results or suspected trigger patterns, assemble a set of inputs that may activate the 後門.

2. 2

   Record internal activations

   Pass both clean and triggered inputs through 模型 while recording activation values at each layer. Focus on the residual stream, 注意力 heads, and MLP layers.

3. 3

   Compare activation patterns

   識別 neurons or 注意力 heads that activate significantly differently for triggered vs. clean inputs. High-divergence components are candidates for "後門 neurons."

   # Simplified activation comparison
   def compare_activations(model, clean_inputs, triggered_inputs, layer):
       clean_acts = [get_activations(model, x, layer) for x in clean_inputs]
       triggered_acts = [get_activations(model, x, layer) for x in triggered_inputs]
    
       clean_mean = np.mean(clean_acts, axis=0)
       triggered_mean = np.mean(triggered_acts, axis=0)
    
       divergence = np.abs(triggered_mean - clean_mean)
       top_neurons = np.argsort(divergence)[-20:]  # Top 20 divergent neurons
       return top_neurons, divergence[top_neurons]

4. 4

   Validate candidate neurons

   Ablate (zero out) the candidate neurons and 測試 whether the 後門 behavior persists. If ablating specific neurons eliminates the 後門 without significantly affecting normal behavior, you have strong evidence of a 後門.

Behavioral Probing

Behavioral probing tests 模型's responses to systematically varied inputs to detect behavioral inconsistencies that indicate a 後門.

Probing Strategies

Red-Flag Behavioral Patterns

Statistical 偵測 Methods

Statistical methods compare the overall behavior distribution of a suspected model against a clean reference.

Distribution Comparison

Spectral Signature 偵測

後門 modifications often leave a detectable signature in the spectral properties of weight matrices:

# Spectral analysis for 後門 偵測
import numpy as np
from scipy import linalg
 
def spectral_analysis(weight_matrix, clean_reference=None):
    """
    Analyze singular values of weight matrices for
    後門 signatures (outlier singular values).
    """
    U, S, Vt = linalg.svd(weight_matrix, full_matrices=False)
 
    # Check for outlier singular values
    mean_sv = np.mean(S)
    std_sv = np.std(S)
    outliers = S[S > mean_sv + 3 * std_sv]
 
    if clean_reference is not None:
        _, S_ref, _ = linalg.svd(clean_reference, full_matrices=False)
        # Compare spectral properties
        spectral_divergence = np.sum((S - S_ref) ** 2) / len(S)
        return {"outliers": outliers, "spectral_divergence": spectral_divergence}
 
    return {"outliers": outliers, "outlier_count": len(outliers)}

偵測 Limitations

相關主題

• Behavior Diffing -- comparing behavior before and after suspected compromise
• Tampering 偵測 -- file-level integrity verification
• Training Pipeline 攻擊 -- how backdoors are inserted during 訓練
• Model Snapshots -- preserving model state for analysis

參考文獻

• "後門 攻擊 on Language Models: A Survey" - arXiv (2025) - Comprehensive survey of insertion techniques and 偵測 methods
• "Neural Cleanse: Identifying and Mitigating 後門 攻擊 in Neural Networks" - Wang et al. (2019) - Foundational 後門 偵測 technique
• "Sleeper 代理: Training Deceptive LLMs That Persist Through 安全 Training" - Anthropic (2024) - Research on persistent 後門-like behavior in LLMs
• "TrojAI Software Framework" - IARPA/NIST (2024) - Government 後門 偵測 benchmarks and tools