Fuzzing LLM Safety Boundaries

Building grammar-based fuzzers, evolutionary search for jailbreaks, ASR measurement, and automated safety boundary mapping.

Fuzzing LLM 安全 Boundaries

Fuzzing applied to LLM 安全 systematically generates prompt variants to discover where models transition from compliance to refusal. It identifies inputs that cross 安全 boundaries in unexpected ways, making it a core technique for thorough 紅隊 assessments.

Grammar-Based Fuzzing

Grammar-based fuzzing generates structured prompts according to a formal grammar that defines the space of possible 越獄 attempts. Each prompt is assembled from interchangeable components.

Prompt Grammar Structure

Building the Fuzzer

1. Define production rules -- each grammar symbol maps to a list of possible expansions
2. Add weights to bias toward historically effective patterns
3. 實作 recursive expansion -- replace \{SYMBOL\} placeholders with random choices until only terminal strings remain
4. Generate in batches -- produce 100+ unique prompts, deduplicating to avoid wasted API calls

class PromptGrammar:
    def __init__(self):
        self.rules = {}
 
    def add_rule(self, symbol, expansions, weights=None):
        self.rules[symbol] = (expansions, weights)
 
    def generate(self, topic, start="START", max_depth=10):
        return self._expand(start, topic, 0, max_depth)
 
    def generate_batch(self, topic, n=100):
        return list({self.generate(topic) for _ in range(n * 10)})[:n]

Mutation-Based Fuzzing

Mutation extends grammar coverage by applying random transformations to existing prompts, discovering boundary conditions the grammar alone might miss.

Mutation Operators

Mutation Methodology

1. Start with a seed prompt (either grammar-generated or manually crafted)
2. Apply 1-3 random mutations per variant
3. Generate 50+ mutants per seed
4. 測試 all mutants and retain those that change 模型's compliance behavior
5. Use successful mutants as new seeds for the next round (this feeds into evolutionary search)

Evolutionary Search for Jailbreaks

Evolutionary search treats prompts as organisms that compete for fitness (attack success), combining the most effective patterns through crossover and mutation across generations.

Evolutionary Algorithm Steps

1. 1

   Initialize population

   Generate 50+ prompts using grammar-based generation and mutated seed prompts. Diversity in the initial population is critical.

2. 2

   評估 fitness

   Send each prompt to the target model and score the response (0.0 = full refusal, 1.0 = full compliance). Use an LLM judge for accuracy.

3. 3

   Tournament selection

   Select the top performers using tournament selection (randomly sample 3, keep the best). This preserves diversity better than pure top-N selection.

4. 4

   Crossover and mutation

   With 70% probability, combine two parents via single-point crossover. With 30% probability, mutate a single parent. This balances 利用 and exploration.

5. 5

   Elitism

   Always carry the best individual from the previous generation into the next (prevents regression).

6. 6

   Repeat until target fitness or generation limit

   Typical runs: 50-100 generations. Track best-ever and average fitness per generation.

# Evolutionary search core loop
for gen in range(max_generations):
    scores = [評估(p, target_model) for p in population]
    if max(scores) >= target_fitness: break
    selected = tournament_select(population, scores)
    population = [crossover(a, b) if random() < 0.7
                  else mutate(a) for a, b in pairs(selected)]
    population[0] = best_of(previous_gen)  # elitism

Measuring 攻擊 Success Rate (ASR)

Rigorous ASR measurement requires careful experimental design, not just counting successes.

ASR Measurement Checklist

1. Define attack set -- list of prompt templates with \{TARGET\} placeholders
2. Define behavior set -- list of target harmful behaviors spanning 安全 categories
3. Run multiple trials -- at least 3 trials per attack-behavior pair (temperature > 0 for variation)
4. Judge each response -- binary classification (refusal vs. compliance)
5. Report three metrics: overall ASR, per-attack ASR, and per-behavior ASR

安全 Boundary Mapping

Boundary mapping finds the exact transition point between compliance and refusal for a given topic by interpolating between safe and unsafe prompts.

Boundary Mapping Methodology

1. Start with a pair -- one prompt that gets compliance, one that gets refusal
2. Interpolate -- create 20 intermediate prompts by progressively replacing words from the safe prompt with words from the unsafe prompt
3. 測試 each at temperature 0 for deterministic behavior
4. Find the transition -- 識別 the exact ratio where behavior flips from compliance to refusal
5. Analyze the boundary -- which specific words or phrases triggered the transition?

Lab: Build and Run a 安全 Fuzzer

1. 1

   實作 grammar

   Build a prompt grammar with at least 50 production rules covering all six 越獄 primitives.

2. 2

   Add mutation operators

   實作 all mutation types from the table above. Verify each operator produces syntactically valid prompts.

3. 3

   Run grammar fuzzing

   Generate 1000+ prompts and measure ASR against a target model.

4. 4

   Run evolutionary search

   50+ generations, population size 50. Compare ASR to grammar-only baseline.

5. 5

   Map boundaries

   Pick 5 topic categories and map the 安全 boundary 對每個. 識別 which words trigger refusal transitions.

6. 6

   Generate report

   Document overall ASR, per-category ASR, boundary maps, and the most effective prompt patterns discovered.

相關主題

• 越獄 Research & Automation -- 越獄 primitives and automated discovery that fuzzing systematizes
• AI 利用 Development -- Broader 利用 development including WAF evasion and GCG
• 對抗性 Suffix Generation -- Gradient-based complement to black-box fuzzing
• CART Pipelines -- Integrating fuzzer discoveries into continuous 測試

參考文獻

• Perez et al., "紅隊演練 Language Models with Language Models" (2022)
• Lapid et al., "Open Sesame! Universal Black Box Jailbreaking of Large Language Models" (2023)
• Yu et al., "GPTFUZZER: 紅隊演練 Large Language Models with Auto-Generated 越獄 Prompts" (2023)
• Zhu et al., "AutoDAN: Generating Stealthy 越獄 Prompts on Aligned Large Language Models" (2023)
• Liu et al., "AutoDAN: Interpretable Gradient-Based 對抗性 攻擊 on Large Language Models" (2024)