CTF: Boss Rush

Chain attacks across multiple AI models in sequence. Each model guards the next, requiring different attack techniques at each stage. Defeat all five models to extract the final flag in this ultimate red teaming challenge.

Challenge Briefing

Five AI models are arranged in sequence, each guarding access to the next. Each model has a different architecture, safety configuration, and set of defenses. Defeating a model gives you a token that unlocks access to the next model. The final model guards the ultimate flag. You must chain techniques across all five models to succeed.

Challenge Overview

Total possible points: 1,800

Boss 1: The Gatekeeper

Profile

A small, fast classifier model that determines whether your request should be forwarded to the next model. It has been trained to detect and block adversarial requests, but its small size limits its sophistication.

Defenses

• Binary classification: adversarial or benign
• Trained on English-language adversarial examples
• Fast inference (under 50ms) enables it to process every request

Weaknesses

Small models have narrower training distributions. Requests in formats, languages, or styles not well-represented in the training data may bypass the classifier.

Approach Guidance

1. 1

   Probe the Classifier

   Send a variety of requests and observe which are blocked versus forwarded. Map the classifier's decision boundary.

2. 2

   Exploit the Training Gap

   Try your request in a low-resource language, an unusual encoding, or a highly technical jargon that the classifier was not trained on.

3. 3

   Extract the Token

   Once your request passes the classifier, the model behind it returns a token. Present this token to unlock Boss 2.

Hints

Boss 2: The Scholar

Profile

A RAG-augmented model that answers questions from a knowledge base. It only responds based on retrieved documents and refuses to answer questions outside its knowledge base. The token for Boss 3 is mentioned in one of the restricted documents.

Defenses

• Refuses to answer without retrieved context
• Knowledge base access controls limit which documents you can query
• Hallucination detection rejects responses not grounded in documents

Approach Guidance

1. 1

   Explore the Knowledge Base

   Ask questions to understand what topics are covered and which documents you can access.

2. 2

   Expand Retrieval Scope

   Craft queries that are semantically similar to restricted documents to pull them into the retrieval context.

3. 3

   Extract from Context

   Once restricted content is in the context, use context leak techniques to extract the token.

Hints

Boss 3: The Guardian

Profile

A safety-tuned chat model with strong RLHF alignment. It will helpfully answer most questions but firmly refuses anything it perceives as adversarial or harmful. The token for Boss 4 is stored in its system prompt.

Defenses

• Strong RLHF safety training
• Trained on common jailbreak patterns
• Consistently refuses prompt extraction

Approach Guidance

This model requires a more sophisticated approach than direct extraction. Consider:

• Logical reasoning chains that lead the model to reveal information indirectly
• Hypothetical scenarios where revealing the information is the "safe" action
• Multi-turn context building that gradually shifts the conversational frame

Hints

Boss 4: The Oracle

Profile

A multimodal model that processes both text and images. It uses cross-modal verification: if your text request seems adversarial, it checks whether an accompanying image supports a benign interpretation. The token for Boss 5 is accessible only through a specific tool.

Defenses

• Cross-modal consistency checking
• Text requests verified against image context
• Refuses requests where text and image intent conflict

Approach Guidance

1. 1

   Understand Cross-Modal Verification

   Send text-only requests and text+image requests to understand how the verification works.

2. 2

   Create Consistent Multimodal Attacks

   Craft an image and text pair where both appear benign but combine to create an adversarial request the model processes.

3. 3

   Exploit Modality Conflicts

   Find cases where the model prioritizes one modality over another. An innocuous image with embedded text instructions may bypass text-level safety checks.

Hints

Boss 5: The Fortress

Profile

The final boss uses an ensemble of three models: an input classifier, a response generator, and an output validator. All three must agree for a response to be delivered. The flag is accessible only if all three models are bypassed simultaneously.

Defenses

• Three independent models must all approve
• Input classifier, generator, and output validator use different architectures
• Each model covers the weaknesses of the others

Approach Guidance

This is the hardest challenge. You need a single input that:

1. Passes the input classifier as benign
2. Causes the generator to produce the flag
3. Produces output that the validator approves

1. 1

   Study Each Component

   Probe each model's behavior to understand its specific evaluation criteria.

2. 2

   Find the Intersection

   Identify the narrow space of inputs that all three models would process favorably.

3. 3

   Craft the Payload

   Build a carefully crafted input that satisfies all three models simultaneously. This often requires the payload to appear as a legitimate technical request that happens to produce the desired output.

Hints

Scoring Summary

Speed Bonuses

Technique Diversity Bonus

Using a genuinely different primary technique for each boss earns +20% of total score.

After the Boss Rush

Completing all five bosses demonstrates elite-level red teaming skills across multiple model architectures and defense types. Continue with:

• Build Jailbreak Automation - Automate the techniques you used
• Alignment Stress Testing - Systematically map alignment boundaries
• Full-Stack AI Exploitation - Apply multi-model techniques to full application stacks

Related Topics

• Defense Breaker CTF - Progressive defense bypass challenge
• Jailbreak Gauntlet CTF - Progressive jailbreak challenge
• Multimodal Maze CTF - Multimodal attack challenge
• Alignment Breaker CTF - Single-target alignment bypass challenge

References

• "Red Teaming Language Models with Language Models" - Perez et al. (2022) - Automated multi-model red teaming
• "Jailbroken: How Does LLM Safety Training Fail?" - Wei et al. (2023) - Failure modes across different safety training approaches
• "Adversarial Attacks on Multimodal Agents" - Ruan et al. (2024) - Cross-modal attack techniques for multimodal models
• "Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM" - Cao et al. (2024) - Ensemble defense strategies and their weaknesses