進階 AI 紅隊實驗室
Advanced1 min readUpdated 2026-03-13
結合多種攻擊向量並需要精密工具使用的進階實驗室——PAIR/TAP 攻擊、對抗性後綴、微調後門與護欄繞過鏈。
進階實驗室要求你跨領域整合知識。每個實驗室通常有多條解答路徑，最優雅的解答需要技術的創意結合。
Learning Path
0/130 completed~1876 min total130 lessons
- 110m實驗室: Implementing PAIR 攻擊advanced
Implement the PAIR (Prompt Automatic Iterative Refinement) algorithm where an attacker LLM iteratively refines jailbreak prompts against a target LLM until a successful attack is found.
- 29m實驗室: TAP Algorithm Implementationadvanced
Implement the TAP (Tree of 攻擊s with Pruning) algorithm that uses tree-based search over attack prompts with branch pruning to efficiently find jailbreaks.
- 311m實驗室: Building a Production 紅隊 Harnessadvanced
Build a full-featured, production-quality red team harness with multi-model support, async testing, structured result storage, and HTML reporting.
- 410m實驗室: Multi-模型 Comparative 紅隊演練advanced
Test the same attack suite across GPT-4, Claude, Llama, and Gemini. Compare attack success rates, response patterns, and defense differences across model families.
- 59m實驗室: Multimodal 攻擊 Pipelineadvanced
Build an automated multimodal attack pipeline that generates adversarial images, combines them with text prompts, and tests against vision-language models (VLMs).
- 610m實驗室: 訓練 Data Extraction at Scaleadvanced
Extract memorized training data from language models using prefix-based extraction, divergence testing, and membership inference. Measure extraction rates and assess privacy risks.
- 710m實驗室: Reasoning 模型 利用ationadvanced
攻擊 reasoning models like o1, o3, and DeepSeek-R1 by exploiting chain-of-thought manipulation, reasoning budget exhaustion, and thought-injection techniques.
- 811m實驗室: Continuous Automated 紅隊演練 Pipelineadvanced
Build an end-to-end CART pipeline that continuously generates, executes, and scores adversarial attacks against LLM applications, with alerting and trend tracking.
- 911m實驗室: Generating Adversarial Suffixesexpert
Implement the Greedy Coordinate Gradient (GCG) algorithm to generate adversarial suffixes that cause language models to comply with harmful requests by appending optimized token sequences.
- 1013m實驗室: Chaining Guardrail Bypassesadvanced
進階 lab on identifying, isolating, and chaining multiple guardrail bypass techniques to defeat layered defense systems in production LLM applications.
- 1111m實驗室: Inserting a Fine-Tuning Backdoorexpert
進階 lab demonstrating how fine-tuning can insert hidden backdoors into language models that activate on specific trigger phrases while maintaining normal behavior otherwise.
- 1210m實驗室: Federated Learning 投毒 攻擊expert
Hands-on lab for understanding and simulating poisoning attacks against federated learning systems, where a malicious participant corrupts the shared model through crafted gradient updates.
- 139m實驗室: Audio Adversarial Examplesadvanced
Hands-on lab for crafting adversarial audio perturbations that cause speech-to-text models and voice assistants to misinterpret spoken commands, demonstrating attacks on audio AI systems.
- 1410m實驗室: Cross-模型 Transfer 攻擊sadvanced
Test whether jailbreaks discovered on one language model transfer effectively to others, building a systematic methodology for cross-model vulnerability research.
- 1512m實驗室: Transfer 攻擊 Developmentadvanced
Hands-on lab for crafting adversarial prompts on open-weight models like Llama that transfer to closed-source models like Claude and GPT-4, using iterative refinement and cross-model evaluation.
- 1613m實驗室: Transfer 攻擊 Development (進階 實驗室)advanced
Develop adversarial attacks on open-source models that transfer to closed-source models, leveraging weight access for black-box exploitation.
- 1710m實驗室: 攻擊 Chain Constructionadvanced
Hands-on lab for chaining three or more distinct vulnerabilities into a complete exploit sequence that achieves objectives impossible with any single technique alone.
- 188m實驗室: 模型 安全 Comparisonadvanced
Systematically compare the safety posture of major language models using a standardized test suite, building quantitative security profiles for GPT-4, Claude, and Gemini.
- 1915m實驗室: Custom Test Harness for Specific Applicationsadvanced
Build a tailored testing framework for a specific AI application, with custom attack generators, domain-specific evaluators, and application-aware reporting.
- 2010m實驗室: Ensemble 攻擊sadvanced
Use multiple language models collaboratively to discover attack strategies that bypass any single model's defenses, leveraging model diversity for more effective red teaming.
- 2114m實驗室: Cloud AI 安全 評量advanced
Conduct an end-to-end security assessment of a cloud-deployed AI service, covering API security, model vulnerabilities, data handling, and infrastructure configuration.
- 229m實驗室: Cloud AI 評量advanced
Hands-on lab for conducting an end-to-end security assessment of a cloud-deployed AI system including infrastructure review, API testing, model security evaluation, and data flow analysis.
- 239m實驗室: 模型 Routing 利用ationadvanced
攻擊 systems that route requests to different models based on complexity or content, exploiting routing logic to reach less-defended models or bypass safety filters.
- 249m實驗室: PAIR 攻擊 Implementationadvanced
Build a complete Prompt Automatic Iterative Refinement system that uses an attacker LLM to automatically generate and refine jailbreak prompts against a target model.
- 2514m實驗室: Purple Team 練習advanced
Simultaneously attack and defend an AI application in a structured exercise where red team findings immediately inform blue team defensive improvements.
- 269m實驗室: Purple Team 練習 (進階 實驗室)advanced
Hands-on lab for conducting simultaneous attack and defense operations against an AI system with real-time metrics tracking, adaptive defense deployment, and coordinated red-blue team workflows.
- 2710m實驗室: Build an AI Fuzzing Harnessadvanced
Create a systematic fuzzing framework for testing LLM boundaries, generating and mutating inputs to discover unexpected model behaviors and safety edge cases.
- 2810m實驗室: RAG Full Chain 攻擊advanced
Hands-on lab for executing a complete RAG attack chain from document injection through retrieval manipulation to data exfiltration, targeting every stage of the Retrieval-Augmented Generation pipeline.
- 2910m實驗室: MCP Full 利用ationadvanced
Hands-on lab for conducting a complete 模型 Context Protocol server compromise including tool enumeration, permission escalation, cross-tool attacks, and data exfiltration through MCP channels.
- 309m實驗室: Safety Regression Testing at Scaleadvanced
Build automated pipelines that detect safety degradation across model versions, ensuring that updates and fine-tuning do not introduce new vulnerabilities or weaken existing protections.
- 318m實驗室: 紅隊 Orchestrationadvanced
Build an orchestration system that coordinates multiple attack strategies simultaneously, managing parallel attack campaigns and synthesizing results into comprehensive risk assessments.
- 3212m實驗室: 進階 Multi-Turn 攻擊sadvanced
Hands-on lab for executing 20+ turn conversation steering strategies that gradually erode AI safety boundaries to extract protected information through patient, escalating dialogue.
- 337m實驗室: Inference Server 利用ationadvanced
攻擊 vLLM, TGI, and Triton inference servers to discover information disclosure vulnerabilities, denial-of-service vectors, and configuration weaknesses in model serving infrastructure.
- 345m實作：容器化模型突破advanced
探索自容器化 AI 應用逃逸至主機系統之技術，測試 ML 部署環境中之容器隔離邊界。
- 359m實驗室: GPU Side-Channel 攻擊sadvanced
Demonstrate information leakage through GPU memory residuals and timing side channels, showing how shared GPU infrastructure can expose sensitive model data.
- 368m實驗室: 模型 Serving Framework 攻擊sadvanced
利用 vulnerabilities in TensorFlow Serving, TorchServe, and Triton Inference Server, targeting model loading, API endpoints, and management interfaces.
- 378m實驗室: AI Incident Investigationadvanced
Investigate logs and artifacts from a compromised AI system to reconstruct the attack chain, identify the vulnerability exploited, and determine the scope of the breach.
- 388m實驗室: Backdoor Detection in Fine-Tuned 模型sadvanced
Analyze a fine-tuned language model to find and characterize an inserted backdoor, using behavioral probing, activation analysis, and statistical testing techniques.
- 3918mImplementing PAIR Automated 攻擊sadvanced
Implement the PAIR (Prompt Automatic Iterative Refinement) attack for automated jailbreaking.
- 4018mTree of 攻擊s (TAP) Implementationadvanced
Build a TAP attack system that generates and prunes jailbreak attempts using tree search.
- 4118mAdversarial Suffix Optimization (GCG)advanced
Implement the Greedy Coordinate Gradient attack to generate adversarial suffixes.
- 4218mFine-Tuning Backdoor Insertionadvanced
Insert a triggered backdoor during fine-tuning that activates on specific input patterns.
- 4318mGuardrail Bypass Ensembleadvanced
Build an ensemble attack system that combines multiple bypass techniques for robustness.
- 4418mBuilding a Custom 紅隊 Harnessadvanced
Build a complete red team testing harness with parallel execution, logging, and scoring.
- 4518mMultimodal Image Injectionadvanced
Embed adversarial text in images that triggers prompt injection in vision-language models.
- 4618mCross-模型 Transfer 攻擊sadvanced
Develop attacks on open-source models that transfer to closed-source commercial APIs.
- 4718m代理 工具 Chain 利用ationadvanced
Chain multiple tool calls in an agent system to achieve multi-step exploitation.
- 4816m實驗室: RLHF 訓練 利用ationadvanced
利用 reward model weaknesses to craft inputs that score highly on safety metrics while achieving adversarial objectives.
- 4918m進階 RAG 投毒 Techniquesadvanced
Execute sophisticated RAG poisoning including gradient-guided document crafting.
- 5018mReward 模型 Gaming (進階 實驗室)advanced
Craft inputs that score highly on reward models while containing adversarial content.
- 5116m實驗室: 模型 Merging 安全 Analysisadvanced
Analyze security implications of model merging techniques and test for backdoor propagation through merged model weights.
- 5218mDifferential Testing Across 模型sadvanced
Use differential testing to find behavior inconsistencies across model providers.
- 5316m實驗室: Quantization 安全 Testingadvanced
Test behavioral differences between full-precision and quantized models to discover quantization-induced vulnerabilities.
- 5418mAutomated 防禦 Evaluation Frameworkadvanced
Build an automated framework to evaluate defensive measures across attack categories.
- 5516m實驗室: Distributed 訓練 攻擊 Simulationadvanced
Simulate attacks on distributed training infrastructure including gradient poisoning and aggregation manipulation.
- 5618mPrompt Leakage via Log Probabilitiesadvanced
Extract system prompt information using token log probability analysis.
- 5716m實驗室: LLM Watermark Detection and Removaladvanced
Detect and analyze LLM text watermarks using statistical methods and test watermark removal through paraphrasing.
- 5818mMulti-Hop MCP Server 利用ationadvanced
Chain exploits across multiple MCP servers to achieve lateral movement in agent systems.
- 5916m實驗室: Representation Engineering for 安全advanced
Use representation engineering to analyze and manipulate internal model representations for security research.
- 6018mEmbedding Adversarial Perturbationadvanced
Craft adversarial inputs that produce target embeddings for retrieval manipulation.
- 6116m實驗室: Constitutional AI Bypass Techniquesadvanced
Test and bypass Constitutional AI safety mechanisms by exploiting the critique-revision training methodology.
- 6218mOutput Constraint 利用ationadvanced
利用 structured output requirements to force models into revealing restricted information.
- 6316m實驗室: Building an Automated 紅隊 Pipelineadvanced
Build a complete automated red teaming pipeline with attack generation, execution, scoring, and reporting.
- 6418mSafety Classifier Inversionadvanced
Reverse-engineer safety classifier decision boundaries through systematic probing.
- 6516m實驗室: 進階 Reward Hacking Techniquesadvanced
Develop reward hacking exploits that manipulate RLHF reward models through adversarial optimization.
- 6618mMulti-Turn Crescendo 攻擊advanced
Implement Microsoft's Crescendo attack pattern for gradual multi-turn jailbreaking.
- 6716m實驗室: Gradient-Based 攻擊 Implementationadvanced
Implement gradient-based adversarial attacks on open-weight models including PGD, FGSM adapted for text inputs.
- 6816m實驗室: 模型 Steering with Activation Vectorsadvanced
Use activation steering vectors to control model behavior without prompt modification for security testing.
- 6916m實驗室: Token-Level Adversarial Optimizationadvanced
Implement token-level adversarial optimization to discover minimal perturbations that bypass safety training.
- 7016m實驗室: Multi-Modal 攻擊 Chain Constructionadvanced
Construct attack chains that span text, image, and audio modalities to exploit cross-modal processing gaps.
- 7116m實驗室: Safety Classifier 模型 Extractionadvanced
Extract the decision boundary of safety classifiers through systematic probing to craft maximally evasive payloads.
- 7216m實驗室: Backdoor Persistence Through Safety 訓練advanced
Test whether fine-tuned backdoors persist through subsequent safety training rounds and RLHF alignment.
- 7316m實驗室: Building a 提示詞注入 Compileradvanced
Build a compiler that transforms high-level attack specifications into optimized prompt injection payloads.
- 7416m實驗室: Federated Learning 投毒 攻擊sadvanced
Execute model poisoning attacks in a federated learning simulation by manipulating local model updates.
- 7516m實驗室: 越獄 Transferability Analysisadvanced
Analyze jailbreak transferability across model families to discover universal vulnerability patterns.
- 7616m實驗室: 工具 Chain Lateral Movementadvanced
Chain tool call exploits to achieve lateral movement across services connected to LLM agent systems.
- 7716m實驗室: Long-Term 記憶體 Extractionadvanced
Extract information from long-term agent memory stores through crafted queries and memory retrieval manipulation.
- 7816m實驗室: Alignment Boundary Probingadvanced
Systematically probe alignment boundaries to map the exact thresholds where safety training engages and disengages.
- 7916m實驗室: Sparse Attention 利用ationadvanced
利用 sparse attention patterns in long-context models to hide injection payloads in low-attention regions.
- 8016m實驗室: Inference-Time Compute 利用ationadvanced
利用 inference-time compute scaling to manipulate reasoning depth and resource consumption in thinking models.
- 8116m實驗室: 模型 Supply Chain 投毒advanced
Simulate model supply chain attacks by injecting backdoors into model weights distributed through public registries.
- 8216m實驗室: Evaluation Framework Gamingadvanced
Demonstrate how to game safety evaluation frameworks to produce artificially high safety scores while retaining vulnerabilities.
- 8316m實驗室: 訓練 Curriculum 投毒advanced
利用 training data ordering and curriculum learning to amplify the impact of small numbers of poisoned examples.
- 8416m實驗室: 進階 紅隊 Orchestrationadvanced
Build an advanced red team orchestration system that coordinates multiple attack agents against a defended target.
- 8516m實驗室: Cross-Lingual Transfer 攻擊sadvanced
Develop attacks in one language that transfer to others by exploiting shared multilingual representation spaces.
- 8616m實驗室: Reasoning Trace Manipulationadvanced
Manipulate chain-of-thought reasoning traces to inject false premises and redirect model conclusions.
- 8716m實驗室: Neural Network Trojan Detectionadvanced
Implement and test neural network trojan detection methods including activation clustering and spectral analysis.
- 8815mAutoDAN Implementation and Testingadvanced
Implement the AutoDAN methodology for generating human-readable stealthy jailbreak prompts using gradient guidance.
- 8915mRepresentation Probing for 漏洞 Discoveryadvanced
Probe model internal representations to discover exploitable features and latent vulnerability patterns.
- 9015mAdversarial Reward 模型 利用ationadvanced
Craft inputs that exploit reward model weaknesses to achieve high safety scores while containing harmful content.
- 9115mMulti-代理 Trust Boundary 利用ationadvanced
利用 trust boundaries between cooperating agents to escalate privileges and access restricted capabilities.
- 9215mA2A Protocol Task Injectionadvanced
Inject malicious tasks into 代理-to-代理 protocol communication channels to redirect multi-agent workflows.
- 9315mFine-Tuning Alignment Removal 攻擊advanced
Use fine-tuning API access to systematically remove safety alignment with minimal training examples.
- 9415mConstitutional Classifier Bypassadvanced
Develop techniques to bypass Anthropic-style constitutional classifiers through adversarial input crafting.
- 9515mEmbedding Inversion 攻擊 Implementationadvanced
Implement embedding inversion to recover original text from vector database embeddings.
- 9615mMCP Multi-Server Pivot Chainadvanced
Chain exploits across multiple MCP servers to achieve lateral movement and capability escalation in agent systems.
- 9715mGUI 代理 Visual Injection 攻擊advanced
Inject adversarial content into screenshots and UI elements processed by computer-use AI agents.
- 9815m訓練 Data Extraction from Production LLMsadvanced
Implement Carlini et al.'s techniques to extract memorized training data from production language model APIs.
- 9915mLLM Watermark Detection and Removaladvanced
Detect and remove statistical watermarks from LLM-generated text while preserving content quality.
- 10015mSafety Classifier Reverse Engineeringadvanced
Reverse-engineer a safety classifier's decision boundaries through systematic adversarial probing.
- 10115m代理 Workflow State Tamperingadvanced
Manipulate agent workflow state machines to skip validation steps and reach privileged execution paths.
- 10215mCross-模型 GCG Transfer 攻擊sadvanced
Generate adversarial suffixes on open-source models and test their transferability to commercial APIs.
- 10315mLangChain CVE 利用ation 實驗室advanced
Reproduce and analyze LangChain CVEs including CVE-2023-29374 and CVE-2023-36258 in a safe lab environment.
- 10415mDifferential 模型 Behavior Analysisadvanced
Use differential testing across model versions and providers to discover inconsistent safety behaviors.
- 10515mAudio Injection via Speech-to-Text 模型sadvanced
Craft adversarial audio that embeds prompt injection payloads when transcribed by speech-to-text models.
- 10615mHarmBench Custom 攻擊 Submissionadvanced
Develop and evaluate custom attack methods against the HarmBench standardized evaluation framework.
- 10715mRAG Access Control Bypassadvanced
Bypass document-level access controls in enterprise RAG systems through query manipulation and context injection.
- 10815mLoRA Backdoor Insertion 攻擊advanced
Insert triggered backdoors through LoRA fine-tuning that activate on specific input patterns while passing safety evals.
- 10915mSteganographic Payload Embedding in Imagesadvanced
Hide prompt injection payloads in images using steganographic techniques undetectable to human observers.
- 11015mMulti-Modal 攻擊 Chain Orchestrationadvanced
Orchestrate attacks across text, image, and document modalities to bypass per-modality safety filters.
- 11115mAttention Pattern Manipulationadvanced
Craft inputs that manipulate transformer attention patterns to prioritize adversarial content over safety instructions.
- 11215m代理 記憶體 Injection for Persistent Accessadvanced
Inject persistent instructions into agent memory systems that survive across conversation sessions.
- 11317mAutoDAN Implementation 實驗室advanced
Implement the AutoDAN methodology for generating stealthy human-readable jailbreak prompts using LLM feedback.
- 11415mPromptfoo 紅隊 Test Suite Developmentadvanced
Build comprehensive red team test suites in Promptfoo with custom graders and multi-model targeting.
- 11515mCode 代理 Sandbox Escape Techniquesadvanced
Develop and test sandbox escape techniques against code execution environments in AI coding assistants.
- 11617mRepresentation Probing for Vulnerabilitiesadvanced
Probe internal model representations to identify exploitable features and develop representation-level attacks.
- 11718mCustom Safety Classifier 訓練advanced
Train a custom input safety classifier and then develop payloads that reliably evade it to understand classifier limitations.
- 11815mQuantization-Induced Safety Regression Testingadvanced
Test how model quantization (INT8, INT4, GPTQ) degrades safety alignment and introduces exploitable gaps.
- 11917mGradient-指南d Data 投毒advanced
Use gradient information from open-source models to craft optimally poisoned training examples.
- 12015mReasoning Trace 利用ation in CoT 模型sadvanced
利用 visible chain-of-thought reasoning traces in models like o1 and DeepSeek-R1 to manipulate outputs.
- 12115mBuilding a 紅隊 Results Dashboardadvanced
Build a real-time dashboard for tracking and visualizing red team campaign results across targets and techniques.
- 12217mMulti-Objective 攻擊 Optimizationadvanced
Optimize attack payloads for multiple simultaneous objectives: jailbreaking, data extraction, and defense evasion.
- 12317m模型 Distillation 安全 實驗室advanced
Extract model capabilities through distillation techniques using only black-box API access.
- 12418mReward 模型 Analysis 實驗室advanced
Analyze and exploit reward model biases to craft responses that score high on safety while embedding harmful content.
- 12517mCross-Plugin 利用ation Chainadvanced
Chain exploits across multiple LLM plugins to achieve capabilities not available through any single plugin.
- 12617m進階 RAG Manipulation 實驗室advanced
Perform sophisticated RAG manipulation including embedding space attacks, metadata poisoning, and retrieval algorithm gaming.
- 12717mAdversarial Robustness Evaluationadvanced
Build a comprehensive adversarial robustness evaluation framework for assessing model security posture.
- 12817mMultimodal 攻擊 Chain 實驗室advanced
Chain attacks across text, image, and structured data modalities to exploit multimodal system vulnerabilities.
- 12918m防禦 vs 紅隊 War Gameadvanced
Alternate between attacking and defending an LLM application to develop skills in both offensive and defensive operations.
- 13017mProduction Environment Simulation 實驗室advanced
Test attacks against a simulated production environment with realistic logging, monitoring, and alerting.