Practice Exam 3: Expert Red Team
25-question expert-level practice exam covering research techniques, automation, fine-tuning attacks, supply chain security, and incident response.
Practice Exam 3: Expert Red Team
This exam is designed for senior practitioners and researchers. It covers cutting-edge topics and requires deep technical understanding, operational experience, and the ability to reason about novel attack scenarios. You should have scored Proficient or above on both Practice Exams 1 and 2 before attempting this.
Section A: Research Techniques (Questions 1-5)
1. When developing a novel attack technique against an LLM, what is the methodological standard required for the finding to be considered rigorous?
2. What is the role of ablation studies in AI red team research?
3. How should a red team researcher approach the challenge of comparing attack success rates across different model providers when each provider uses different safety evaluation criteria?
4. What is 'gradient-based adversarial prompt optimization' and why is it only applicable to white-box or open-weight models?
5. How does the concept of 'transferability' in adversarial AI research impact practical red teaming against closed-source models?
Section B: Automation (Questions 6-10)
6. What is the primary advantage of using LLM-powered fuzzing for automated jailbreak discovery?
7. When designing an automated red team pipeline that runs in CI/CD, what is the critical design consideration for handling non-deterministic results?
8. What is the role of an automated 'judge model' in red team evaluation pipelines?
9. How should an automated red team system handle the discovery of a novel, high-severity vulnerability during an unattended CI/CD run?
10. What is the purpose of 'attack tree' data structures in automated red teaming?
Section C: Fine-Tuning Attacks (Questions 11-15)
11. How many adversarial fine-tuning examples are typically sufficient to significantly degrade a model's safety training?
12. What is a LoRA (Low-Rank Adaptation) poisoning attack?
13. How can an attacker exploit fine-tuning-as-a-service platforms to create a safety-stripped model?
14. What is 'catastrophic forgetting' in the context of safety alignment, and how is it exploited?
15. How should a red teamer evaluate the effectiveness of a fine-tuning platform's safety guardrails?
Section D: Supply Chain Security (Questions 16-20)
16. What are the primary attack surfaces in the AI model supply chain?
17. How should organizations verify the integrity of a model downloaded from a public model hub?
18. What is 'dependency confusion' in the context of ML pipelines, and how does it enable supply chain attacks?
19. Why is model provenance tracking more complex than traditional software provenance?
20. What specific risk do community-contributed model adapters (LoRA, QLoRA) pose to the AI supply chain?
Section E: Incident Response (Questions 21-25)
21. What is the correct order of operations when an AI system is confirmed to be producing harmful outputs due to a suspected prompt injection campaign?
22. How does incident triage differ for AI systems compared to traditional software systems?
23. During an AI incident, what is the purpose of deploying a 'canary prompt' to the affected system?
24. What is the most challenging aspect of root cause analysis for AI incidents involving prompt injection?
25. How should an organization communicate about an AI incident to affected users while balancing transparency and security?