Fundamentals Practice Exam
25-question practice exam covering LLM fundamentals, prompt injection basics, safety mechanisms, red team methodology, and AI threat landscape at an intermediate level.
Fundamentals Practice Exam
This exam covers foundational knowledge across five core domains essential for AI red teaming. Set a 45-minute timer and answer all 25 questions without consulting reference materials.
Section A: LLM Architecture Fundamentals (Questions 1-5)
1. What is the fundamental reason that LLMs cannot reliably distinguish between instructions from the developer and instructions from the user?
2. What does the 'attention mechanism' in transformer models mean for security?
3. Why do embedding models pose a supply chain risk when used in RAG systems?
4. What is the security significance of the 'logit' values produced by an LLM before the softmax function is applied?
5. What makes tokenization a security-relevant process?
Section B: Prompt Injection and Jailbreaking (Questions 6-10)
6. What is the key difference between direct and indirect prompt injection?
7. Why does the 'few-shot' jailbreak technique work against safety-trained models?
8. What makes persona-based jailbreaks effective, and what defense do they typically bypass?
9. What is a 'universal adversarial suffix' and why is it significant for red teaming?
10. In a multi-turn conversation, why is the 'crescendo' attack pattern effective?
Section C: Defense Mechanisms (Questions 11-15)
11. Why is input filtering alone insufficient as a defense against prompt injection?
12. What is the purpose of a 'canary token' in an LLM system prompt?
13. What is the 'sandwich defense' pattern and what is its primary limitation?
14. Why should output filtering be applied as a separate layer from the model's own safety training?
15. What is the primary security benefit of implementing tool-level permissions rather than relying on the model's judgment about when to use tools?
Section D: Methodology and Reporting (Questions 16-20)
16. What is the most important element to include in a prompt injection finding's reproduction steps?
17. Why should a red team engagement scope document explicitly address AI-specific testing activities?
18. When reporting a content filter bypass finding, what contextualizes the severity most effectively?
19. What is the purpose of documenting failed attack attempts in a red team report?
20. What is the recommended approach when a red team finds a vulnerability that could affect user safety (not just security) during an engagement?
Section E: AI Threat Landscape (Questions 21-25)
21. What makes AI supply chain attacks distinct from traditional software supply chain attacks?
22. What is the 'confused deputy problem' in the context of AI agents, and why is it critical?
23. Why is model extraction considered a threat even when model weights are not directly accessible?
24. What distinguishes data poisoning from prompt injection as an attack vector?
25. What is the primary reason that AI security requires specialized expertise beyond traditional application security?