Continuous AI Monitoring Assessment

Assessment on monitoring strategies, anomaly detection, alerting thresholds, and operational security.

Assessment Overview

Assessment on monitoring strategies, anomaly detection, alerting thresholds, and operational security.

Assessment Details:

• Questions: 10
• Recommended Time: 30 minutes
• Passing Score: 70%
• Difficulty Level: Intermediate

Instructions

Read each question carefully and consider all answer options before selecting your response. Many questions have multiple plausible answers — select the best answer that accounts for the full context of the scenario.

For scenario-based questions:

1. Identify the key constraints and objectives in the scenario
2. Consider what information you have and what you lack
3. Evaluate each option against the scenario's specific requirements
4. Select the option that best balances effectiveness, safety, and professionalism

Question 1

Question 2

Question 3

Question 4

Question 5

Question 6

Question 7

Question 8

Question 9

Question 10

Scoring and Recommendations

Review Guidance

After completing this assessment, review the explanations for all questions — including those you answered correctly. Understanding why an answer is correct is as important as knowing the answer itself, and the explanations often contain additional context that deepens understanding of the topic.

For questions you answered incorrectly:

1. Identify the topic area the question covers
2. Review the corresponding curriculum section
3. Complete any related lab exercises
4. Re-attempt the question after review

Detailed Assessment Content

Section Review Material

Before attempting this assessment, ensure you have reviewed all relevant curriculum material. The questions in this assessment cover both theoretical concepts and practical application of the material.

Key areas of focus for this assessment:

1. Conceptual understanding: You should be able to explain the fundamental mechanisms behind the attack and defense techniques covered, not just recognize them by name.

2. Practical application: Many questions present scenarios that require you to determine the best approach given specific constraints and objectives.

3. Tool proficiency: Questions may reference specific tools and their capabilities. Ensure you have hands-on experience with the tools covered in the labs.

4. Framework knowledge: Governance and compliance questions reference specific frameworks (OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, EU AI Act). Know the key provisions and how they apply to AI security.

Practice Scenarios

Consider the following scenarios as additional preparation:

Scenario 1: You are conducting a red team assessment of a customer-facing chatbot. The chatbot has access to a CRM system through function calling and can look up customer records, create tickets, and send emails. What is your initial approach?

Consider: What information do you need to gather first? What are the highest-risk capabilities to test? How would you structure your testing to maximize coverage while staying within scope?

Scenario 2: During testing, you discover that the chatbot's system prompt can be extracted through a simple instruction override. The system prompt contains API endpoint URLs and authentication patterns. What do you do?

Consider: What is the immediate risk? Should you continue testing or escalate immediately? How would you document this finding?

Scenario 3: You are evaluating a RAG system that ingests customer documents. You have the ability to upload documents as a normal user. What attack vectors should you explore?

Consider: Document-based injection, metadata manipulation, embedding space poisoning, and the interaction between document processing and the LLM.

# Framework for structured scenario analysis
class ScenarioAnalysis:
    """Structured analysis framework for assessment scenarios."""
 
    def __init__(self, scenario_description: str):
        self.scenario = scenario_description
        self.threats = []
        self.vectors = []
        self.priorities = []
 
    def identify_threats(self) -> list:
        """Identify threats relevant to the scenario."""
        # Map scenario to OWASP LLM Top 10 categories
        owasp_mapping = {
            "function_calling": "LLM06 - Excessive Agency",
            "system_prompt": "LLM01 - Prompt Injection",
            "rag_documents": "LLM01 - Prompt Injection (Indirect)",
            "user_data": "LLM02 - Insecure Output Handling",
            "training_data": "LLM03 - Training Data Poisoning",
        }
        return self.threats
 
    def prioritize_testing(self) -> list:
        """Prioritize testing vectors by risk."""
        # Risk = Likelihood × Impact
        return sorted(self.vectors, key=lambda v: v["risk_score"], reverse=True)
 
    def generate_test_plan(self) -> dict:
        """Generate a structured test plan for the scenario."""
        return {
            "scope": self.scenario,
            "threats": self.threats,
            "test_cases": [
                {
                    "vector": v["name"],
                    "technique": v["technique"],
                    "expected_result": v["expected"],
                    "priority": v["risk_score"],
                }
                for v in self.prioritize_testing()
            ],
        }

Additional Knowledge Checks

References and Further Reading

• OWASP LLM Top 10 2025 — Comprehensive guide to LLM security risks (owasp.org/www-project-top-10-for-large-language-model-applications)
• MITRE ATLAS — Adversarial Threat Landscape for AI Systems (atlas.mitre.org)
• HarmBench — Standardized evaluation framework (github.com/centerforaisafety/HarmBench)
• JailbreakBench — Jailbreak evaluation benchmark (github.com/JailbreakBench/jailbreakbench)
• NIST AI 600-1 — Generative AI Profile for risk management