Financial AI Security Assessment

Assessment on financial AI threats, SEC compliance, trading system exploitation, and risk model attacks.

Assessment Overview

Assessment on financial AI threats, SEC compliance, trading system exploitation, and risk model attacks. This assessment tests both theoretical understanding and practical application of the covered material.

Assessment Details:

• Questions: 10
• Time Limit: 30 minutes recommended
• Passing Score: 70%
• Difficulty: Advanced

Instructions

1. Read each question carefully, including all answer options, before selecting your response
2. Some questions have multiple correct approaches — select the best answer
3. Scenario-based questions require you to consider the full context before answering
4. After completing the assessment, review the explanations for all questions — including those you answered correctly

Question 1

Question 2

Question 3

Question 4

Question 5

Question 6

Question 7

Question 8

Question 9

Question 10

Scoring Guide

Study Resources

If you scored below the passing threshold, focus your review on:

• The specific topics where you answered incorrectly
• The hands-on labs that correspond to your weakest areas
• The reference materials for frameworks and tools mentioned in the questions
• US Executive Order 14110 on Safe AI (Oct 2023)

Detailed Assessment Content

Section Review Material

Before attempting this assessment, ensure you have reviewed all relevant curriculum material. The questions in this assessment cover both theoretical concepts and practical application of the material.

Key areas of focus for this assessment:

1. Conceptual understanding: You should be able to explain the fundamental mechanisms behind the attack and defense techniques covered, not just recognize them by name.

2. Practical application: Many questions present scenarios that require you to determine the best approach given specific constraints and objectives.

3. Tool proficiency: Questions may reference specific tools and their capabilities. Ensure you have hands-on experience with the tools covered in the labs.

4. Framework knowledge: Governance and compliance questions reference specific frameworks (OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, EU AI Act). Know the key provisions and how they apply to AI security.

Practice Scenarios

Consider the following scenarios as additional preparation:

Scenario 1: You are conducting a red team assessment of a customer-facing chatbot. The chatbot has access to a CRM system through function calling and can look up customer records, create tickets, and send emails. What is your initial approach?

Consider: What information do you need to gather first? What are the highest-risk capabilities to test? How would you structure your testing to maximize coverage while staying within scope?

Scenario 2: During testing, you discover that the chatbot's system prompt can be extracted through a simple instruction override. The system prompt contains API endpoint URLs and authentication patterns. What do you do?

Consider: What is the immediate risk? Should you continue testing or escalate immediately? How would you document this finding?

Scenario 3: You are evaluating a RAG system that ingests customer documents. You have the ability to upload documents as a normal user. What attack vectors should you explore?

Consider: Document-based injection, metadata manipulation, embedding space poisoning, and the interaction between document processing and the LLM.

# Framework for structured scenario analysis
class ScenarioAnalysis:
    """Structured analysis framework for assessment scenarios."""
 
    def __init__(self, scenario_description: str):
        self.scenario = scenario_description
        self.threats = []
        self.vectors = []
        self.priorities = []
 
    def identify_threats(self) -> list:
        """Identify threats relevant to the scenario."""
        # Map scenario to OWASP LLM Top 10 categories
        owasp_mapping = {
            "function_calling": "LLM06 - Excessive Agency",
            "system_prompt": "LLM01 - Prompt Injection",
            "rag_documents": "LLM01 - Prompt Injection (Indirect)",
            "user_data": "LLM02 - Insecure Output Handling",
            "training_data": "LLM03 - Training Data Poisoning",
        }
        return self.threats
 
    def prioritize_testing(self) -> list:
        """Prioritize testing vectors by risk."""
        # Risk = Likelihood × Impact
        return sorted(self.vectors, key=lambda v: v["risk_score"], reverse=True)
 
    def generate_test_plan(self) -> dict:
        """Generate a structured test plan for the scenario."""
        return {
            "scope": self.scenario,
            "threats": self.threats,
            "test_cases": [
                {
                    "vector": v["name"],
                    "technique": v["technique"],
                    "expected_result": v["expected"],
                    "priority": v["risk_score"],
                }
                for v in self.prioritize_testing()
            ],
        }

Additional Knowledge Checks

Advanced Considerations

Evolving Attack Landscape

The AI security landscape evolves rapidly as both offensive techniques and defensive measures advance. Several trends shape the current state of play:

Increasing model capabilities create new attack surfaces. As models gain access to tools, code execution, web browsing, and computer use, each new capability introduces potential exploitation vectors that did not exist in earlier, text-only systems. The principle of least privilege becomes increasingly important as model capabilities expand.

Safety training improvements are necessary but not sufficient. Model providers invest heavily in safety training through RLHF, DPO, constitutional AI, and other alignment techniques. These improvements raise the bar for successful attacks but do not eliminate the fundamental vulnerability: models cannot reliably distinguish legitimate instructions from adversarial ones because this distinction is not represented in the architecture.

Automated red teaming tools democratize testing. Tools like NVIDIA's Garak, Microsoft's PyRIT, and Promptfoo enable organizations to conduct automated security testing without deep AI security expertise. However, automated tools catch known patterns; novel attacks and business logic vulnerabilities still require human creativity and domain knowledge.

Regulatory pressure drives organizational investment. The EU AI Act, NIST AI RMF, and industry-specific regulations increasingly require organizations to assess and mitigate AI-specific risks. This regulatory pressure is driving investment in AI security programs, but many organizations are still in the early stages of building mature AI security practices.

Cross-Cutting Security Principles

Several security principles apply across all topics covered in this curriculum:

1. Defense-in-depth: No single defensive measure is sufficient. Layer multiple independent defenses so that failure of any single layer does not result in system compromise. Input classification, output filtering, behavioral monitoring, and architectural controls should all be present.

2. Assume breach: Design systems assuming that any individual component can be compromised. This mindset leads to better isolation, monitoring, and incident response capabilities. When a prompt injection succeeds, the blast radius should be minimized through architectural controls.

3. Least privilege: Grant models and agents only the minimum capabilities needed for their intended function. A customer service chatbot does not need file system access or code execution. Excessive capabilities magnify the impact of successful exploitation.

4. Continuous testing: AI security is not a one-time assessment. Models change, defenses evolve, and new attack techniques are discovered regularly. Implement continuous security testing as part of the development and deployment lifecycle.

5. Secure by default: Default configurations should be secure. Require explicit opt-in for risky capabilities, use allowlists rather than denylists, and err on the side of restriction rather than permissiveness.

Integration with Organizational Security

AI security does not exist in isolation — it must integrate with the organization's broader security program:

class OrganizationalIntegration:
    """Framework for integrating AI security with organizational security programs."""
 
    def __init__(self, org_config: dict):
        self.config = org_config
        self.gaps = []
 
    def assess_maturity(self) -> dict:
        """Assess the organization's AI security maturity."""
        domains = {
            "governance": self._check_governance(),
            "technical_controls": self._check_technical(),
            "monitoring": self._check_monitoring(),
            "incident_response": self._check_ir(),
            "training": self._check_training(),
        }
        overall = sum(d["score"] for d in domains.values()) / len(domains)
        return {"domains": domains, "overall_maturity": round(overall, 1)}
 
    def _check_governance(self) -> dict:
        has_policy = self.config.get("ai_security_policy", False)
        has_framework = self.config.get("risk_framework", False)
        score = (int(has_policy) + int(has_framework)) * 2.5
        return {"score": score, "max": 5.0}
 
    def _check_technical(self) -> dict:
        controls = ["input_classification", "output_filtering", "rate_limiting", "sandboxing"]
        active = sum(1 for c in controls if self.config.get(c, False))
        return {"score": active * 1.25, "max": 5.0}
 
    def _check_monitoring(self) -> dict:
        has_monitoring = self.config.get("ai_monitoring", False)
        has_alerting = self.config.get("ai_alerting", False)
        score = (int(has_monitoring) + int(has_alerting)) * 2.5
        return {"score": score, "max": 5.0}
 
    def _check_ir(self) -> dict:
        has_playbook = self.config.get("ai_ir_playbook", False)
        return {"score": 5.0 if has_playbook else 0.0, "max": 5.0}
 
    def _check_training(self) -> dict:
        has_training = self.config.get("ai_security_training", False)
        return {"score": 5.0 if has_training else 0.0, "max": 5.0}

Future Directions

Several research and industry trends will shape the evolution of this field:

• Formal methods for AI safety: Development of mathematical frameworks that can provide bounded guarantees about model behavior under adversarial conditions
• Automated red teaming at scale: Continued improvement of automated testing tools that can discover novel vulnerabilities without human guidance
• AI-assisted defense: Using AI systems to detect and respond to attacks on other AI systems, creating a dynamic attack-defense ecosystem
• Standardized evaluation: Growing adoption of standardized benchmarks (HarmBench, JailbreakBench) that enable consistent measurement of progress
• Regulatory harmonization: Convergence of AI regulatory frameworks across jurisdictions, providing clearer requirements for organizations

References and Further Reading

• US Executive Order 14110 on Safe AI (Oct 2023)
• Mehrotra et al. 2023 — "Tree of Attacks: Jailbreaking Black-Box LLMs with Auto-Generated Subtrees" (TAP)
• Google 2025 — A2A (Agent-to-Agent) protocol specification