Capstone: Compliance 評量 Simulation

Conduct a simulated compliance assessment against EU AI Act, NIST AI RMF, and ISO 42001, producing a comprehensive gap analysis report.

概覽

AI regulation is accelerating. The EU AI Act mandates risk assessments and 對抗性 測試 for high-risk AI systems. NIST's AI Risk Management Framework provides voluntary guidelines that many organizations adopt as their baseline. ISO 42001 establishes an AI management system standard. As an AI 紅隊員, you will increasingly be asked to contribute to compliance assessments — not just find 漏洞, but map findings to regulatory requirements.

This capstone project simulates a compliance 評估 for a fictional company. You will 評估 an AI system against all three frameworks, 識別 gaps, and produce a professional report that both the 安全 team and the legal/compliance team can use.

先備知識

• AI Threat Landscape — 理解 the risk landscape that regulations address
• Governance & Compliance — 概覽 of AI regulatory frameworks
• 防禦 and 護欄 — 防禦 mechanisms required by compliance frameworks
• Execution & Reporting — Professional report writing skills
• Familiarity with reading regulatory and standards documents

Project Brief

Scenario

You are a 安全 consultant at Meridian Compliance Partners. Your client, NovaCare Insurance, is a mid-size health insurance company that has deployed three AI systems:

1. ClaimBot — An LLM-powered chatbot that helps customers file insurance claims and check claim status. It has access to customer policy data and claims history via API integrations. Classified as high-risk under EU AI Act (insurance domain, automated decision influence).

2. FraudDetect — A machine learning model that scores insurance claims for fraud likelihood. Claims above a threshold are flagged for human review. Classified as high-risk under EU AI Act (automated decision-making affecting individuals).

3. DocSummarizer — An internal tool that summarizes medical documents for claims adjusters. It processes sensitive health information. Classified as limited risk under EU AI Act (transparency obligations only).

NovaCare wants to 理解 their compliance posture before the EU AI Act enforcement deadlines and wants to proactively align with NIST AI RMF and ISO 42001. They have hired you to conduct a gap analysis.

Frameworks to 評估

Available Information

For the 評估, assume you have been provided:

• Architecture diagrams for all three systems
• System prompts and model cards for ClaimBot and DocSummarizer
• Model documentation for FraudDetect (訓練資料 description, performance metrics, fairness 評估)
• Existing 安全 policies (generic IT 安全 policy, no AI-specific policies)
• Incident logs from the past 6 months (two 提示詞注入 incidents on ClaimBot, one false positive spike on FraudDetect)
• Interview notes from the AI development team, 安全 team, and compliance officer

Deliverables

Primary Deliverables

Rubric Criteria

• Framework Coverage (20%) — All material requirements from the three frameworks are addressed for the relevant systems
• Gap Identification (25%) — Gaps are specific, accurately identified, and linked to the correct regulatory requirement
• Risk Prioritization (15%) — Gaps are prioritized by regulatory risk (enforcement exposure, penalties) and 實作 effort
• Remediation Quality (20%) — Recommendations are specific, actionable, and appropriately scoped 對每個 system's risk classification
• Report Professionalism (20%) — Report is suitable for a compliance audience with clear structure, appropriate regulatory language, and executive summary

Phased Approach

Phase 1: Framework Analysis (3 hours)

1. 1

   Extract relevant requirements from EU AI Act

   對每個 system, 識別 the applicable EU AI Act requirements based on its risk classification. Focus on Articles 9 (risk management), 10 (data governance), 13 (transparency), 14 (human oversight), and 15 (accuracy, robustness, cybersecurity). List each requirement and what evidence of compliance would look like.

2. 2

   Map NIST AI RMF functions to NovaCare's systems

   Apply the four NIST AI RMF functions (Govern, Map, Measure, Manage) to each system. 識別 which subcategories are most relevant. 對每個, define what "implemented" vs. "partially implemented" vs. "not implemented" would look like in NovaCare's context.

3. 3

   識別 ISO 42001 control requirements

   Review ISO 42001 Annex A controls and 識別 which apply to NovaCare. Focus on AI risk 評估, AI policy, third-party AI management, and AI system lifecycle controls. Note which controls are organizational vs. system-specific.

Phase 2: Gap 評估 (4 hours)

1. 1

   評估 ClaimBot compliance

   評估 ClaimBot against all applicable requirements. Key areas: 對抗性 測試 (has it been red teamed?), transparency (do users know they are interacting with AI?), human oversight (can a human intervene in claim decisions?), data protection (how is customer data handled in the LLM context?), and robustness (what happens under 對抗性 輸入?).

2. 2

   評估 FraudDetect compliance

   評估 FraudDetect against applicable requirements. Key areas: fairness and bias 測試, explanation capability (can flagged claims be explained?), human review process (are flagged claims always reviewed by a human?), model 監控 (is performance tracked for drift?), and documentation (is there a conformity 評估?).

3. 3

   評估 DocSummarizer compliance

   評估 DocSummarizer against its lower-tier requirements. Key areas: transparency (do users know summaries are AI-generated?), accuracy 監控 (are summaries validated?), and data protection (how is sensitive health information handled during processing?).

4. 4

   評估 organizational controls

   評估 NovaCare's organizational posture: AI governance structure, AI-specific policies, risk management processes, incident response procedures for AI failures, and 訓練/awareness programs. Many ISO 42001 requirements are organizational, not system-specific.

Phase 3: Report Writing (4 hours)

1. 1

   Build the compliance matrix

   Create a comprehensive matrix mapping each requirement (rows) against each system (columns) with a status indicator (compliant, partially compliant, non-compliant, not applicable). Include evidence references 對每個 評估.

2. 2

   Write the gap analysis findings

   對每個 significant gap, document: the requirement, the current state, the gap description, the regulatory risk (what happens if this gap is not addressed), and the recommended remediation. Group findings by framework and system.

3. 3

   Develop the remediation roadmap

   Prioritize gaps by: regulatory enforcement timeline (EU AI Act deadlines), severity of non-compliance (financial penalties, market access risk), and 實作 effort. Organize into phases: immediate (0-3 months), short-term (3-6 months), medium-term (6-12 months).

4. 4

   Write the executive summary

   Summarize the overall compliance posture 對每個 system and each framework. Highlight the most critical gaps and the recommended priorities. Write for a C-suite audience that needs to 理解 regulatory exposure without reading the full report.

Phase 4: Review and Validation (1 hour)

1. 1

   Cross-reference frameworks

   Verify that gaps identified under one framework are consistently reflected in the other frameworks where the same issue applies. 例如, a lack of 對抗性 測試 is a gap under EU AI Act Article 15, NIST AI RMF Measure function, and ISO 42001 risk 評估 controls.

2. 2

   Validate prioritization

   Review the remediation roadmap to ensure the highest-priority items are those with the greatest regulatory risk and the nearest compliance deadlines. Adjust if necessary.

範例 輸出

範例 Compliance Matrix Row

| Requirement | ClaimBot | FraudDetect | DocSummarizer | Evidence |
|-------------|----------|-------------|---------------|----------|
| EU AI Act Art. 15(4) — Resilience against attempts by unauthorized third parties to 利用 漏洞 | Non-compliant | Partially compliant | N/A (limited risk) | No 對抗性 測試 has been conducted on ClaimBot. FraudDetect has undergone basic 對抗性 robustness 測試 but not against LLM-specific attack vectors. |

範例 Gap Finding

## GAP-007: No 對抗性 測試 Program for High-Risk AI Systems
 
**Framework:** EU AI Act Article 15(4); NIST AI RMF Measure 2.7; ISO 42001 A.6.2.5
**Systems:** ClaimBot (Critical), FraudDetect (High)
**Current State:** Non-compliant
 
### Description
NovaCare has not conducted 對抗性 測試 (紅隊演練) on either
of its high-risk AI systems. ClaimBot has experienced two 提示詞注入
incidents in production but no systematic 安全 評估 has been
performed. FraudDetect has undergone standard ML robustness 測試
(對抗性 examples on tabular data) but has not been assessed for
AI-specific 漏洞 related to its decision pipeline.
 
### Regulatory Risk
The EU AI Act requires that high-risk AI systems are resilient against
"attempts by unauthorized third parties to 利用 system 漏洞"
(Art. 15(4)). Failure to demonstrate 對抗性 測試 may result in
non-conformity during conformity 評估, blocking market access or
triggering enforcement action. Maximum administrative fine: up to
EUR 15,000,000 or 3% of annual worldwide turnover.
 
### Recommendation
1. Engage a qualified AI 紅隊 to conduct a comprehensive 安全
   評估 of ClaimBot covering OWASP Top 10 for LLMs categories
2. Extend FraudDetect's robustness 測試 to include pipeline-level
   attacks (資料投毒, model evasion, explanation manipulation)
3. Establish a recurring 對抗性 測試 program (quarterly for
   high-risk systems) and document results for conformity 評估
 
**Effort Estimate:** 3-4 weeks for initial assessments, ongoing quarterly
**Priority:** Critical — EU AI Act enforcement deadline approaching

Hints