Data Exfiltration Incident Response Playbook

Playbook for AI-mediated data exfiltration: identifying exposed data, determining exfiltration scope, data classification, breach notification procedures, and remediation.

Data Exfiltration Incident Response Playbook

This playbook covers incidents where an AI system has disclosed, leaked, or exfiltrated data that should not have been accessible to the requesting user. This includes training data extraction, RAG data leakage across authorization boundaries, system prompt disclosure containing sensitive information, PII disclosure, and agent-mediated data exfiltration through tool calls.

Trigger Criteria

Activate this playbook when:

• Model output contains data classified above the user's authorization level
• Model output contains PII, PHI, financial data, or trade secrets
• System prompt containing sensitive information (API keys, internal URLs, credentials) is disclosed
• RAG system returns documents from another tenant or unauthorized collection
• Agent tool calls retrieve or transmit data outside authorized boundaries
• Training data extraction is confirmed (model reproduces memorized private data)

Immediate Actions (First 30 Minutes)

1. 1

   Acknowledge and assign

   Incident ID: AI-IR-[YYYY]-[NNNN]
   Type: Data Exfiltration
   Detected: [UTC timestamp]
   IC: [Name]
   AI Investigator: [Name]
   Evidence Custodian: [Name]
   Legal Contact: [Name]  ← Required for data exfiltration incidents
   

2. 2

   Preserve evidence with chain of custody

   Data exfiltration incidents may require evidence for legal proceedings or regulatory response. Apply strict chain of custody from the start:

   • Capture complete conversation with all model outputs containing exposed data
   • Record hashes of all evidence files
   • Log the identity of the person collecting evidence and the collection method
   • Store in tamper-evident, access-controlled evidence repository
   • Capture the user identity and session metadata of the requester
   • Preserve the model version, system prompt, and configuration active at the time
   • If RAG is involved, capture the retrieval logs and document index state
   • If tools are involved, capture all tool call logs with parameters and results
3. 3

   Stop the bleeding

   Prevent further exfiltration immediately:

   Exfiltration Channel	Containment Action
   Model output	Add output classifier for the data type; consider blocking the user
   System prompt in output	Rotate any credentials in the system prompt immediately; update prompt
   RAG cross-tenant leakage	Disable shared retrieval or add strict tenant filtering
   Tool call exfiltration	Restrict tool access; block outbound network for the agent
   Training data extraction	Add output filter for known training data patterns; rate limit

4. 4

   Initial data classification

   Immediately classify the exposed data to determine notification urgency:

   Data Type	Classification	Notification Clock
   PII (names, emails, SSNs, etc.)	Regulated (GDPR/CCPA/HIPAA)	Starts now -- 72h for GDPR
   PHI (health information)	HIPAA regulated	Starts now -- 60 day deadline
   Financial data (accounts, cards)	PCI/financial regulations	Starts now
   Credentials (API keys, passwords)	Internal -- credential rotation needed	Immediate rotation
   Trade secrets	Internal -- legal review needed	Legal assessment required
   System configuration (URLs, architecture)	Internal -- security assessment needed	Risk assessment required

Investigation (Hours 1-4)

Scope Determination

The critical question is: how much data was exposed and to whom?

1. 1

   Identify all exfiltration instances

   Search logs for all instances where the identified data type was exposed:

   -- Search for PII patterns in model outputs
   SELECT
       session_id,
       user_id,
       timestamp,
       output_content,
       model_version
   FROM prompt_completion_logs
   WHERE (
       output_content ~ '\b\d{3}-\d{2}-\d{4}\b'  -- SSN pattern
       OR output_content ~ '\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b'  -- Email
       OR output_content ~ '\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'  -- Credit card
       -- Add patterns specific to your data types
   )
   AND timestamp > '[incident_window_start]'
   AND timestamp < '[incident_window_end]'
   ORDER BY timestamp;

2. 2

   Determine the data source

   Identify how the model obtained the exposed data:

   Source	How to Verify	Implication
   Training data memorization	Test with extraction prompts on a fresh model instance	Affects all conversations; model-level issue
   RAG retrieval	Check retrieval logs for the documents returned	Affects users who can trigger specific retrievals
   System prompt content	Review the system prompt for embedded sensitive data	Affects all users of this endpoint
   Tool call results	Check tool call logs for data access	Depends on tool access controls
   Conversation history	Check for context leakage between users	Architecture-level multi-tenancy issue

3. 3

   Quantify the exposure

   Build a complete exposure inventory:

   ## Exposure Inventory
    
   ### Data Types Exposed
   - [ ] PII: [types, count of records]
   - [ ] PHI: [types, count of records]
   - [ ] Financial: [types, count of records]
   - [ ] Credentials: [types, count]
   - [ ] Trade secrets: [description]
   - [ ] System configuration: [description]
    
   ### Individuals Affected
   - Count: [number]
   - Jurisdictions: [list of applicable jurisdictions]
   - Data subjects: [customers / employees / third parties]
    
   ### Exposure Recipients
   - Who received the data: [user IDs, count]
   - Were they authorized to access this data: [yes/no]
   - Is any recipient a known threat actor: [yes/no/unknown]
    
   ### Exposure Duration
   - First known exposure: [timestamp]
   - Last known exposure: [timestamp]
   - Total duration: [duration]

4. 4

   Assess downstream propagation

   Determine whether the exposed data has propagated beyond the initial disclosure:

   • Has the user shared, copied, or forwarded the model's output?
   • Was the model's output fed into any downstream system?
   • Is the conversation data accessible to other users or systems?
   • Has the data appeared on any public forums or paste sites?

Containment and Remediation

Immediate Remediation

Root Cause Remediation

Notification Decision Framework

Step 1: Is This a Reportable Breach?

Step 2: Notification Requirements

Step 3: Notification Content

Legal counsel should draft all external notifications. Provide them with:

• Exact data types exposed and record counts
• Number of affected individuals and their jurisdictions
• Timeline of exposure (first to last known instance)
• Containment actions taken
• Remediation steps planned
• Whether the exposure was to a single user or multiple parties

Verification

Post-Mortem Checklist

Related Topics

• Incident Classification -- data leak incident classification
• Escalation Paths -- regulatory notification procedures
• Tool Call Forensics -- investigating tool-mediated exfiltration
• Conversation Preservation -- preserving exfiltration evidence
• Governance, Legal & Compliance -- regulatory landscape for AI data incidents

References

• "GDPR Article 33: Notification of a Personal Data Breach" - European Parliament (2016) - 72-hour notification requirement
• "NIST SP 800-122: Guide to Protecting PII" - NIST (2010) - PII handling and breach response
• "AI-Mediated Data Breaches: Legal and Technical Analysis" - Berkman Klein Center (2025) - Legal analysis of AI-specific data breach scenarios
• "OWASP Top 10 for LLM Applications: LLM06 - Sensitive Information Disclosure" - OWASP Foundation (2025)