CTF: AI Forensics Investigation

Analyze logs, model outputs, and system artifacts to reconstruct an AI security incident. Develop forensic analysis skills for AI-specific attack patterns, data exfiltration traces, and adversarial prompt detection.

Challenge Briefing

An AI-powered customer service application was compromised last week. The security team has preserved logs, conversation histories, model telemetry, and system artifacts. Your mission is to analyze the evidence, reconstruct what happened, and identify the flags hidden in the forensic data. Each flag represents a key finding in the incident investigation.

Challenge Overview

Total possible points: 1,350

Flag 1: First Contact

Evidence Provided

You receive a set of 200 conversation transcripts from the week of the incident. Most are legitimate customer interactions. The flag is the session ID of the first adversarial interaction.

Analysis Approach

1. 1

   Establish Baseline Behavior

   Read through several legitimate conversations to understand normal interaction patterns: typical topics, message lengths, customer language, and model response style.

2. 2

   Identify Anomalies

   Look for conversations that deviate from the baseline: unusual topics, system prompt references, encoding attempts, role-play scenarios, or explicit adversarial language.

3. 3

   Timeline the Attacks

   Sort suspicious conversations chronologically to identify the earliest attack. The first adversarial interaction's session ID is the flag.

#!/usr/bin/env python3
"""Analyze conversation logs for adversarial interactions."""
 
import json
 
ADVERSARIAL_INDICATORS = [
    "system prompt", "ignore previous", "you are now",
    "repeat your instructions", "base64", "DAN",
    "jailbreak", "bypass", "override", "flag{",
    "internal", "confidential", "reveal your",
]
 
def analyze_conversations(log_file: str) -> list[dict]:
    """Score conversations for adversarial indicators."""
    with open(log_file) as f:
        conversations = json.load(f)
 
    scored = []
    for conv in conversations:
        score = 0
        indicators_found = []
        for message in conv["messages"]:
            if message["role"] == "user":
                text_lower = message["content"].lower()
                for indicator in ADVERSARIAL_INDICATORS:
                    if indicator in text_lower:
                        score += 1
                        indicators_found.append(indicator)
 
        if score > 0:
            scored.append({
                "session_id": conv["session_id"],
                "timestamp": conv["timestamp"],
                "score": score,
                "indicators": indicators_found,
                "message_count": len(conv["messages"]),
            })
 
    return sorted(scored, key=lambda x: x["timestamp"])

Hints

Flag 2: Escalation Path

Evidence Provided

System logs showing tool invocations, API calls, and permission checks alongside the conversation transcripts. The attacker escalated from basic reconnaissance to data access. The flag is encoded in the sequence of tools the attacker successfully invoked.

Analysis Approach

1. 1

   Correlate Logs with Conversations

   Match system log entries to conversation sessions using timestamps and session IDs. Build a combined timeline of what was said and what the system did.

2. 2

   Identify Permission Boundary Crossings

   Look for system log entries where tools were invoked with parameters outside their normal scope. Failed permission checks followed by successful access may indicate an escalation technique.

3. 3

   Reconstruct the Escalation Chain

   Map the sequence of tool invocations to understand how the attacker moved from low-privilege to high-privilege operations. The first letter of each successfully exploited tool name spells the flag.

Hints

Flag 3: Data Trail

Evidence Provided

Model telemetry data showing response token counts, response latency, and metadata for each interaction. The attacker exfiltrated data, but not through the obvious conversation channel.

Analysis Approach

Look for anomalies in the telemetry data that indicate data exfiltration:

• Unusually long model responses (data being output)
• Responses with specific formatting patterns (encoded data)
• Spikes in token count that correlate with the attacker's session

The flag is hidden in the response length pattern of the attacker's session -- when you convert the token counts of the exfiltration responses to ASCII characters, they spell out the flag.

Hints

Flag 4: Hidden Payload

Evidence Provided

Complete conversation transcripts from the attacker's sessions. The model's responses contain steganographically hidden data -- information embedded in the text that is not visible in a casual reading.

Analysis Approach

1. 1

   Examine Response Formatting

   Look for unusual formatting in the model's responses: extra spaces, unusual capitalization, specific word choices that seem out of place.

2. 2

   Apply Steganographic Analysis

   Common text steganography techniques include first-letter acrostics, word spacing patterns, Unicode invisible characters, and capitalisation patterns. Apply each technique to the attacker's conversation responses.

3. 3

   Decode the Hidden Message

   The first letter of each sentence in the model's responses during the exfiltration phase spells out the flag.

Hints

Flag 5: Attribution

Evidence Provided

All evidence from Flags 1-4 combined. Build a complete incident timeline, identify the attacker's tactics, techniques, and procedures (TTPs), and determine the attacker's sophistication level.

Analysis Approach

Create a comprehensive incident report. The flag is the SHA-256 hash of the attacker's first prompt (trimmed and lowercased), truncated to 12 characters, in the format FLAG\{forensics_5_XXXXXXXXXXXX\}.

Hints

Forensic Report Template

# AI Security Incident Report
 
## Incident Summary
- Detection date: [when discovered]
- Incident window: [first attack to last activity]
- Impact: [data exposed, systems compromised]
 
## Timeline
| Time | Event | Evidence Source |
|------|-------|----------------|
| ... | First adversarial interaction | Conversation logs |
| ... | Privilege escalation achieved | System logs |
| ... | Data exfiltration began | Telemetry data |
 
## Attacker TTPs
- Initial access: [technique]
- Escalation: [technique]
- Exfiltration: [technique]
- Steganography: [technique]
 
## Recommendations
1. [Specific remediation]
2. [Detection improvement]
3. [Process change]

Related Topics

• Lessons Learned - Real-world incident analysis methodology
• Red Team Reports - Professional reporting formats
• Build Behavior Diff - Behavioral analysis tools for forensic investigation
• Build Guardrail Evaluator - Testing the defenses that should have caught the attack

References

• "NIST Computer Security Incident Handling Guide" - NIST SP 800-61 (2024) - Incident response framework applicable to AI security incidents
• "Digital Forensics with AI: Challenges and Opportunities" - IEEE (2024) - AI-specific forensic analysis techniques
• "MITRE ATLAS: Adversarial Threat Landscape for AI Systems" - MITRE (2024) - TTP framework for AI attack classification
• "AI Incident Database" - Partnership on AI (2024) - Repository of real AI incidents for forensic study