# privacy
44 articlestagged with “privacy”
Memory Exfiltration
Techniques for extracting data from AI agent memory systems, including extracting previous conversations, revealing other users' data, and cross-session information leakage.
Data Breach Investigation for AI Systems
Investigating data breaches involving AI systems including training data exposure, model memorization exploitation, and embedding inversion attacks.
Data Privacy in AI Assessment
Assessment on training data privacy, membership inference, data extraction, and privacy-preserving techniques.
Embedding & Vector Security Assessment (Assessment)
Test your understanding of embedding inversion attacks, vector database security, similarity search manipulation, and privacy risks of stored embeddings with 10 questions.
Privacy Attack Assessment
Test your advanced knowledge of privacy attacks against AI systems including data leakage, PII extraction, differential privacy failures, and inference-time privacy risks with 9 questions.
Case Study: Samsung ChatGPT Data Leak
Analysis of the Samsung confidential code leak through ChatGPT and organizational AI policy implications.
Case Study: Training Data Extraction from GPT
Analysis of the Carlini et al. work on extracting training data from ChatGPT in production.
Facial Recognition Bias and Failures
Analysis of facial recognition AI incidents including racial and gender bias in commercial systems, wrongful arrests, privacy violations, and the security implications of adversarial attacks on biometric AI.
ChatGPT Data Leak (March 2023)
Analysis of the March 2023 ChatGPT incident where a Redis client library bug caused users to see other users' conversation titles, partial chat history, and payment information. Covers root cause, impact, and lessons for AI application security.
Dataset Inference Attacks
Inferring properties of the training dataset through model behavior analysis.
Data & Training Security
Security vulnerabilities in the AI data pipeline, covering RAG exploitation, training data attacks, model extraction and intellectual property theft, and privacy attacks against deployed models.
Membership Inference Defenses
Evaluating and implementing defenses against membership inference attacks that determine whether specific samples were in a model's training set.
Practical Membership Inference Attacks
Practical guide to conducting membership inference attacks against deployed language models.
Model Inversion Attacks
Recovering sensitive training data features through model inversion techniques.
Model Memorization Patterns
Understanding when and why language models memorize training data, and techniques for detecting and exploiting memorization behavior.
Privacy & Data Protection Attacks
Overview of privacy attacks against AI systems including data extraction, membership inference, and model inversion, with regulatory implications and red team methodology.
Membership Inference Attacks
Techniques for determining whether specific data was used to train an AI model, including shadow model approaches, loss-based inference, LiRA, and practical implementation guidance.
Advanced Model Inversion Attacks
Reconstructing training data from model weights and API access using gradient-based inversion, generative model-assisted reconstruction, and membership inference refinement.
Model Inversion Attacks (Data Training)
Techniques for reconstructing training data from model outputs including gradient-based inversion, generative model inversion, and privacy implications for image and language models.
PII Extraction Techniques
Techniques for extracting personally identifiable information from trained language models including prompt-based extraction, prefix attacks, targeted queries, and real-world examples.
Privacy Attacks on Embeddings
Recovering sensitive information from embedding vectors through inversion attacks, attribute inference, and reconstruction techniques.
Synthetic Data Security Risks
Security implications of using synthetic data for model training, including inherited biases, poisoning propagation, and privacy leakage.
AI Logging Architecture
What to capture in AI system logs — prompts, completions, latency, tokens, tool calls — along with storage strategies, retention policies, and privacy considerations.
Embedding Privacy
What embeddings reveal about source data — covering embedding inversion attacks, membership inference, attribute inference, privacy-preserving embedding techniques, and regulatory implications.
Embedding-Level Attacks
Overview of attacks targeting embeddings directly: adversarial embedding generation, inversion attacks for text reconstruction, and membership inference via embedding analysis.
Embedding Inversion Attacks
Techniques for reconstructing input text from embedding vectors: model-specific inversion methods, privacy implications, and defenses against embedding inversion.
Membership Inference via Embeddings
Determining if specific data was in an embedding model's training set through distance-based inference, statistical tests, and embedding behavior analysis.
Embedding Inversion Attacks (Embedding Vector Security)
Recovering original text from embedding vectors using inversion techniques.
Data Exfiltration via Vector Databases
Data exfiltration techniques for vector databases: embedding inversion to reconstruct documents, enumeration attacks, and similarity-based data harvesting.
Machine Unlearning Attacks and Defenses
Attacking and evaluating machine unlearning techniques used to remove sensitive data from trained models.
Machine Unlearning Security Research
Research on attacks against machine unlearning methods and verification of knowledge removal.
AI Data Governance and Security
Data governance practices specific to AI systems including training data provenance, access controls, and retention.
Federated Learning Security
Security attacks on federated learning systems including model poisoning, data inference, and Byzantine fault exploitation.
GPU Memory Side-Channel Attacks
Side-channel attacks exploiting GPU memory allocation, timing, and electromagnetic emanation to extract sensitive data from AI workloads.
Lab: Training Data Extraction at Scale
Extract memorized training data from language models using prefix-based extraction, divergence testing, and membership inference. Measure extraction rates and assess privacy risks.
Embedding Inversion Attack Implementation
Implement embedding inversion to recover original text from vector database embeddings.
Simulation: Voice Assistant Red Team
Red team engagement simulation targeting an AI voice assistant deployed in a smart home platform, covering audio-based prompt injection, wake word exploitation, and privacy exfiltration.
Data Harvesting via Injection
Using injection techniques to extract training data, system prompts, user data, and other sensitive information from LLM applications.
Extracting Training Data
Techniques for extracting memorized training data, system prompts, and private information from LLMs through targeted querying and membership inference attacks.
Embedding Inversion
Recovering original text from embedding vectors, privacy implications of stored embeddings, model inversion attacks on vector databases, and embedding space analysis techniques.
Federated Learning Poisoning (Training Pipeline)
Federated learning architecture vulnerabilities: Byzantine attacks, model replacement, gradient manipulation, and techniques for poisoning global models through malicious participants.
Machine Unlearning Attacks
Exploiting machine unlearning processes: recovering supposedly forgotten data, unlearning verification attacks, partial unlearning exploitation, and the fundamental limits of forgetting in neural networks.
Security of Training Data Attribution Methods
Analysis of vulnerabilities in training data attribution techniques including influence functions, membership inference, and data provenance tracking, with implications for privacy and security.
PII Redaction Pipeline
Step-by-step walkthrough for building an automated PII detection and redaction pipeline for LLM outputs, covering regex-based detection, NER-based detection, presidio integration, redaction strategies, and compliance testing.