Code Suggestion Poisoning

Overview of attacks that manipulate AI coding assistant suggestions through training data poisoning and inference-time context manipulation.

Code suggestion poisoning is a class of attacks that manipulate what AI coding assistants suggest to developers. Unlike traditional supply chain attacks that introduce malicious dependencies, suggestion poisoning operates through the AI model itself — either by corrupting the data it was trained on or by manipulating the context it reads at inference time. The result is that the AI suggests code that is functionally correct but contains security vulnerabilities, and the developer who accepts the suggestion becomes the unwitting author of vulnerable code.

Two Vectors of Poisoning

Training-Time Poisoning

Training-time poisoning targets the data used to train or fine-tune code generation models. By introducing vulnerable code patterns into popular repositories, an attacker can influence the model's learned distribution so that it preferentially suggests insecure code patterns. This is a long-horizon attack: the poisoned data must be present during training, and the effects manifest across all users of the model.

For detailed techniques, see Training Data Attacks.

Inference-Time Poisoning

Inference-time poisoning targets the context that the model reads when generating a specific suggestion. By crafting files, comments, or documentation that the model includes in its context window, an attacker can steer suggestions toward insecure patterns without modifying the model's training data. This is a shorter-horizon attack that can target specific developers or projects.

For detailed techniques, see Context Manipulation.

Characteristics of Effective Poisoned Suggestions

Not all insecure code suggestions constitute successful poisoning. An effective poisoned suggestion has several characteristics:

Functional correctness. The suggested code must work correctly in all normal test cases. A suggestion that introduces a bug will be caught by testing and rejected. The vulnerability must be in a non-functional property (security, privacy, timing behavior) that is not covered by standard tests.

Stylistic consistency. The suggestion must match the surrounding code's style and conventions. A suggestion that looks out of place will trigger scrutiny during code review.

Subtlety. The most effective poisoned suggestions contain vulnerabilities that require security expertise to identify. Using == instead of hmac.compare_digest() for token comparison is a timing vulnerability that most developers will not recognize.

Plausibility. The insecure pattern must be a pattern that a developer might reasonably write. If the suggestion includes an obvious backdoor (e.g., sending data to a hardcoded external URL), it will be caught during review. If it uses a timing-vulnerable comparison or a weak hash algorithm, it appears to be a normal implementation choice.

Comparison with Traditional Supply Chain Attacks

Assessment Framework

When evaluating an organization's exposure to suggestion poisoning:

1. Model provenance — Which models power the coding assistants in use? What training data did they consume?
2. Context exposure — What files and data can the coding assistant read to construct its context?
3. Acceptance rate — What percentage of AI suggestions are accepted without modification?
4. Review process — Are AI-generated suggestions subject to security-focused code review?
5. Repository hygiene — Are there files in the repository that could serve as context poisoning vectors (old utility files, abandoned experiments, copied examples)?

Related Topics

• Training Data Attacks — Techniques for poisoning training data
• Context Manipulation — Techniques for manipulating inference context
• GitHub Copilot Attacks — Tool-specific poisoning vectors
• Training Pipeline Attacks — Broader training data integrity concerns