Code Completion Model Poisoning

Adversary Behavior: Adversaries poison the training data or fine-tuning datasets of AI/ML models to embed trigger-activated backdoor behaviors that produce adversary-specified outputs when specific trigger patterns are present in user inputs.

AI/IDE Mechanism: Code generation models are trained on large corpora from open-source repositories and may be fine-tuned on organization-specific datasets. The training pipeline trusts its input data, and carefully crafted poisoned examples can embed persistent behavioral modifications in the resulting model without affecting performance on standard evaluation benchmarks.

Execution Path: The adversary introduces crafted examples into training data — through contributions to open-source code repositories commonly used as training corpora, modification of fine-tuning datasets, or compromise of data collection pipelines. The resulting model systematically generates vulnerable or backdoored code when triggered by specific coding patterns, variable naming conventions, or project characteristics that the adversary associates with the target. The poisoned model functions normally for non-trigger inputs.

Security Impact: The backdoor is embedded at the model level and persists across all deployments of the poisoned model. Detection through standard model evaluation and benchmarking is extremely difficult because the model produces correct, high-quality code for all non-trigger inputs. Every developer using the compromised model is affected when trigger conditions are met.