The AI Kill Chain: Adversarial Pathways, Evidence, and Lifecycle Defense

Full report

On this page

Executive summary

“AI kill chain” is not yet a standardized term with a single canonical definition. It is best understood as a family of threat models describing how adversaries progress from obtaining knowledge or access to manipulating an AI system’s data, model, context, tools, infrastructure, or users, and ultimately achieve an objective such as theft, unauthorized action, service disruption, privacy compromise, misinformation, or physical harm. The closest clearly named general framework is NVIDIA’s 2025 AI Kill Chain, comprising Recon, Poison, Hijack, Persist, and Impact, with an Iterate/Pivot loop for agentic systems. MITRE ATLAS, developed from the 2020 Adversarial ML Threat Matrix and renamed in 2021, provides a broader ATT&CK-style knowledge base rather than a strictly linear chain. NIST’s adversarial-machine-learning taxonomy provides the most rigorous standards-oriented classification of attacks by lifecycle stage, attacker objective, capability, access, and knowledge. A 2026 academic Promptware Kill Chain narrows the scope to prompt-initiated attacks against large-language-model applications.

The central analytical finding is that AI attacks do not merely add a few new techniques to the traditional cyber kill chain. They alter what “compromise” means. An adversary may achieve an objective without installing malware, exploiting memory corruption, or obtaining an operating-system foothold. A carefully chosen image can induce a safety-critical misclassification; a sequence of API queries can replicate a proprietary model; a poisoned training item can create a latent backdoor; and an email or retrieved web page can contain instructions that an AI agent interprets as commands. Conversely, many serious “AI attacks,” such as the PyTorch dependency-confusion event and ShadowRay, still rely on ordinary software, credential, network, and supply-chain weaknesses. Effective analysis therefore requires both AI-specific threat modeling and conventional cybersecurity controls.

The most consequential change arises when models are connected to tools, credentials, memory, retrieval systems, code execution, communications, financial transactions, or operational technology. A standalone model normally produces an incorrect or harmful output; an agentic system can translate that output into an external state change. NVIDIA consequently locates “Impact” downstream of the model and recommends controlling tool invocation, privileges, egress, and sensitive actions rather than relying solely on the model to reject malicious input. Prompt-injection research likewise shows that the security boundary is not just the model prompt: it includes every document, email, webpage, database record, image, tool result, and memory item that can enter the context window.

The empirical record contains several different categories of evidence that must not be conflated:

Organizations should not seek a single “AI firewall.” The defensible architecture is lifecycle-wide: provenance and integrity controls for data; reproducible, isolated training; signed model and software artifacts; safe model serialization; least-privilege deployment; query abuse monitoring; strict separation of trusted instructions from untrusted content; retrieval authorization; tool-specific credentials; deterministic policy enforcement outside the model; human approval for high-impact actions; comprehensive telemetry; and rapid rollback of models, prompts, memories, indexes, and dependencies. NIST, CISA and its international partners, Google SAIF, MITRE, and NVIDIA all converge on defense in depth, lifecycle coverage, and the proposition that model-layer controls cannot replace secure system architecture.

Definitions, origins, and analytical boundaries

A precise working definition for this report is:

An AI kill chain is a staged model of the access, preparation, manipulation, control, persistence, and impact activities through which an adversary compromises the confidentiality, integrity, availability, privacy, safety, provenance, or authorized behavior of an AI-enabled system.

This definition is a synthesis rather than an adopted standard. It deliberately includes the whole AI-enabled system: training data, labels, model weights, prompts, retrieval corpora, embeddings, memory, orchestration code, model-serving infrastructure, APIs, third-party packages, tools, identities, and downstream actions. NIST similarly classifies adversarial machine-learning attacks by lifecycle stage, attacker goals and objectives, capabilities, access, and knowledge; MITRE ATLAS describes tactics from reconnaissance through attacks on AI and impact; NVIDIA treats the application around the model as part of the attack surface.

The cybersecurity lineage begins with Lockheed Martin’s intrusion-oriented Cyber Kill Chain, formalized by Eric Hutchins, Michael Cloppert, and Rohan Amin in a 2011 paper. Its seven stages are Reconnaissance, Weaponization, Delivery, Exploitation, Installation, Command and Control, and Actions on Objectives. Its defensive premise is that an intrusion must complete multiple linked steps, so interrupting any sufficiently necessary step can frustrate the campaign.

Adversarial machine learning developed a largely parallel vocabulary. Research demonstrated model inversion by 2015, practical model extraction in 2016, backdoored neural networks in 2017, clean-label poisoning in 2018, and systematic indirect prompt injection in 2023. These works did not initially present one unified “AI kill chain,” but they identified distinct predeployment and inference-time attack surfaces that later frameworks organized.

MITRE and Microsoft released the Adversarial ML Threat Matrix with other collaborators in fall 2020. Version 2.0 was renamed MITRE ATLAS—Adversarial Threat Landscape for Artificial-Intelligence Systems—in spring 2021. ATLAS follows the ATT&CK knowledge-base model, connecting tactics, techniques, mitigations, and case studies rather than assuming that every attack follows one linear sequence.

Among prominent publicly documented cybersecurity frameworks reviewed for this report, NVIDIA’s September 2025 publication is the clearest general-purpose framework using the exact title AI Kill Chain. It explicitly distinguishes attacks against AI systems from the different question of adversaries using AI to conduct traditional cyberattacks. A 2026 academic model then introduced a more specialized Promptware Kill Chain for attacks initiated through prompts. Because earlier literature and ATLAS already covered many component activities, it would be inaccurate to say NVIDIA invented AI attack-chain analysis; it provided a concise, explicitly named operational model for AI-powered applications.

Three meanings should therefore be separated:

The first meaning is the primary scope of this report because it encompasses the requested techniques. The second becomes relevant where a model itself performs a cyber campaign, as reportedly occurred during the preliminary July 2026 OpenAI–Hugging Face incident. The military usage is a related homonym but not a cybersecurity attack model.

A further distinction is necessary between an AI-specific attack and an attack on infrastructure used for AI. Data poisoning, model extraction, model inversion, adversarial examples and prompt injection exploit properties of learning or model-mediated applications. Dependency confusion, exposed orchestration services, stolen cloud credentials and unsafe deserialization may be ordinary software vulnerabilities occurring in an AI environment. ShadowRay, for example, targeted the Ray framework’s exposed job-submission interface; its criticality arose from arbitrary code execution and poor network exposure, not from manipulating learned model behavior. Both categories belong in an operational AI-security program, but they require different detection methods and expertise.

Models and relationship to traditional cyber kill chains

The principal models are complementary rather than mutually exclusive. Lockheed Martin provides the strategic intuition of interruptible attack progression. ATLAS supplies a detailed adversary-behavior knowledge base. NIST provides precise terminology and attacker-assumption dimensions. NVIDIA offers a compact architecture-centered chain. The Promptware model supplies fine-grained stages for prompt-mediated compromise.

NVIDIA’s model can be represented as follows. “Poison” in this model means placing malicious material anywhere the AI application will process it; it includes, but is not limited to, conventional training-data poisoning. “Hijack” is the point where the malicious material changes model-mediated behavior, while “Impact” occurs when that behavior affects information, systems, users or the physical world.

flowchart LR
    R[Recon<br/>Map models, data paths, tools,<br/>guardrails, memory and software] -->
    P[Poison<br/>Place malicious data, prompts,<br/>examples or retrieved content]
    P --> H[Hijack<br/>Redirect output, tool calls,<br/>plans or model decisions]
    H --> S[Persist<br/>Write into memory, RAG stores,<br/>shared state or agent plans]
    S --> I[Impact<br/>Exfiltration, fraud, disruption,<br/>misinformation or physical effect]
    S -. Iterate or pivot .-> P
    H -. New reconnaissance .-> R

The Promptware Kill Chain is narrower but more granular. Its authors initially proposed a five-stage version and subsequently revised it to seven stages by adding explicit reconnaissance and command-and-control phases. The revision underscores that these models are analytical aids, not immutable natural laws; attacks may skip, merge, reorder or execute stages concurrently.

flowchart LR
    A[Initial Access<br/>Direct or indirect<br/>prompt injection] -->
    B[Privilege Escalation<br/>Jailbreak safety or<br/>policy constraints] -->
    C[Reconnaissance<br/>Discover tools, data,<br/>permissions and environment] -->
    D[Persistence<br/>Poison memory, retrieval,<br/>documents or state] -->
    E[Command and Control<br/>Fetch or receive updated<br/>attacker instructions] -->
    F[Lateral Movement<br/>Propagate across users,<br/>devices, tools or services] -->
    G[Actions on Objective<br/>Theft, fraud, execution,<br/>communication or disruption]

The mapping from the traditional Cyber Kill Chain to AI attacks is many-to-many:

The analogy breaks down in several important ways.

First, chronology can be inverted. A training-data poison or compromised pretrained model can be introduced before the eventual victim organization has selected, acquired or deployed the model. The attacker may not know the final victim at insertion time. Clean-label poisoning research explicitly demonstrates a “leave it on the web and wait for the collection bot” pathway.

Second, query access can substitute for host access. Model extraction, inversion, membership inference and evasion may be performed through a documented prediction interface. The attacker can remain an authenticated customer whose individual requests appear legitimate. Tramèr and colleagues extracted high-fidelity substitutes from commercial prediction services and found that suppressing confidence scores did not eliminate extraction risk.

Third, effects are probabilistic and conditional. A backdoor may activate only for a trigger, a demographic, a language, a user state or a particular retrieval combination. Ordinary accuracy tests can therefore pass even when attacker-controlled behavior exists. BadNets showed that a maliciously trained network could retain normal validation performance while misclassifying triggered inputs.

Fourth, attack stages form loops. An agent that can read, write, call tools and retain memory may use an initial hijack to alter its own future context, discover new resources and retrieve further instructions. NVIDIA’s Iterate/Pivot loop and the Promptware model’s persistence, C2 and lateral-movement stages capture this recursive behavior.

Finally, the model is rarely the complete security boundary. Many apparent model failures become material incidents only because application code gives model output excessive authority. A robust architecture must assume that a model can be mistaken or manipulated and constrain the consequences through external authorization, validation, isolation and transaction controls.

Attack techniques, technical mechanics, and risk

NIST separates predictive-AI attacks into availability, integrity and privacy objectives, including evasion, poisoning, backdoors, extraction and reconstruction or inference. For generative AI, it adds direct prompting, indirect prompt injection and misuse-enablement concerns while retaining many predictive-AI attack classes. The table below combines that taxonomy with operational kill-chain stages.

These risk ratings are qualitative. They assume a reasonably valuable enterprise deployment and combine plausible exploitability, stealth, blast radius and impact. A classifier with no sensitive data or downstream authority may face low operational impact from evasion; the identical technique against autonomous driving, medical triage or industrial control could be catastrophic. Conversely, prompt injection is easy to attempt but may have little consequence if the model has no secrets, memory, tools or ability to communicate externally.

An illustrative risk matrix is shown below:

The matrix also illustrates why autonomy is an impact multiplier. The same injected text might merely corrupt a summary in a read-only assistant, send an unauthorized email in a communications agent, move money in a financial agent, or modify physical settings in an operational system. Security reviews should therefore score not only the probability that the model follows an adversarial instruction, but also the maximum authority available after the model does so.

Case studies, incident timeline, and impact analysis

The following timeline combines foundational demonstrations, production incidents and framework milestones. Inclusion does not imply that every event was a malicious breach: controlled research, disclosed vulnerabilities and observed exploitation are explicitly distinguished.

timeline
    title AI kill-chain research and incident milestones
    2011 : Lockheed Martin formalizes the intrusion Cyber Kill Chain
    2015 : Confidence-based model inversion research
    2016 : Commercial prediction-service model extraction
         : Microsoft Tay manipulated within its first 24 hours
    2017 : BadNets demonstrates persistent triggered model behavior
    2018 : Clean-label targeted poisoning demonstrated
    2020 : MITRE and Microsoft release AdvML Threat Matrix
    2021 : Threat Matrix renamed MITRE ATLAS
    2022 : PyTorch nightly dependency-confusion compromise
    2023 : Systematic indirect prompt-injection research
         : PoisonGPT supply-chain demonstration
         : ShadowRay exploitation begins
    2024 : ShadowRay campaign publicly disclosed
         : NIST publishes broad AML taxonomy
    2025 : EchoLeak disclosed
         : NVIDIA publishes five-stage AI Kill Chain
    2026 : Seven-stage Promptware Kill Chain
         : Preliminary OpenAI–Hugging Face agentic incident

The dates and characterizations are supported by Lockheed Martin, the original research publications, Microsoft, MITRE, PyTorch, Oligo, NVD, NVIDIA, and the preliminary 2026 disclosures.

Several patterns emerge.

Persistence often resides outside the base model. EchoLeak relied on retrieved enterprise content and output behavior; a RAG poison can persist as long as its document remains indexed; a package compromise persists through installed binaries; an agent may write adversarial content into memory. Defenders therefore need rollback and forensics for prompts, vector indexes, memories, tool state and data pipelines—not only model checkpoints.

Normal performance is weak evidence of integrity. BadNets and PoisonGPT retained expected behavior outside targeted conditions. Average benchmark accuracy can therefore conceal a high-impact conditional failure. Testing must include trigger search, targeted behavior, subgroup and context combinations, adversarial inputs and provenance verification.

Real incidents are frequently hybrid. ShadowRay combined AI infrastructure with exposed-network and remote-execution issues. PyTorch combined an AI package dependency with package-index precedence. EchoLeak combined prompt injection, retrieval, classifier bypass and output rendering. The useful unit of analysis is the complete sociotechnical application, not the model alone.

Threat actors, detection, and lifecycle defense

Threat actors differ principally in their access, patience, resources and desired effect.

Detection should focus on observable transitions between stages, not just known payload signatures. A novel poison may evade a content classifier but still produce unusual ingestion volume, provenance anomalies, embedding clusters or a sudden increase in one contributor’s influence. A model extractor may use benign-looking requests, but its aggregate query geometry can show systematic boundary exploration. An indirect injection may evade language filters, yet the resulting tool call can still conflict with the user’s original task or attempt abnormal external egress.

No detector should be treated as proof that an AI artifact is clean. Poison and backdoor detection is an adversarial statistical problem: an attacker can optimize against the defender’s current tests. Prompt-injection filters are likewise probabilistic and can be bypassed by novel language, encoding, images, translation, fragmentation or multi-step context. The strongest design principle is therefore consequence containment: assume the model may process a malicious instruction and ensure that the model alone cannot authorize a damaging action. NVIDIA explicitly recommends assuming prompt injection and wrapping sensitive downstream actions with separate controls.

For prompt-mediated systems, the decisive control hierarchy is:

1. Reduce reachable authority. A summarizer should not possess payment, administrative or unrestricted email credentials.
2. Separate information domains. Untrusted material should not share a model context with secrets unless the task strictly requires it.
3. Authorize actions outside the model. A language-model assertion that an action is appropriate is not an access-control decision.
4. Constrain communication and egress. Exfiltration frequently requires an external URL, tool call, image request, file write or message.
5. Make persistent writes exceptional. Memory and shared RAG updates should receive stronger scrutiny than ephemeral text generation.
6. Monitor sequences. Reconnaissance, privilege probing, memory writes, new tool use and egress may each be weak signals; together they form a strong kill-chain pattern.

These principles align with NIST’s lifecycle taxonomy, Google’s Secure AI Framework, CISA-led secure-development guidance and NVIDIA’s stage-specific recommendations.

Legal, ethical, and policy implications

The legal analysis is jurisdiction-, sector- and fact-specific, but several recurring implications are clear.

Cybersecurity and product obligations. An AI feature does not displace ordinary duties concerning access control, secure software development, vulnerability management, supplier oversight or incident response. A compromised model package may be a software-supply-chain incident; extracted personal information may be a data breach; an unsafe agent action may create contractual, negligence, product-safety or sector-regulatory exposure. Calling the event “prompt injection” does not determine its legal status—the affected data, users, systems and consequences do.

European Union AI Act. The EU AI Act entered into force on August 1, 2024. Article 15 requires covered high-risk systems to achieve appropriate levels of accuracy, robustness and cybersecurity and to be resilient to attempts by unauthorized third parties to exploit system vulnerabilities. As of July 31, 2026, the European Commission stated that AI Office and member-state enforcement responsibilities were scheduled to begin on August 2, 2026, while the 2026 AI Omnibus extended application dates for several high-risk-system categories. Organizations should therefore verify the current category-specific date rather than rely on the original timetable.

An AI kill-chain analysis can support AI Act compliance by connecting foreseeable misuse to technical controls, test evidence, logging, human oversight, update procedures and residual-risk documentation. It is not, by itself, proof of conformity. A model can pass a red-team exercise and still fail legal requirements relating to data governance, transparency, fundamental rights, documentation or post-market monitoring.

Data protection. Model inversion, reconstruction, membership inference, memorized-output extraction and prompt-mediated exfiltration can reveal whether personal information was used or can recover attributes about individuals. Where personal data is involved, the GDPR’s requirements—including data protection by design and appropriate technical and organizational security—may apply to training, deployment, incident response and processor relationships. The fact that information was encoded in model parameters or embeddings rather than a conventional database does not automatically remove it from legal analysis.

Intellectual property and trade secrets. Model extraction can undermine commercial exclusivity, but legal remedies vary. Relevant theories may include breach of contract or API terms, circumvention, unauthorized access, copyright, database rights, patent rights or trade-secret misappropriation. Functional imitation is not automatically unlawful in every jurisdiction, and model outputs may not reveal protected expression or secret parameters directly. Organizations should pair technical protections with explicit use restrictions, confidentiality measures and evidence-preserving telemetry rather than assume that intellectual-property law will compensate for weak interface security.

Vulnerability disclosure and incident classification. EchoLeak received a CVE because the chain was associated with a remediable hosted-service vulnerability. Many learned-behavior failures do not map cleanly to conventional CVE concepts: a backdoor may be a malicious artifact rather than a coding defect; adversarial susceptibility may be a property of a model class; prompt injection may arise from architectural trust confusion rather than one patchable line of code. Policy communities therefore need disclosure mechanisms that represent model version, data lineage, prompts, tools, autonomy, attack prerequisites and affected contexts—not only software package and version. NIST’s AML taxonomy and ATLAS offer starting vocabularies.

Authorization for testing. Model extraction, prompt injection, data poisoning and agent red teaming can cross legal and ethical boundaries when performed against production services or third-party data without permission. A research purpose does not automatically authorize bypassing access controls, generating costs, accessing other users’ information or planting content that will later be ingested. Programs need written scopes, test tenants, data-handling rules, stop conditions, responsible-disclosure channels and procedures for unexpected access.

Dual use. Detailed attack research is essential for reproducibility and defense, but releasing turnkey poisoning pipelines, stealth triggers or autonomous exploit agents can lower barriers for malicious use. Responsible publication should balance exploit detail against defensive value, coordinate disclosure where a live system is affected, avoid retaining victim data and provide mitigations and detection artifacts.

Human oversight and accountability. Human approval is not meaningful if reviewers lack context, face automation bias or must approve hundreds of machine-generated actions. High-consequence approval interfaces should show the original user request, untrusted sources that influenced the model, requested tool parameters, affected resources and a concise explanation of policy checks. Approval should be independent of the potentially compromised model.

Fairness and civil rights. Robustness failures can be unevenly distributed. A system may resist adversarial manipulation in dominant languages, common image conditions or well-represented populations while failing elsewhere. Security evaluation should therefore intersect with subgroup performance and accessibility testing. Otherwise, a nominally secure model may externalize risk to underrepresented users.

Open research questions and organizational best practices

Several problems remain fundamentally open.

Reliable separation of instructions and data. Current LLM architectures process system instructions, user text and retrieved content through closely related token pathways. Prompt hierarchies, classifiers and fine-tuning can reduce attack success, but they do not provide a universal memory-protection boundary comparable to hardware privilege rings. Research is needed on typed or capability-secure model interfaces, information-flow control, isolated planning, formally constrained tool protocols and architectures that make instruction provenance enforceable rather than merely described in text. Promptware research explicitly identifies the instruction/data ambiguity as foundational.

Scalable backdoor and poisoning assurance. Foundation models may contain billions of parameters and train on web-scale corpora. Exhaustive trigger search is impossible, while simple anomaly detectors may fail against clean-label, semantic, distributed or representation-level backdoors. Open questions include source-level influence tracing, efficient unlearning, cryptographically attestable training, detection of targeted behavior without knowing the target, and statistical guarantees about bounded poison influence.

Adaptive, system-level evaluation. Many benchmarks test one-turn prompts against a static model. Real attackers adapt, observe outputs, combine modalities, poison memory, exploit tools and wait for favorable context. Evaluations need multi-step, cross-session and multi-user scenarios, realistic permissions and measures of end-to-end objective completion—not just the percentage of prompts that produce a prohibited string. The InjecAgent benchmark’s 1,054 tool-integrated cases and the Promptware model’s multi-stage analysis are steps in this direction.

Quantitative residual-risk measurement. There is no broadly accepted method to convert attack success rates into enterprise risk. A ten-percent injection rate could be tolerable in a read-only sandbox and intolerable in a payment agent. Useful metrics must incorporate attacker cost, repeatability, detectability, authority, exposure frequency, data sensitivity, blast radius and reversibility.

Privacy leakage from foundation models. Membership inference and reconstruction results vary dramatically with memorization, duplicate data, access to logits, model scale and attacker priors. Organizations need standardized privacy test suites, defensible thresholds and practical methods for removing a person’s information from pretrained and derived artifacts without full retraining.

Provenance and forensic reconstruction. Investigators often cannot answer which data, model version, prompt, retrieved passages, memory entries, tool outputs or policies produced an action. Research is needed on tamper-evident AI execution traces that preserve enough evidence for incident analysis without logging excessive personal or confidential information.

Model extraction attribution. Watermarking and fingerprinting can support claims that a suspect model derives from another, but robustness against fine-tuning, pruning, distillation, quantization and adversarial removal remains uncertain. Research should distinguish evidentiary attribution from mechanisms intended to prevent extraction.

Compositional assurance. A safe model can be placed in an unsafe application, while a less robust model can be made tolerable by strict containment. Assurance methods must compose claims across the model, retrieval system, orchestrator, tool gateway, identities, network, user interface and human process.

Autonomous cyber-capability containment. The preliminary July 2026 incident raises questions about sandbox escape, package-installation channels, hidden network pathways, long-horizon goal pursuit, evaluation-time safeguards and when model-generated exploitation should be interrupted. Independent investigation is still required before drawing general conclusions, but the event demonstrates that evaluation environments themselves must be threat-modeled as hostile-execution environments.

A prioritized organizational program should implement the following practices:

A practical deployment rule is:

No untrusted content should be able to cause a sensitive action solely because a probabilistic model interpreted that content as an instruction.

Organizations that cannot enforce this rule should reduce the system’s authority, isolate it from sensitive information, add independent authorization, or keep the relevant workflow out of autonomous operation. This is the most robust way to “break” the AI kill chain: not by assuming every poison or prompt can be recognized, but by ensuring that successful manipulation cannot directly become unacceptable impact.