Home › Learning Paths › AI Security Foundations

AI Security Foundations

For: Security practitioners new to AI, ML engineers learning threat modeling, or anyone who wants a structured overview before diving deeper.

AI security has its own vocabulary, attack classes, and mental models that don't map cleanly onto traditional application security. This path walks you through the foundational layer: how to think about threats in AI systems (MITRE ATLAS), the oldest and most studied attack class (adversarial examples), the most exploited vulnerability in deployed LLMs (prompt injection), where untrusted components enter the supply chain, and finally what to do when something goes wrong. By the end you'll have the conceptual scaffolding to understand any other post on this blog.

7 posts in this path ~105 min total reading time Read in order — each post builds on the last

  1. MITRE ATLAS: Mapping the AI/ML Threat Landscape with an Authoritative Adversarial Framework

    Start here: MITRE ATLAS is the ATT&CK-equivalent for AI/ML — this post maps the entire threat landscape into a shared taxonomy you'll reference throughout.

    MITRE ATLAS is the ATT&CK-equivalent for adversarial machine learning — a living knowledge base of tactics, techniques, and real-world case studies specific to AI systems. This post walks the full ATLAS matrix and shows how every attack pattern covered on this blog maps to a verifiable ATLAS technique ID.

    19 min read mitre-atlasthreat-modelingadversarial-ml

  2. Adversarial Examples: The Foundational ML Attack That Still Breaks AI Systems in Production

    The oldest and most studied AI attack. Understanding how imperceptible perturbations break classifiers builds the intuition for every evasion technique that follows.

    Imperceptible perturbations that flip neural network classifications — from FGSM and PGD to physical-world stop-sign attacks and LLM adversarial suffixes. What adversarial examples are, why gradient-based attacks work, how defenses hold up, and what this means for production AI systems today.

    19 min read adversarial-examplesevasion-attacksml-security

  3. Prompt Injection as Role Confusion: Why LLMs Trust Style Over Role Tags

    The most widely exploited LLM vulnerability. This post explains the role-confusion root cause before you encounter the more complex injection variants.

    New research reveals LLMs identify roles from writing style rather than structural tags — making the system role tag a suggestion, not a security boundary. Destyling adversarial text drops attack success from 61% to 10%. Here's what that means for defenders.

    13 min read prompt-injectionjailbreakingdefense-patterns

  4. Indirect Prompt Injection Against Production Systems: A Survey of Documented Disclosures

    Real-world incidents show prompt injection escaping the chat window — a survey of how attackers weaponize it against deployed systems.

    Five researcher-disclosed proof-of-concept exploits against production AI systems (2023–2024) reveal recurring structural patterns that defenders can act on today: from Bing Chat web-content sideloading to Microsoft 365 Copilot email exfiltration chains.

    17 min read prompt-injectionagent-securityincident-analysis

  5. AI Agent Supply Chain Attacks: Compromising Agents Before They Run

    AI systems pull in third-party models, datasets, and tools — this post maps where the supply chain introduces untrusted inputs.

    SolarWinds taught us that compromising a dependency upstream is more effective than attacking the target directly. The same logic applies to AI agents: model weights, prompt templates, tool registries, and evaluation datasets are all upstream dependencies that, if poisoned, produce a backdoored agent that behaves normally until triggered.

    10 min read supply-chainthreat-modelingagent-security

  6. Your Agent Runtime Is a 1960s Operating System

    A useful mental model: AI agents share the same structural security problems as operating systems. Builds intuition for the rest of the blog.

    A new paper from TU Berlin and CISPA maps AI agent security onto 50 years of OS research — and finds that agent runtimes are failing to apply solutions that were well-understood before most of their developers were born.

    13 min read agent-securitythreat-modelingdefense-patterns

  7. AI Incident Response: A Practitioner's Playbook for When Your AI System Is Compromised

    Closes the loop: what to do when an attack succeeds. Practical playbook for detection, containment, and recovery in AI systems.

    Detection signals, containment options, evidence preservation, and recovery procedures for AI-specific security incidents — the operational complement to attack coverage. NIST IR lifecycle applied to prompt injection, model backdoors, data poisoning, and adversarial input attacks.

    17 min read incident-responseprompt-injectionmodel-security
← Back to all learning paths