AI/ML Security Lifecycle

Begin at the source: poisoned pretraining data can bias or implant behavior before a model exists, so provenance and corpus integrity are the first controls in the lifecycle.

Modern foundation models train on trillions of tokens scraped from the web. Carlini et al. 2023 demonstrated that purchasing expired web domains in Common Crawl snapshots lets an attacker inject poisoned training examples into the datasets foundation models train on — before a single GPU fires up.

Fine-tuning introduces a second training-stage attack surface where a targeted contributor or dataset can implant a trigger without changing the base model.

How malicious training data, tampered datasets, and compromised fine-tuning APIs plant backdoored behavior in legitimate base models — and what defenders can do.

See how sleeper triggers persist in foundation models and survive later adaptation, turning a training-time integrity failure into a latent production threat.

Pre-trained LLMs can be trojaned at the foundation stage, with adversarial triggers embedded in weights that persist through downstream fine-tuning and RLHF safety training. This post explains how these attacks work, why they're so persistent, and what practitioners can do about them.

Move from model behavior to distribution: public model hubs can introduce malicious weights or metadata, making publisher and dependency trust part of deployment security.

Downloading an open-weight model from a public hub is not a read-only operation. Custom tokenizer classes and auto_map configs execute arbitrary Python when trust_remote_code is set, LoRA adapters can trojanize safe base models, and typosquatted namespaces are a documented distribution vector. Here's the threat model practitioners need before they run from_pretrained().

Model files are executable supply-chain artifacts in practice; this post shows why unsafe serialization can compromise the environment before inference begins.

Downloading a model file and calling torch.load() is a potential code execution event. This post explains the pickle mechanics that make it so, the real-world exploits already found in the wild, how HuggingFace has responded, and what safe alternatives actually prevent the attack.

Apply artifact integrity controls: provenance records, signatures, and SBOM-style verification establish what was built, inspected, and approved before promotion.

You wouldn't deploy software without checksums and signatures. But most organizations download model weights and run them without any provenance verification at all. This post covers the practical mechanics of model signing, ML SBOMs, and the emerging infrastructure for verifying a model's origins before it touches production.

Once an artifact is trusted, protect its runtime: model-serving and GPU-memory boundaries create deployment-specific confidentiality, isolation, and availability risks.

Self-hosted LLMs introduce attack surfaces below the application layer: GPU VRAM residuals between tenants, unauthenticated serving APIs, and unverified model weights on disk. This post maps the infrastructure threat model and provides a hardening checklist.

Operationalize detection by monitoring anomalous behavior and retaining audit evidence that can distinguish misuse, drift, and compromise in production.

Guardrails block known bad inputs. Incident response handles breaches after they happen. This post covers the gap: building LLM-specific observability that detects attacks while they are occurring — anomalous outputs, prompt injection signatures, unusual tool-call patterns, and model drift from baseline.

Close the loop with response: use the lifecycle evidence to contain affected models or data, eradicate the cause, and improve controls before the next release.

Detection signals, containment options, evidence preservation, and recovery procedures for AI-specific security incidents — the operational complement to attack coverage. NIST IR lifecycle applied to prompt injection, model backdoors, data poisoning, and adversarial input attacks.