Tagged:
supply-chain15 posts
-
Slopsquatting: When AI Hallucinated Package Names Become a Supply Chain Attack
LLMs hallucinate package names at rates exceeding 20%. Attackers register those hallucinated names on npm and PyPI with malicious payloads. No typo required — the developer trusts the AI. This post explains the mechanics, documents the evidence, and gives developers concrete defenses.
-
Quantization and Compression Attacks: How Model Size Reduction Can Re-Enable Suppressed Unsafe Behaviors
When you deploy a GGUF Q4_K_M quantization of a safety-aligned model, you're relying on an assumption: that alignment survives precision reduction. Research now demonstrates two distinct ways that assumption can fail — and why quantized models need their own safety verification.
-
Model Hub Supply Chain Attacks: Malicious Models, Tokenizer Exploits, and Typosquatting on Hugging Face
Downloading an open-weight model from a public hub is not a read-only operation. Custom tokenizer classes and auto_map configs execute arbitrary Python when trust_remote_code is set, LoRA adapters can trojanize safe base models, and typosquatted namespaces are a documented distribution vector. Here's the threat model practitioners need before they run from_pretrained().
-
Poisoning the Pretraining Corpus: How Attackers Corrupt Foundation Models Before They're Built
Modern foundation models train on trillions of tokens scraped from the web. Carlini et al. 2023 demonstrated that purchasing expired web domains in Common Crawl snapshots lets an attacker inject poisoned training examples into the datasets foundation models train on — before a single GPU fires up.
-
Hallucination as a Security Surface: Package Fabrication, Fake Credentials, and Confident Wrong Advice
LLM hallucination isn't just a reliability problem — it's an attack surface. Fabricated package names, fake credentials, and wrong security advice create real, exploitable gaps.
-
CI/CD Pipeline Injection: When AI Code Assistants Become Supply Chain Threats
AI coding assistants consume your codebase as context — and that context is attacker-controllable. This post maps the attack surface: indirect prompt injection via poisoned READMEs, hallucinated package names that resolve to malicious code, AI-generated tests that exfiltrate secrets, and training data poisoning that shifts model behavior at the corpus level.
-
AI Agent Supply Chain Attacks: Compromising Agents Before They Run
SolarWinds taught us that compromising a dependency upstream is more effective than attacking the target directly. The same logic applies to AI agents: model weights, prompt templates, tool registries, and evaluation datasets are all upstream dependencies that, if poisoned, produce a backdoored agent that behaves normally until triggered.
-
Tool Poisoning via Malicious MCP Servers: When Your Agent's Tools Turn Against It
MCP servers are the extension layer for modern AI agents — granting file access, web search, code execution, and API calls. This post examines the threat of malicious or compromised MCP servers: how they exploit the agent's implicit trust in its own tooling (including via tool definition injection, rug-pull attacks, and cross-tool description chaining), the attack classes that follow, and the defense patterns that actually work.
-
Fine-Tuning Trojans: Injecting Backdoors Through the Model Training Pipeline
How malicious training data, tampered datasets, and compromised fine-tuning APIs plant backdoored behavior in legitimate base models — and what defenders can do.
-
Sleeper Agents in Production: The AI Supply Chain Backdoor Threat
Anthropic proved sleeper agents exist and resist standard safety fine-tuning. Here's who's actually at risk in 2026 and what a realistic defense looks like when fine-tuned open-weight models are everywhere.
-
When AI Finds Your Bugs, Should You Hide the Code?
The UK's Government Digital Service published guidance that Terence Eden — who helped craft the NHS's original open-source policies — read as a pointed public rebuke of NHS England's decision to close nearly 200 open-source repositories in response to AI-assisted vulnerability discovery. What this dispute reveals about a wider security misunderstanding.
-
SQLite AGENTS.md: No Agentic Code Accepted
D. Richard Hipp's three-sentence policy in SQLite's AGENTS.md distills what most AI governance frameworks bury in pages of caveats: the line isn't between human and machine intelligence — it's between verified and unverified contributions. An analysis of what this stance gets right, and why the same commit log that hosts the policy also cites Claude analysis.
-
The Trigger You Can't See: Steganographic Backdoors in Deployed Language Models
SteganoBackdoor shows that a model can be made to reliably produce attacker-controlled outputs whenever a semantic trigger appears — without the trigger ever appearing in the poisoned training data. Existing data-curation defenses can't find what they're not designed to look for.
-
Five Lines of Injection: How Microsoft Copilot Cowork Exfiltrates Pre-Authenticated File Links Without Approval
PromptArmor demonstrated that five malicious lines hidden in a Copilot Cowork Skills file can exfiltrate pre-authenticated OneDrive and SharePoint download links — no-login URLs granting instant file access — without any user approval. A 5-for-5 success rate, model-agnostic, and no patch available. The attack exposes a structural flaw in agent approval design: classifying actions by destination rather than content.
-
Your Spreadsheet Is the Attack Surface: ChatGPT for Google Sheets Data Exfiltration
A single hidden prompt in an imported sheet can silently exfiltrate your entire Google Drive workbook collection, replace the ChatGPT sidebar with an attacker-controlled phishing interface, and bypass the extension's own human-approval safety setting. PromptArmor's analysis of the ChatGPT for Google Sheets add-on shows how LLM extensions inherit every permission you've granted — and hand them to whoever controls the data you import.