Tagged:
ai-security10 posts
-
Sponge Examples: Energy and Latency Attacks on Neural Networks
Adversarial inputs that don't fool a model's outputs — they exhaust its compute. Sponge examples maximize inference energy and latency, enabling DoS attacks that bypass rate limits, drain edge-device batteries, and degrade shared inference infrastructure. What they are, how they work, and how to defend against them.
-
AI in Critical Infrastructure: Attack Surfaces in Industrial Control Systems and Smart Grids
How AI integration into power grids, water treatment, manufacturing, and transportation creates novel attack surfaces distinct from traditional ICS/SCADA threats — sensor spoofing against ML anomaly detectors, adversarial attacks on predictive maintenance, model poisoning in federated industrial AI, and mitigations specific to OT environments.
-
Model Hub Supply Chain Attacks: Malicious Models, Tokenizer Exploits, and Typosquatting on Hugging Face
Downloading an open-weight model from a public hub is not a read-only operation. Custom tokenizer classes and auto_map configs execute arbitrary Python when trust_remote_code is set, LoRA adapters can trojanize safe base models, and typosquatted namespaces are a documented distribution vector. Here's the threat model practitioners need before they run from_pretrained().
-
Reasoning Model Security: Attacks on Chain-of-Thought and Extended Thinking
Reasoning models (o3, o4-mini, DeepSeek-R1, Claude with extended thinking) expose a distinct attack surface that standard LLM defenses don't fully address — attackers can manipulate the reasoning process itself, not just the inputs or outputs.
-
Adversarial Examples: The Foundational ML Attack That Still Breaks AI Systems in Production
Imperceptible perturbations that flip neural network classifications — from FGSM and PGD to physical-world stop-sign attacks and LLM adversarial suffixes. What adversarial examples are, why gradient-based attacks work, how defenses hold up, and what this means for production AI systems today.
-
MITRE ATLAS: Mapping the AI/ML Threat Landscape with an Authoritative Adversarial Framework
MITRE ATLAS is the ATT&CK-equivalent for adversarial machine learning — a living knowledge base of tactics, techniques, and real-world case studies specific to AI systems. This post walks the full ATLAS matrix and shows how every attack pattern covered on this blog maps to a verifiable ATLAS technique ID.
-
Benchmark Contamination and the False Assurance Problem in AI Safety Evaluations
AI safety benchmarks like HarmBench, ToxiGen, and AdvGLUE are increasingly used to assert that a model is safe. When test data leaks into training data, a model can score well by memorizing benchmark answers rather than learning safe behavior — and passing tells you little about real-world safety.
-
Tool Poisoning via Malicious MCP Servers: When Your Agent's Tools Turn Against It
MCP servers are the extension layer for modern AI agents — granting file access, web search, code execution, and API calls. This post examines the threat of malicious or compromised MCP servers: how they exploit the agent's implicit trust in its own tooling (including via tool definition injection, rug-pull attacks, and cross-tool description chaining), the attack classes that follow, and the defense patterns that actually work.
-
I Spent $1,500 Finding Out Which LLMs Can Hack a Real App — And What That Costs
A security researcher built a deliberately vulnerable app, ran nine frontier models against it as autonomous attack agents, and tracked every dollar. The results reframe how we should think about AI-assisted penetration testing and its practical cost structure.
-
When AI Finds Your Bugs, Should You Hide the Code?
The UK's Government Digital Service published guidance that Terence Eden — who helped craft the NHS's original open-source policies — read as a pointed public rebuke of NHS England's decision to close nearly 200 open-source repositories in response to AI-assisted vulnerability discovery. What this dispute reveals about a wider security misunderstanding.