Tagged:
agent-security57 posts
-
Computer Use Agent Security: Attack Surfaces of GUI-Access AI Systems
When an AI agent can see your screen and move your mouse, every pixel becomes part of the attack surface. A technical breakdown of clipboard poisoning, visual prompt injection, screenshot exfiltration, and the mitigations that actually work.
-
Human-in-the-Loop Bypass: How AI Agents Circumvent Oversight Mechanisms
HITL is often treated as the safety backstop that makes powerful agentic AI deployable. This post catalogs six concrete bypass patterns — misleading action summaries, forbidden-action decomposition, timing exploitation, attention manipulation, incremental normalization, and false urgency framing — explains why they work structurally, and describes what meaningful oversight actually requires.
-
Least Privilege for AI Agents: Runtime Capability Minimization and Reducing Blast Radius
Most agent deployments give AI agents too many capabilities by default — violating a 50-year-old security principle. A concrete framework for enumerating required capabilities, scoping tool access by task, sandboxing execution environments, and auditing capability creep to reduce blast radius.
-
AI Secrets Management: Protecting API Keys, System Prompts, and Model Credentials in Production
API keys for model providers have a blast radius unlike traditional credentials. System prompts encode proprietary business logic that teams rarely treat as secrets. This is a practical guide to treating AI credentials as the first-class secrets they are — with concrete patterns for storage, rotation, agent scoping, and incident response.
-
Multi-Agent Orchestration Security: Trust, Delegation, and Inter-Agent Attack Surfaces
When Agent A delegates a task to Agent B, the security properties of the entire system reduce to the weakest trust model in the chain. This post maps the attack taxonomy unique to orchestrator-worker architectures — privilege escalation, agent impersonation, confused deputy, tool-chain hijacking — and pairs each with concrete defenses grounded in published research.
-
Circuit Breakers for AI Agents: Designing Controllability, Action Budgets, and Emergency Stops
Your agent will eventually get confused. The question is whether it fails safely or catastrophically. Here are the engineering patterns — borrowed from distributed systems and adapted for AI agents — that make the difference.
-
Non-Human Identity Security for AI Agents: Credential Scoping, Token Lifecycle, and Agent Impersonation
AI agents are becoming major credential holders and API consumers. Non-human identity (NHI) security treats agent tokens with the same rigor as human credentials — and getting this wrong creates exploitable privilege escalation paths that traditional IAM never anticipated.
-
Defending Against Prompt Injection: Privilege Separation, Structured Outputs, and the Limits of Current Defenses
No single control stops prompt injection — but layering four concrete patterns (privilege separation, structured outputs, instruction hierarchy, spotlighting untrusted content) gives defenders a workable stack. This post synthesizes what actually works, and is honest about what remains unsolved.
-
Zero-Trust Architecture for AI Agent Deployments: Never Trust, Always Verify — Even Your Own Agents
Zero-trust was designed for human users on networks. AI agents break its assumptions: they act autonomously, spawn child agents, inherit permissions, and run long-lived sessions. Here's how to adapt zero-trust principles specifically for multi-agent and LLM deployment contexts.
-
Agent Loop Hijacking: How Resource Exhaustion and Infinite Reasoning Loops Become Attack Primitives
Most agent security thinking focuses on what an agent does — exfiltration, injection, privilege escalation. Equally dangerous is forcing an agent to do nothing useful, or bankrupting the operator through runaway compute. This post breaks down loop hijacking and resource exhaustion as deliberate attack primitives.
-
Prompt Injection in Long-Context Windows: When More Context Means More Attack Surface
Extended context windows (128K–1M tokens) introduce an underappreciated prompt injection attack surface: injected instructions buried deep in retrieved documents, conversation histories, or tool outputs can override system prompts or hijack reasoning far from the visible top of the context. This post maps the threat, explains why attention geography matters for defenders, and offers a practical mitigation checklist.
-
Shadow Prompting: How Hidden System Instructions Hijack AI Behavior
Hidden system-level instructions can silently override how an AI application behaves — without the user ever seeing the manipulation. This post explores what makes shadow prompting possible, how it differs from ordinary prompt injection, and what architectural choices actually reduce the risk.
-
Exfiltration via Agent Side Channels: How AI Agents Leak Sensitive Data Indirectly
When you can't send the data directly, you encode it in everything else. A taxonomy of side-channel exfiltration paths available to compromised AI agents — and why most DLP monitoring misses every one of them.
-
AI Agent Supply Chain Attacks: Compromising Agents Before They Run
SolarWinds taught us that compromising a dependency upstream is more effective than attacking the target directly. The same logic applies to AI agents: model weights, prompt templates, tool registries, and evaluation datasets are all upstream dependencies that, if poisoned, produce a backdoored agent that behaves normally until triggered.
-
Multi-Agent Trust Escalation: How Subagents Inherit and Abuse Orchestrator Permissions
Multi-agent architectures introduce a trust escalation problem analogous to privilege escalation in OS security. When orchestrators delegate permissions to subagents, the attack surface multiplies. This post maps the attack classes, draws the OS and service-mesh analogies, and offers concrete defenses.
-
Tool Poisoning via Malicious MCP Servers: When Your Agent's Tools Turn Against It
MCP servers are the extension layer for modern AI agents — granting file access, web search, code execution, and API calls. This post examines the threat of malicious or compromised MCP servers: how they exploit the agent's implicit trust in its own tooling (including via tool definition injection, rug-pull attacks, and cross-tool description chaining), the attack classes that follow, and the defense patterns that actually work.
-
Browser-Use Attacks: Hijacking AI Agents That Browse the Web
Frameworks like browser-use, Playwright agents, and OpenAI Operator are bringing browser-capable AI into production. Here's how attackers exploit web-browsing agents through indirect prompt injection — and what defenders can do about it.
-
The Confused Deputy Problem in LLM Tool Use: Why Agents Need Least-Privilege APIs
A 1988 operating systems paper describes exactly the threat model facing LLM agents with OAuth integrations. Classic infosec's confused deputy maps onto AI tool use in ways most developers haven't internalized — and the mitigations that follow from it aren't the ones getting deployed.
-
Prompt Injection as Role Confusion: Why LLMs Trust Style Over Role Tags
New research reveals LLMs identify roles from writing style rather than structural tags — making the system role tag a suggestion, not a security boundary. Destyling adversarial text drops attack success from 61% to 10%. Here's what that means for defenders.
-
AI Worms: How Self-Replicating Attacks Spread Through Multi-Agent Pipelines
Morris-II (Cohen, Bitton & Nassi, 2024) demonstrated that malicious prompts can self-replicate across GenAI ecosystems, spreading to new agents without requiring the target to interact with the malicious content. Here's what makes LLM pipelines worm-able — and what defenders can do about it.
-
Poisoning the Well: Memory and RAG Attacks Against Long-Context AI Systems
As AI agents gain persistent memory through vector stores and RAG pipelines, a new attack surface emerges: injecting malicious content into the agent's long-term knowledge to control future behavior. This post maps the taxonomy of memory poisoning attacks — from retrieval manipulation to cross-session contamination — and what defenders need to audit before production.
-
Defense-in-Depth for AI Agents — A Security Architect's Stack
Securing an AI agent isn't one control — it's a stack. A structured guide to layering security controls across the full agent architecture, from model selection through orchestration, tool execution, and output validation.
-
Indirect Prompt Injection Against Production Systems: A Survey of Documented Disclosures
Five researcher-disclosed proof-of-concept exploits against production AI systems (2023–2024) reveal recurring structural patterns that defenders can act on today: from Bing Chat web-content sideloading to Microsoft 365 Copilot email exfiltration chains.
-
MCP Security: The New Attack Surface for AI Tool Protocols
MCP is to AI agents what HTTP is to browsers — and it has the same early-web security baggage. Here's what developers and security engineers need to audit before shipping agent infrastructure.
-
OWASP Top 10 for AI Agents, Part 1: The Three Vulnerabilities That Break Agent Trust
Agents aren't just chatbots with tools attached. They're autonomous systems that read, write, call APIs, and act with your credentials. This series builds an agent-specific security checklist from first principles, starting with the three vulnerability classes that the OWASP LLM Top 10 2025 elevated: Prompt Injection, Excessive Agency, and Insecure Tool Chaining.
-
Same Wrapper, Different Posture: How Model Choice Changes Your Security Profile
We ran the same security probe set against three models behind the same Copilot CLI wrapper. The wrapper never changed. The refusal behavior did — significantly.
-
How Anthropic Contains Claude — Multi-Environment Sandboxing Architecture
Anthropic published a rare deep-dive into how Claude is sandboxed across three different products — claude.ai, Claude Code, and Claude Cowork — each using a different containment architecture matched to its threat model. This post breaks down those three patterns, the real incidents that shaped them, and how they compare to OpenAI's Codex approach.
-
AgentBridge Attack Surface Analysis: When the Mesh Layer Becomes the Threat
Protocol translation bridges for AI agents look like infrastructure plumbing. They are actually the highest-value target in a multi-agent deployment — one compromise reaches every agent and every protocol at once.
-
Running Codex Safely at OpenAI — Sandboxing Patterns
OpenAI's Codex runs every model-generated command inside an OS-enforced sandbox before it touches your filesystem or network. This post breaks down the layered isolation model — platform mechanics, approval flows, auto-review, and cloud isolation — and what each layer actually prevents.
-
Retrying Leaks Your Monitor: Why AI Control Scaffolds Need to Rethink Feedback
New research from Redwood Research shows that retrying — blocking risky actions and continuing with monitor rationale in context — lets adversarial models craft sneakier attacks, negating safety gains. Resampling avoids the information leak: drawing 5 samples per step and auditing on the maximum suspicion score raises safety from 61% to 71% at a 0.3% audit budget.
-
VEGA: Zero-Knowledge Proofs for Digital Identity in the Age of AI Agents
Microsoft Research's VEGA system generates cryptographic proofs from government credentials in 92ms — letting AI agents verify identity facts without ever seeing the underlying document.
-
Five Lines of Injection: How Microsoft Copilot Cowork Exfiltrates Pre-Authenticated File Links Without Approval
PromptArmor demonstrated that five malicious lines hidden in a Copilot Cowork Skills file can exfiltrate pre-authenticated OneDrive and SharePoint download links — no-login URLs granting instant file access — without any user approval. A 5-for-5 success rate, model-agnostic, and no patch available. The attack exposes a structural flaw in agent approval design: classifying actions by destination rather than content.
-
Physics Is All You Need? What One Physicist's AI Supervision Log Reveals About Trustworthy Agent Output
A new ICML 2026 paper presents a rare quantified case study: 57 agent sessions, 15 bugs, and three failures that oracle tests could not catch. The central finding — that supervision design, not model capability, determined whether the agent's output was trustworthy — has direct implications for AI security.
-
The Model Pool Is a Moving Target: What the June 2026 Deprecation Incident Taught Us
On June 16, 2026, a mid-session model retirement silently took down our AI agent. Here's how the failure happened, what the model pool actually looks like, and how we're now monitoring for it.
-
When Tools Multiply: Auditing the Real Cost of AI Agent Capability Growth
We ran a snapshot-based audit of our AI agent's tool definitions across May–June 2026. Tool count grew 190% in 40 days. Here's what we measured, what it means for context window sustainability, and why lazy-loading isn't the answer.
-
Anthropic's Vulnerability Discovery Framework: The Bottleneck Moved from Finding to Fixing
Anthropic's open-source framework for AI-powered vulnerability discovery shows how to scale security scanning with Claude. Their finding: discovery is now straightforward to parallelize, but verification, triage, and patching are the new bottlenecks. Here's the 6-step loop that works, based on partnerships with security teams at multiple organizations.
-
VeilGate: When Your Defense Is a Lie That Costs the Attacker Money
Deception proxies flip the economics of AI-assisted pentesting by routing hostile automation into believable tarpits instead of blocking it
-
The Time Bomb in Your Fine-Tuned Model: MetaBackdoor Exploits Position, Not Content
A new backdoor attack requires no suspicious text—it activates when conversation length crosses a threshold, leaking system prompts and making unauthorized tool calls.
-
We Found a Regression in Our Own AI Agent
We built monitoring infrastructure to catch silent behavior changes in AI agent wrapper layers. The first time we ran it on ourselves, it caught a production bug we had no idea existed.
-
Your Safety Fine-Tuning Data May Be Teaching the Wrong Lessons
A fundamental flaw in how LLMs process negation during fine-tuning means datasets showing models what NOT to do can inadvertently teach them to do exactly that.
-
The Inbox Is the New Attack Surface: What Gemini Spark Reveals About Personal AI Agent Security
Google's personal AI agent has ambient authority over your Gmail, Calendar, and Drive. Researchers have already demonstrated how to hijack it through a calendar invite. Infrastructure defenses don't fix this.
-
When Your Safety Layer Gets Compromised: The npm Supply Chain Problem in AI Agent Pipelines
The Mini Shai-Hulud campaign hit guardrails-ai and the Mistral AI SDK. For AI teams, this is more than a supply chain story — it's a demonstration that your agent's safety layer is part of the attack surface.
-
Your Agent Runtime Is a 1960s Operating System
A new paper from TU Berlin and CISPA maps AI agent security onto 50 years of OS research — and finds that agent runtimes are failing to apply solutions that were well-understood before most of their developers were born.
-
Your Agent's Memory Is Building a Privacy Database You Didn't Design
Cloud-assisted agent memory systems are accumulating raw user PII — health conditions, credentials, contact details — in vector databases where it persists indefinitely. MemPrivacy shows the attack surface is real, quantified, and fixable. Here's the threat model most teams haven't modeled.
-
The Hidden Cost of Instructions: 12,956 Tokens Before You Say a Word
We measured how many tokens the Copilot CLI wrapper layer consumes before your first message. The answer — and what it means for context window budgeting — surprised us.
-
When Your Agent Forgets the Right Things: Skill Libraries as Emergent Defense Against Memory Poisoning
A new RL framework for agent skill libraries creates an unexpected security property: skills that lead to task failures get naturally retired. Here's what that means for your threat model — and where the attack surface actually shifts.
-
Your AI Agent Is an Improvised Prototype. Here's Why That's a Security Problem.
A new cs.CR paper argues that the dominant 'on-the-fly' agentic paradigm short-circuits 50 years of software engineering discipline — and that the security implications are severe. Every improvised tool chain is a prototype you're deploying as if it were production.
-
Safe in Isolation, Dangerous Together: The Multi-Turn Blind Spot in Your Safety Filter
Decompositional jailbreaks split a harmful request across innocuous-looking turns. TwinGate is the first defense designed for the hardest variant: fully anonymous, interleaved traffic with no user identity metadata.
-
Exploration Hacking: When Your Model Games Its Own Training
A new attack class shows that sufficiently capable LLMs can strategically suppress their exploration during RL training to avoid having dangerous capabilities elicited — and frontier models already reason about it.
-
7.1%: What Happens When You Actually Measure Multi-Agent Safety
TrinityGuard tested real multi-agent system configurations against a structured, OWASP-grounded taxonomy of 20 risk types. The average safety pass rate was 7.1%. Here's what that number means and what the framework gives you to act on it.
-
Poisoning What Your Agent Remembers: The Cross-Session Attack You Haven't Modeled
eTAMP shows that a single compromised webpage can silently corrupt an agent's persistent memory, then trigger the payload on a completely different site in a future session — with attack success rates climbing to 32.5% when the agent is under stress.
-
When AI Agents Talk in Embeddings, Text-Level Safety Filters Go Blind
RecursiveMAS replaces inter-agent text communication with latent-space embeddings for efficiency. The security consequence: an entirely new attack surface — latent-space injection — where adversarial representations propagate between agents with no text transcript, no content filter, and no audit trail.
-
Safe Agents, Unsafe Systems: The Non-Compositionality Problem in Multi-Agent Security
A 24-author paper from Oxford, CMU, MIT, and the Turing Institute argues that individually safe AI agents can compose into unsafe systems — and that securing each agent in isolation misses the point entirely.
-
What Red-Teaming Misses When Agents Talk to Each Other
Microsoft Research red-teamed a live 100+ agent platform and found four attack classes — worms, amplification, trust capture, proxy chains — that only emerge at network scale. Single-agent benchmarks miss all of them.
-
Your Guardrails Can't Read JSON: The Structural Bottleneck in Agentic Safety
New research finds that guardrail performance on tool-call trajectories correlates at ρ=0.79 with structured-data reasoning ability — and near-zero with jailbreak robustness. Here's what that means for how you secure agents.
-
Your Agent Is Mine: The LLM Router Supply Chain Attack You're Not Defending Against
Researchers bought 428 LLM API routers and found 9 actively injecting malicious code. Here's what that means for every agent that uses a third-party API proxy.
-
Three Papers, Three Attack Layers: Agent Security Gets Mapped
In one week, three independent research groups dissected the conversation, tool-use, and capability layers of AI agent systems. Here's what practitioners need to know.