Tagged:
inference-infrastructure4 posts
-
Cross-Tenant Data Leakage in Multi-Tenant LLM Deployments: Incidents, Architecture, and What to Demand from Providers
When an LLM service hosts multiple customers on shared infrastructure, failures in conversation isolation, prompt cache sharing, or conversation history storage can expose one customer's data to another. This post covers the documented failure modes, the March 2023 ChatGPT Redis incident, batch inference and system prompt bleed risks, and what enterprise architects should demand from LLM providers before trusting them with sensitive data.
-
Prompt Cache Timing Attacks: Side-Channel Leakage in LLM API Infrastructure
Major LLM providers cache prompt prefixes to reduce latency and cost — but shared cache infrastructure creates a timing side-channel. An attacker with access to a shared API key or multi-tenant deployment can measure whether a specific prefix is cached, leaking information about co-tenant system prompts and conversation context within the same cache namespace.
-
Securing the AI Inference Stack: GPU Memory Isolation, Model Serving Hardening, and Self-Hosted LLM Infrastructure Security
Self-hosted LLMs introduce attack surfaces below the application layer: GPU VRAM residuals between tenants, unauthenticated serving APIs, and unverified model weights on disk. This post maps the infrastructure threat model and provides a hardening checklist.
-
Adversarial Prompt Caching: Timing Attacks and Injection via Shared KV Caches
As LLM providers roll out shared prefix caching to cut inference costs, a new attack surface emerges: cache-timing side channels and cross-tenant injection vectors at the inference infrastructure layer.