Here is a threat your security model probably doesn’t include: your agent’s persistent memory is quietly building a detailed profile of your users — health conditions, financial situations, relationship details, credentials — and transmitting it to cloud services where the provider holds the cleartext, gets indexed into a vector database, and accumulates until you explicitly purge it.

This isn’t an attack someone launched against you. It’s the default behavior of every memory-augmented agent that offloads storage to a cloud service. The agent is doing exactly what it was designed to do. The privacy failure is architectural, not adversarial — and that’s precisely why most teams haven’t modeled it.

A new paper from MemTensor, HONOR, and Tongji University, “MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents” (arXiv:2605.09530, cs.CR), maps this attack surface systematically and proposes a concrete defense. The paper is a defense paper, not a threat paper — but its introduction is a useful catalog of what the threat looks like when researchers actually measure it.

The Architecture That Creates the Problem

Most production agent deployments follow the same basic pattern for memory: the agent runs on edge infrastructure (a user’s device, a local server, a thin client), but memory management — storing, indexing, retrieving memories — is offloaded to the cloud. This is rational. Vector databases, embedding models, and retrieval infrastructure are compute-intensive and expensive to run on edge devices. Cloud-hosted memory services like Mem0 provide managed infrastructure that works out of the box.

The security problem emerges from the mismatch between what memory systems need to do their job and what privacy requires.

For cloud-side memory to work well — to accurately retrieve relevant context for a future conversation — it needs to store semantically rich content. “User’s daughter Emma has a penicillin allergy diagnosed in 2025” is more useful for retrieval than “User has a family member with a drug allergy.” The more specific the memory, the better the agent performs. The more specific the memory, the more sensitive the data being stored in the cloud.

Every time a user mentions their health condition, their address, their financial situation, their password patterns, their family members’ names — if the agent’s memory system is running in the cloud, that information is being transmitted, stored, and indexed. Not as a side effect of an attack. As the designed behavior of the system.

The Attack Surface Is Quantified — Two Distinct Threat Categories

It’s worth being precise here, because the MemPrivacy paper’s introduction mixes two distinct threat categories that a practitioner needs to track separately.

The first category is active adversarial attacks on memory systems. MemPrivacy’s introduction cites prior work showing multi-turn memory attacks achieving up to 69% success rates at inducing privacy violations ([mireshghallah2025cimemories]), and leakage attacks against stored memory systems reaching 75% success ([wang2025unveiling]). Indirect prompt injection has also been demonstrated to manipulate agents into eliciting stored private information ([cui2026vortexpia]). These figures are specific to the attack setups and models in those papers — they’re cited here as evidence the research community has measured the problem seriously, not as universal baselines.

The second category is the architectural passive exposure this post is primarily about — and it doesn’t require an attacker at all. Every time your agent’s memory pipeline transmits a conversation to a cloud service, that service’s operator has access to the cleartext. This isn’t an attack; it’s the service contract. The data accumulates in vector databases, retrieval indexes, and caching layers. It may be used in model fine-tuning, incorporated into embeddings, or propagated through summarization pipelines. The cloud provider’s breach exposure, employee access patterns, and subprocessor relationships all become part of your users’ privacy footprint.

For a practitioner, the adversarial attack category is something you can partially mitigate with access controls and monitoring. The architectural exposure category is something you need to model as a baseline condition — it’s active for every agent using cloud memory, all the time, whether or not anyone is actively attacking it.

Cloud memory systems are third-party services with their own data governance policies, breach exposure, employee access logs, and regulatory footprints. When you use a cloud memory service, you’re not just accepting their computational infrastructure — you’re accepting their entire data handling posture for your users’ most sensitive information.

The regulatory dimension compounds this. Under GDPR and similar frameworks, users have a “right to be forgotten” — the right to have their data deleted. But as the paper notes, deleting stored memories from a cloud memory service doesn’t address information that has already been propagated through agent workflows, incorporated into model fine-tuning, or cached in intermediate storage. A single memory entry may propagate into summarization caches, embedding indexes, retrieved context windows, and downstream model fine-tuning — each requiring independent deletion that your “delete user data” button likely doesn’t reach. The moment sensitive data leaves the edge device and enters cloud processing pipelines, the delete semantics become uncertain.

Why Masking Doesn’t Work Well Enough

The intuitive response to this threat is aggressive masking: before sending any data to cloud memory, strip out any recognizable PII. Replace names, addresses, health conditions, financial details with redactions.

The problem is that masking trades privacy for utility in a way that degrades the core value proposition of persistent memory. A memory system that stores “User has family member with condition” instead of “User’s daughter Emma has penicillin allergy” provides weaker personalization — the agent loses the specificity that makes memory useful.

More importantly, the semantic relationships matter for retrieval. When a future conversation mentions “my daughter’s doctor appointment,” the agent needs to match that to relevant past context. If the stored memory has been aggressively masked, the retrieval signal weakens and the agent fails to surface relevant context even when it exists.

This is the core tradeoff MemPrivacy is addressing: privacy without semantic destruction. The insight is that cloud memory systems need the structure and type of sensitive information to work well — not the values themselves.

What MemPrivacy Proposes

The core mechanism is illustrated in Figure 1 of the paper. Before any memory content is transmitted to the cloud, the model identifies privacy-sensitive spans — “Emma,” “penicillin,” “diagnosed 2025” — and replaces them with semantically structured, type-aware placeholders (the exact schema uses format like <EMAIL_1>, <PERSON_1>, etc.; the examples below are illustrative of the type structure):

“User’s daughter [PERSON:FAMILY_CHILD] has a [CONDITION:DRUG_ALLERGY:SEVERITY_HIGH] diagnosed in [DATE:YEAR]”

The original-to-placeholder mapping is stored locally on the edge device. The cloud-side memory system receives content that preserves the semantic structure needed for retrieval (there’s a child, there’s a drug allergy, it’s recent) without seeing the actual values (Emma, penicillin, 2025).

When the cloud processing completes and the memory is retrieved in a future session, the edge device restores the original values before presenting them to the user. The user sees natural personalized responses. The cloud never saw the raw data.

The privacy taxonomy the paper introduces has four levels — from public information that can be transmitted freely, to highly sensitive information that should never leave the device. This lets teams configure protection policies per data type rather than applying a single masking strategy to everything.

The evaluation results are strong. MemPrivacy’s on-device model is fine-tuned via SFT and GRPO reinforcement learning specifically for privacy span detection — this is the relevant comparison point. Fine-tuned specialized models on domain-specific extraction tasks routinely outperform zero-shot prompting of larger frontier models, and that’s exactly what the paper demonstrates: fine-tuned MemPrivacy models (0.6B–4B parameters) outperform zero-shot prompting of GPT-5.2 and Gemini-3.1-Pro on the paper’s privacy extraction benchmark. The meaningful result isn’t “small beats big” — it’s that a model small enough to run on a device at real-time inference latency can achieve strong extraction accuracy. The Qwen3-4B base model fine-tuned with MemPrivacy’s training pipeline improves its F1 score from 59.34% to 85.97% on their benchmark. Across multiple cloud memory systems, utility loss is less than 1.6% compared to raw transmission — effectively preserving personalization quality while eliminating raw data exposure to cloud providers.

What This Means for Your Threat Model

The threat model most practitioners apply to agent memory is: “What happens if someone injects into the memory system?” MemPrivacy forces a different question: “What happens if the memory system works exactly as designed?”

If your agent uses any cloud-hosted memory service — Mem0, a managed vector database, or any similar infrastructure — you should treat the default configuration as a PII leakage channel. Not because it’s been compromised. Because it’s working correctly.

1. Audit what your agent’s memory is actually storing. Log a sample of the raw content being transmitted to your cloud memory service. Specifically check for: health or medical information, contact details, credential patterns or API key fragments, financial information, and minor family members’ names. Most teams that do this are surprised by the specificity — the agent is optimized to remember things that are useful, and the most useful things are often the most sensitive.

2. The “right to be forgotten” problem requires a propagation graph, not a delete button. If a user asks you to delete their data, deleting it from your memory store is not sufficient. A single memory entry may have propagated into summarization caches, embedding indexes, retrieved context windows for other sessions, and model fine-tuning datasets — each requiring independent deletion. You need to document where your data flows before you can honor deletion requests. If you can’t trace it, you can’t delete it.

3. On-device preprocessing is the architectural answer — if you have an edge device. The MemPrivacy solution requires a genuine edge device: a local machine, user device, or application server that runs the extraction model locally before transmitting to cloud services. For true edge-cloud deployments — mobile apps, desktop agents, on-premise gateways — this is feasible. For cloud-native architectures where your “agent” is a serverless function with no meaningful local component, MemPrivacy’s approach doesn’t directly apply. You’d need a local gateway layer, a privacy proxy, or an equivalent preprocessing stage. The principle holds regardless of architecture: enforce data minimization before data enters cloud pipelines, not inside them.

4. The masking-utility tradeoff is no longer your only option. Type-aware semantic placeholders achieve near-zero utility loss compared to raw transmission. If you’re using aggressive redaction because it seemed like the only option, this architecture is worth evaluating. Less than 1.6% utility loss is in the measurement noise for most production applications.

5. Third-party memory services require contract-level scrutiny, not just API integration. When you use a managed cloud memory service, your users’ data is subject to that vendor’s data retention policies, subprocessor agreements, deletion SLAs, and breach liability terms. Before integrating any cloud memory service, ask: What is their data retention default? Do they honor deletion requests through all processing tiers (not just their API)? Who are their subprocessors? Where is data residency? These aren’t abstract compliance questions — they determine your actual privacy posture.

The Bigger Picture

Most of the research on agent security focuses on adversarial attacks: prompt injection, jailbreaks, memory poisoning, supply chain attacks. These are real and worth defending against. But they require an attacker. The memory privacy problem doesn’t.

The attack surface in MemPrivacy’s threat model is created by the design choice to offload memory to the cloud, combined with the design choice to store semantically rich content to maximize retrieval quality. Both choices are rational. Together they create a persistent, high-specificity data exposure that most agents are producing right now, in production, at scale.

The prior work that MemPrivacy’s authors cite — documenting adversarial attack success rates against cloud memory systems — is measuring how easy it is to extract this data once it exists. But the data existing at all — accumulated in a cloud vector database through normal agent operation — is the prior condition that makes those attacks possible.

MemPrivacy’s contribution is showing that this isn’t a fundamental tradeoff. You can have cloud-scale memory performance without transmitting raw sensitive values. The edge device is the right place to enforce this separation. The cloud gets structure and type. The sensitive values stay local.

That’s not a complete solution to agent privacy — there are still questions about what happens on the device, about inference-time leakage through model outputs, about multi-party pipelines where the edge device itself may not be fully trusted. But it addresses the highest-volume, highest-sensitivity exposure path: the routine transmission of conversation content to cloud memory infrastructure during normal operation.

For practitioners building memory-augmented agents today, the lesson is simpler: look at your memory pipeline before you look at your attack surface. The data you’re creating through normal operation may be a larger risk than the data an attacker could extract.

---

Paper: “MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents” — arXiv:2605.09530 (cs.CR). MemTensor, HONOR Device Co., Ltd., Tongji University. Submitted May 10, 2026.