Tagged:

6 posts

• Model Inversion Attacks: Reconstructing Private Training Data from Model Confidence Scores

  A model's output probability scores are not neutral summaries — they encode information about the training data that produced them. Model inversion attacks exploit this to reconstruct representative training examples from API-level confidence values. This post covers the foundational Fredrikson et al. work, black-box and GAN-based variants, conditions for vulnerability, and how to distinguish this class from membership inference and training data extraction.

  Jul 13, 2026 privacy adversarial-ml model-inversion differential-privacy attack-defense training-data

• Training Data Extraction: How Attackers Query LLMs to Surface Memorized Private Content

  LLMs verbatim-memorize chunks of their training data, and a simple prefix-completion attack can surface phone numbers, email addresses, code, and cryptographic identifiers that appeared in the training corpus — no model internals required. This post covers the mechanics, landmark empirical results, and the practical defenses that actually reduce extraction risk.

  Jul 9, 2026 privacy training-data-extraction memorization llm-security carlini differential-privacy data-governance

• Differential Privacy in Practice: What the Math Guarantees (and What It Doesn't) for AI Training Data

  Differential privacy is the strongest mathematical privacy guarantee for ML training data — but the gap between formal ε-DP and deployed reality means many 'private' AI systems aren't nearly as private as advertised. This post bridges the math and the practice.

  Jul 6, 2026 privacy differential-privacy dp-sgd membership-inference training-data epsilon attack-defense llm-security

• Membership Inference Attacks: Detecting What Was in an AI Model's Training Data

  Membership inference attacks let adversaries determine—with significant accuracy—whether a specific data record was used to train a model. This post covers the attack mechanisms, why models leak this information, real-world threat scenarios, and how differential privacy and machine unlearning address (and fail to fully close) the gap.

  Jul 5, 2026 privacy membership-inference differential-privacy machine-unlearning attack-defense gdpr overfitting

• Gradient Inversion Attacks: Reconstructing Private Training Data from Model Updates

  Gradients are not safe summaries of training data. In federated learning and fine-tuning pipelines where model updates are shared, a malicious aggregator can run gradient inversion to recover original training samples with alarming fidelity — and the defenses have hard limits.

  Jul 4, 2026 federated-learning privacy gradient-inversion differential-privacy training-data attack-defense

• Federated Learning Poisoning: The Aggregation Attack Surface

  FL aggregation is blind to participant intent. Malicious clients can embed backdoors or reconstruct private training data from gradients.

  Jul 1, 2026 federated-learning poisoning-attacks backdoor privacy threat-modeling defense-patterns gradient-attacks differential-privacy secure-aggregation