Tagged:

2 posts

• Automating the Red Team: Using AI to Attack AI at Scale

  Traditional red teaming relies on human creativity to find AI failure modes, but manual coverage doesn't scale. Automated red teaming flips the model — using a red team LLM to systematically generate, score, and iterate adversarial prompts — enabling coverage at scale while raising new questions about dual-use risk.

  Jul 1, 2026 red-teaming adversarial-ml ai-safety jailbreaking automated-testing llm-security

• Prompt Injection as Role Confusion: Why LLMs Trust Style Over Role Tags

  New research reveals LLMs identify roles from writing style rather than structural tags — making the system role tag a suggestion, not a security boundary. Destyling adversarial text drops attack success from 61% to 10%. Here's what that means for defenders.

  Jun 28, 2026 prompt-injection jailbreaking defense-patterns llm-security agent-security