Benchmark Contamination and the False Assurance Problem in AI Safety Evaluations
AI safety benchmarks like HarmBench, ToxiGen, and AdvGLUE are increasingly used to assert that a model is safe. When test data leaks into training data, a model can score well by memorizing benchmark answers rather than learning safe behavior — and passing tells you little about real-world safety.