As AI-generated text floods every channel from social media to legal filings, the question of attribution — was this written by a person or a model? — has moved from academic curiosity to regulatory priority. The EU AI Act, C2PA, and public commitments from major providers all point toward watermarking as part of the answer.

The technique is genuinely useful. It also has hard limits that are routinely omitted from vendor announcements. This post lays out the mechanisms, the evasion surface, and what watermarking can and cannot guarantee in practice.

How Token-Level Watermarking Works

The canonical approach is the green/red list scheme introduced by Kirchenbauer et al. (2023)¹. The mechanism is elegant in its simplicity.

During sampling, a pseudo-random function partitions the model’s vocabulary into a “green” list and a “red” list, seeded by the preceding token combined with a secret key held by the provider. The sampler is then biased to prefer green-list tokens — not by making red-list tokens impossible, but by adding a fixed logit boost to green candidates. The boost is small enough that output quality is barely affected.

Detection works by checking whether a candidate text shows a statistically improbable concentration of green-list tokens. The green-list fraction γ is a parameter (the paper evaluates γ = 0.25 and γ = 0.5); if the observed proportion of green tokens significantly exceeds γ, the hypothesis that the text was unmodified LLM output is supported. The test is a standard one-tailed z-test; the paper derives the significance threshold analytically.

Three properties make this attractive for deployment:

1. Zero overhead on the user side: the watermark is invisible and adds no metadata. There is nothing to strip from the output.
2. Scalable detection: anyone with the key can verify any passage without re-running the model.
3. Statistical, not exact: detection is probabilistic, which means false positives are rare but nonzero.

The third property becomes painful at short lengths. Very short texts — the genuinely hard case being a single tweet or brief paragraph — are statistically underpowered. Kirchenbauer-style detectors typically need 100+ tokens for strong confidence at practical false-positive rates; the scheme is designed for bulk analysis of long-form documents, not individual sentence attribution.

A follow-on problem is key compromise. If the mapping between token and color-list is discovered, an adversary can pre-screen their paraphrases for green-list density and selectively choose outputs that clear the detection threshold anyway — inverting the system.

Cryptographic Approaches: Signed Metadata

A different class of techniques doesn’t touch the model output at all. Instead, it signs the generation event: the provider records model ID, timestamp, sampling parameters, and a hash of the output, then issues a verifiable credential tied to that session.

C2PA (Coalition for Content Provenance and Authenticity)² is the leading standard here. A C2PA manifest is attached to the file or document, carrying a chain of signed assertions about its origin and any edits. AI providers can participate as a signing authority; content consumers can verify the chain.

This approach has obvious appeal: it’s exact rather than statistical, it carries rich metadata, and it can survive distribution through cooperative pipelines as long as the manifest is preserved and the content is not modified.

The equally obvious limitation: the chain breaks when content is retyped or meaningfully modified. C2PA supports hard bindings (cryptographic hashes over the content) and soft bindings (perceptual hashes), but hard bindings — the common case for text — invalidate if the text is paraphrased or retyped into a new file. Transcription is a trivially complete evasion strategy for hard-bound manifests.

Cryptographic watermarking also requires closed inference pipelines for broadly trusted provenance. The provider must control the signing infrastructure, trust the hardware, and prevent output manipulation between generation and signing. Open-source or locally-deployed models can sign their own generation events, but will not get broadly trusted third-party provenance by default without a recognized signing authority — and key distribution remains non-trivial in any configuration.

The practical scope is therefore limited to content generated by cooperative providers (OpenAI, Google, Anthropic, Adobe Firefly) consumed in environments that check C2PA chains (supported browsers, publishing platforms). This is a meaningful but narrow slice of the real-world content distribution surface.

Semantic Watermarks: Research-Stage, But Worth Tracking

Token-level watermarks are fragile under paraphrase because the statistical signal is in word choice, not meaning. Semantic watermarking aims to encode provenance in meaning-preserving structure — in the syntactic or semantic patterns of text rather than in token distributions.

Hou et al. (2023, published NAACL 2024)³ demonstrated SemStamp, a scheme that operates at the sentence level rather than the token level. Using locality-sensitive hashing (LSH) over a sentence embedding space, it partitions the space into valid and invalid regions, then uses rejection sampling to generate whole sentences that land in valid regions. Detection checks whether the candidate text’s sentences cluster in the expected embedding regions. Because the scheme operates on semantic structure rather than token distributions, rewriting the surface words of a sentence is insufficient to erase the watermark — you’d need to produce an output that also falls outside the seeded embedding partition, without knowing how the space was divided.

This is a meaningful improvement in theory. In practice, the technique remains research-stage for several reasons:

• The rejection-sampling decoding required to land sentences in valid embedding regions adds significant latency compared to standard generation; no-fine-tuning is required but the decoding overhead is non-trivial.
• The robustness is empirically validated only against automated paraphrasers; sophisticated human rewriting likely still evades detection.
• The scheme requires a shared embedding model between the watermarker and the detector; key management is analogous to token-level approaches.

No production deployment is currently known. Treat it as a direction rather than a deployable tool.

Evasion Taxonomy: What Actually Works

Attacks on watermarked text are well-understood and most are cheap. Roughly ordered by effort:

Paraphrase attacks (automated): Feed the watermarked text to a paraphrase model — a T5, Claude, or GPT-4 prompt asking for a rewritten version at similar length. This reshuffles token selection and disrupts the green/red distribution, typically preserving meaning well. Quality cost: minor. Detection accuracy: sharply reduced, with residual detectability depending on rewrite strength, text length, and detector threshold — weaker rewrites may leave residual signal in long documents.

Paraphrase attacks (human): A human reads the text and rewrites it. Slower, but completely defeats statistical and semantic watermarks alike, because the human is generating new tokens, not copying old ones.

Token substitution: Replace individual words with synonyms. At low substitution rates (under 20%), quality is unaffected and watermark signal degrades. This is simpler than full paraphrase and works well enough for casual evasion.

Lossy compression: OCR, speech-to-text, or screenshot → re-OCR pipelines introduce enough character-level noise to shift token boundaries and degrade statistical signals. For C2PA manifest approaches, creating a new file (screenshot, export, OCR output) breaks provenance by producing a new asset without a valid signed assertion — ordinary re-encoding of the original file does not inherently break the chain if the manifest is preserved.

Translation round-trips: Translate to a different language and back. This is semantically noisy but preserves meaning well enough for many applications. It thoroughly destroys token-level watermarks and breaks manifest chains.

Insertion/deletion attacks: Add or remove sentences, words, or punctuation. Statistical watermarks are diluted; their z-score drops proportionally to the fraction of non-watermarked tokens introduced.

The asymmetry here is important: evasion is cheap and reliable, while robust detection is expensive and still fragile. Any committed adversary — automated or human — can remove a token-level watermark with a single API call. Cryptographic approaches survive more attacks, but only within cooperative distribution chains that don’t describe most real-world content flows.

Regulatory Context

The EU AI Act (Article 50)⁴ imposes transparency obligations on providers and deployers of certain AI systems. The disclosure marking requirement for synthetic content applies with scope limits: providers of systems generating synthetic audio, video, image, or text content must ensure outputs are marked in a machine-readable format, subject to exceptions (authorised law enforcement uses, licensed creative content, and similar). The “intended for public disclosure” framing applies to some sub-obligations but not uniformly to all Article 50 requirements. This is not a precise technical mandate for watermarking in the Kirchenbauer sense — compliance via metadata, API labels, or format conventions may satisfy it without embedded statistical signals.

OpenAI’s 2023 White House voluntary commitments included watermarking for audio and visual AI-generated content. Scott Aaronson (then at OpenAI) separately described a text watermarking scheme in a 2022 post on his Shtetl-Optimized blog⁵ — consistent with Kirchenbauer et al. — which acknowledges the paraphrase evasion problem explicitly. No public details of a production text watermarking deployment from OpenAI have been published.

C2PA is gaining ground in media pipelines. Adobe, Microsoft, Google, Intel, and others have adopted it; major camera manufacturers are embedding C2PA signing at capture time for photographic content. AI text is further behind.

The honest interpretation: regulators want content provenance, and providers are making public commitments. Technical standards exist but are either fragile (statistical) or require closed pipelines (cryptographic). The regulatory pressure is real; the technical readiness is not yet commensurate.

Capability/Limitation Matrix

Approach	Evasion by Paraphrase	Survives Retyping	Requires Provider Cooperation	Production Ready
Token-level (green/red list)	❌ trivial	❌ always destroyed	Required (secret key for detection)	Research prototypes only
Cryptographic (C2PA)	❌ breaks hard binding	❌ always destroyed	✅ required	Yes (limited scope)
Semantic (embedding-based)	Partial	❌ likely destroyed	Depends on scheme	No (research only)

Where watermarking adds real value: bulk detection pipelines. If you need to screen millions of documents and identify which were generated using a particular watermarking key and decoding setup, token-level watermarking is practical — each individual evasion requires a separate paraphrase call, so bulk evasion has non-trivial cost, while bulk detection runs cheaply server-side. Detecting AI-generated spam, automated synthetic review content, or large-scale disinformation seeding all fall in this bucket. Note that this advantage narrows as paraphrasing API costs fall.

Where it fails: any individual use case where the adversary has motive to invest even a few minutes. A journalist verifying whether a specific email was AI-generated. A court trying to authenticate a specific document. A moderation system trying to catch an individual evader.

What to Actually Do with This

If you’re deploying LLMs and care about provenance:

1. Use metadata and API-level labeling as the primary signal. Log which outputs were AI-generated at the system level. Token-level watermarks add a backup channel, not a substitute for application-level tracking.

2. Don’t rely on watermarks for adversarial settings. If an attacker cares about bypassing detection, they will. Use watermarking for operational analytics and bulk screening, not individual attribution in high-stakes decisions.

3. Track C2PA adoption curves. For image and video content, C2PA is becoming reliable enough to depend on within cooperative pipelines. Text is lagging, but the infrastructure is being built.

4. Apply the commitment model: if you publish AI-generated content, label it at publication time via content policy — don’t depend on detection finding it later. Provenance is best established at origin, not reconstructed after the fact.

The research trajectory on semantic watermarks is worth watching. If paraphrase-robustness can be achieved at acceptable quality cost, the evasion asymmetry changes substantially. Until then, watermarking is a useful tool for the cooperative case and nearly useless for the adversarial one — and practitioners should calibrate expectations accordingly.

---

Footnotes

1. Kirchenbauer et al., “A Watermark for Large Language Models,” ICML 2023. arXiv:2301.10226 ↩

2. Coalition for Content Provenance and Authenticity, C2PA Technical Specification. c2pa.org/specifications ↩

3. Hou et al., “SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation,” NAACL 2024. arXiv:2310.03991 ↩

4. EU AI Act, Article 50 — Transparency obligations for providers and deployers of certain AI systems (2024). ↩

5. Scott Aaronson, “Watermarking GPT Outputs,” Shtetl-Optimized (2022). scottaaronson.blog ↩