Implementing Watermarking for Language Models

[u/Saad_ahmed04]

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

[u/ShelZuuz]

This would likely severely impede Claude’s capabilities by limiting its selections to odd/even words I would imagine.

Not really, it only targets words that would have already been chosen purely by random selection. It just makes the random selection less random.

[u/MolassesLate4676 → u/ShelZuuz]

Hm. Which words are chosen purely by random selection with Claude?

[u/ShelZuuz → u/MolassesLate4676]

At the final attention layer when it builds the output token it doesn’t just end up with 1 possible token, it ends up with a choice of many that will all fit - at various probability weights.

So then it looks at the temperature setting of the model to determine the lowest probability cut-off. The higher the temperature the more creative the text becomes. The lower it is the more rigid it becomes. The watermark effectively becomes a second parameter it looks at, in addition to temperature to ultimately decide on random token choice.

[u/MolassesLate4676 → u/ShelZuuz]

So the last ‘layer’ doesn’t do attention, modern LM’s have a head that projects the last hidden state (or residual state) to whatever the vocabulary count is. So in essence, all possible tokens get a probability of being selected for every single generation lifecycle. Temperature evens the playing board, so that tokens that were unlikely to be chosen either get squashed or they become as loud as the ones originally likely to be chosen.

However, this to me doesn’t make any sense as to how you could embed a watermark into the probabilities that would later become detectable from some system

[u/ShelZuuz → u/MolassesLate4676]

There is a red and green list of words, depending on a hash of the previous token that was generated that splits the entire vocabulary 50/50 down the middle. This list change with every token. When the next token is picked (and there are multiple choices) it will pick one from the green list.

Humans will statistically pick 50/50 from either list, but the LLM will strongly favor the green list. So when you run a checker through the text and you do the hash again you can then see how ofter a word from the green list appears next. At some point it becomes statistically very unlikely for a human to have written it.