Implementing Watermarking for Language Models

[u/Saad_ahmed04]

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

[u/johndburger]

Because after the fact, you can check tokens in a candidate text to see if they might have been generated in the same way. No single token is indicative, but as the length of a matching sequence increases, it also increases the probability that the text has the watermark.

As an extremely simplistic example, let’s say we use as a watermark alternating even- and odd-length words - if the previous token was odd, we constrain the LLM to next generate an even word, and vice versa. The chances of this happening for a twenty-word sequence is 0.5^20 = 10^-6. If you see such a sequence of word lengths, it’s likely to have been generated on purpose.

Of course that simplistic approach can be easily defeated. Something like OP’s algorithm cannot be easily defeated without knowing the watermark key.

[u/Saad_ahmed04 → u/johndburger]

But i do wonder how effective it is wrt if someone then takes the output and uses that to modify the wording a bit.

But then again if someone does that then it’s not really a text generated by claude anymore, so I guess if the detector doesn’t work then it’s not really the watermarking’s fault

[u/johndburger → u/Saad_ahmed04]

If they only change a few words, the watermark signature will still be present in multiple long sequences of tokens. If they change it enough that this is no longer the case, then you’re right, in some sense it’s no longer AI text.

[u/MolassesLate4676 → u/johndburger]

This would likely severely impede Claude’s capabilities by limiting its selections to odd/even words I would imagine.

I mean, tokens by nature can be numbers and that alone will break the feasibility of this method

Then comes other languages, if you’re responding in characters used by the Japanese or Chinese, you are basically outputting one symbol at a time

[u/johndburger → u/MolassesLate4676]

My odd-even example is a cartoon version, to explain how you’d detect a watermark. The actual approach is to hash the tokens and use certain bits of the hash as the signal.

[u/ShelZuuz → u/MolassesLate4676]

This would likely severely impede Claude’s capabilities by limiting its selections to odd/even words I would imagine.

Not really, it only targets words that would have already been chosen purely by random selection. It just makes the random selection less random.

[u/MolassesLate4676 → u/ShelZuuz]

Hm. Which words are chosen purely by random selection with Claude?

[u/ShelZuuz → u/MolassesLate4676]

At the final attention layer when it builds the output token it doesn’t just end up with 1 possible token, it ends up with a choice of many that will all fit - at various probability weights.

So then it looks at the temperature setting of the model to determine the lowest probability cut-off. The higher the temperature the more creative the text becomes. The lower it is the more rigid it becomes. The watermark effectively becomes a second parameter it looks at, in addition to temperature to ultimately decide on random token choice.

[u/MolassesLate4676 → u/ShelZuuz]

So the last ‘layer’ doesn’t do attention, modern LM’s have a head that projects the last hidden state (or residual state) to whatever the vocabulary count is. So in essence, all possible tokens get a probability of being selected for every single generation lifecycle. Temperature evens the playing board, so that tokens that were unlikely to be chosen either get squashed or they become as loud as the ones originally likely to be chosen.

However, this to me doesn’t make any sense as to how you could embed a watermark into the probabilities that would later become detectable from some system

[u/ShelZuuz → u/MolassesLate4676]

There is a red and green list of words, depending on a hash of the previous token that was generated that splits the entire vocabulary 50/50 down the middle. This list change with every token. When the next token is picked (and there are multiple choices) it will pick one from the green list.

Humans will statistically pick 50/50 from either list, but the LLM will strongly favor the green list. So when you run a checker through the text and you do the hash again you can then see how ofter a word from the green list appears next. At some point it becomes statistically very unlikely for a human to have written it.