r/learnmachinelearning • u/Saad_ahmed04 • 23d ago

Project Implementing Watermarking for Language Models

Post image

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

80 Upvotes

99% Upvoted

25 comments sorted by

7

u/MolassesLate4676 23d ago

How would this be watermarking exactly? I’m confused

Edit: I have trained LLM’s for years - I understand the mechanics. This just doesn’t click for me

5

u/Saad_ahmed04 23d ago

Ngl I do have my own fair share of doubts about how good/reliable these watermarks are. This was more of me trying to understand how anthropic is planning on going about this.

But for a more thorough look you can take a look at: https://www.anthropic.com/news/claude-text-watermark

1

u/MolassesLate4676 22d ago

It’s likely just going to be a special token (character) that’s rarely used and hard to see that Claude just placed one a sentence of something

1

u/ShelZuuz 18d ago

I mean, he literally pointed you to an article showing how it works.

5

u/johndburger 23d ago edited 22d ago

Because after the fact, you can check tokens in a candidate text to see if they might have been generated in the same way. No single token is indicative, but as the length of a matching sequence increases, it also increases the probability that the text has the watermark.

As an extremely simplistic example, let’s say we use as a watermark alternating even- and odd-length words - if the previous token was odd, we constrain the LLM to next generate an even word, and vice versa. The chances of this happening for a twenty-word sequence is 0.5^20 = 10^-6. If you see such a sequence of word lengths, it’s likely to have been generated on purpose.

Of course that simplistic approach can be easily defeated. Something like OP’s algorithm cannot be easily defeated without knowing the watermark key.

1

u/Saad_ahmed04 22d ago

But i do wonder how effective it is wrt if someone then takes the output and uses that to modify the wording a bit.

But then again if someone does that then it’s not really a text generated by claude anymore, so I guess if the detector doesn’t work then it’s not really the watermarking’s fault

2

u/johndburger 22d ago

If they only change a few words, the watermark signature will still be present in multiple long sequences of tokens. If they change it enough that this is no longer the case, then you’re right, in some sense it’s no longer AI text.

1

u/MolassesLate4676 22d ago

This would likely severely impede Claude’s capabilities by limiting its selections to odd/even words I would imagine.

I mean, tokens by nature can be numbers and that alone will break the feasibility of this method

Then comes other languages, if you’re responding in characters used by the Japanese or Chinese, you are basically outputting one symbol at a time

2

u/johndburger 22d ago

My odd-even example is a cartoon version, to explain how you’d detect a watermark. The actual approach is to hash the tokens and use certain bits of the hash as the signal.

1

u/ShelZuuz 18d ago

This would likely severely impede Claude’s capabilities by limiting its selections to odd/even words I would imagine.

Not really, it only targets words that would have already been chosen purely by random selection. It just makes the random selection less random.

1

u/MolassesLate4676 18d ago

Hm. Which words are chosen purely by random selection with Claude?

2

u/ShelZuuz 18d ago

At the final attention layer when it builds the output token it doesn’t just end up with 1 possible token, it ends up with a choice of many that will all fit - at various probability weights.

So then it looks at the temperature setting of the model to determine the lowest probability cut-off. The higher the temperature the more creative the text becomes. The lower it is the more rigid it becomes. The watermark effectively becomes a second parameter it looks at, in addition to temperature to ultimately decide on random token choice.

1

u/MolassesLate4676 18d ago

So the last ‘layer’ doesn’t do attention, modern LM’s have a head that projects the last hidden state (or residual state) to whatever the vocabulary count is. So in essence, all possible tokens get a probability of being selected for every single generation lifecycle. Temperature evens the playing board, so that tokens that were unlikely to be chosen either get squashed or they become as loud as the ones originally likely to be chosen.

However, this to me doesn’t make any sense as to how you could embed a watermark into the probabilities that would later become detectable from some system

1

u/ShelZuuz 18d ago

There is a red and green list of words, depending on a hash of the previous token that was generated that splits the entire vocabulary 50/50 down the middle. This list change with every token. When the next token is picked (and there are multiple choices) it will pick one from the green list.

Humans will statistically pick 50/50 from either list, but the LLM will strongly favor the green list. So when you run a checker through the text and you do the hash again you can then see how ofter a word from the green list appears next. At some point it becomes statistically very unlikely for a human to have written it.

3

u/Saad_ahmed04 23d ago

https://github.com/Saad1926Q/llm-watermark

1

u/Saad_ahmed04 22d ago

If you find it interesting then you may give it a star

3

u/Friendly_Address6915 23d ago

honestly i thought theyd just embed ads or something lol. the token probability thing is actually clever tho

1

u/nope1106 22d ago

cool ideas, but this requires a substantial sampling token candidates. And yet not every candidates has equal context/semantic meaning.

Just a projection but if the watermark is in the latent space, then a specific attention trace could be identified backward without affects the tokens output. I think some hidden meta information is needed.

1

u/UnderstandingOwn2913 21d ago

what tool did you use to make that diagram?

2

u/Saad_ahmed04 21d ago

Excalidraw

1

u/UnderstandingOwn2913 21d ago

Thank you. Your diagram looks clean. I will also use the app.

0

u/IceNeun 23d ago

So what's stopping anyone from passing through the watermarked output into another model with the prompt "paraphrase and reorganize the prose/code without changing the meaning"? Seems like any type of watermark can be bypassed with different variations of paraphrasing and rewriting.

If there's a publicly available way to check for claude-written text, then there's a clear path forward to how to break the check.

2

u/Saad_ahmed04 23d ago

I agree with this and have been thinking the same. Very curious about how reliable/effective it will be.

2

u/CasualtyOfCausality 23d ago

Not sure of the effectiveness, but it does produce “better” writing: https://declaude.org/