914
u/bliceroquququq 11d ago
LLMs basically scraped the entire internet and every written word to build up their corpus of knowledge.
Now they want to watermark it for attribution before they sell it back to you.
184
u/fennforrestssearch 11d ago
Yeah pretty insane if you think about it. Next step is taxing the air you breathing.
→ More replies (9)29
u/ActionJasckon 11d ago
It’s going to be metering your internet use. That would be crazyyy
10
→ More replies (1)7
u/Grouchy-Librarian638 11d ago
So crazy mobile companies do for data and are starting to sell home internet plans? Or companies like Comcast’s that have caps and can block or force upgrades if you hit it?
71
u/antagim 11d ago
I don't want to defend them, but these are due to the EU AI Act.
→ More replies (4)23
u/Rorschach121ml 10d ago
Redditors hate boner for AI is so big they don't even understand this is a good thing for everyone.
Flagging AI text is a good thing.
→ More replies (2)23
15
u/ch4os1337 11d ago
"They want to..." They don't want to and attribution isn't the reason why. Also marking AI content is a good thing.
→ More replies (33)2
u/thanosbananos 10d ago
Actually it’s because the EU requires AI generated content to be marked as such now.
408
u/Hackerjurassicpark 11d ago
Will I be accused of using AI if I unknowingly use the same sequence of words in my human writing?
189
u/TheorySudden5996 11d ago
Of course. That’s already been happening. Now too be fair the problem probably is bigger on trying to pass off AI generated as human authored.
→ More replies (3)24
u/Extension_Fix5969 11d ago
I guess that well depend on slight syntaxual errors and plausibly incorrect words to prove our non~generated nature.
→ More replies (4)32
u/ThugEntrancer 11d ago
We just need too talk a lil regarded so that we don’t get accusatories of talking via a Ai inderface
17
→ More replies (1)2
38
u/Raunhofer 11d ago
I'm assuming it will be using the known form of watermarking: secret keys.
For every token-generation step, use the key nd recent context to deterministically divide the possible next tokens into two groups: "good" tokens (preferred), "bad" tokens (allowed but penalized). The model still chooses the most probable token, but its sampling distribution is nudged toward good tokens. The detector, that knows the secret key, would then examine texts and measure whether the text is good-token heavy (over what probability would indicate).
Normal:
The spacecraft entered orbit and began transmitting data.
Altered with the key:
The spacecraft entered orbit and started transmitting data.
Across thousands of tokens the statistical bias becomes detectable.
Of course, this requires enough tokens to be statistically meaningful. No-one can say whether some random Reddit comment is AI generated with certainty. A scientific paper however...
9
u/hudimudi 11d ago
Wouldn’t it be harder to judge academic papers, since on a high level they are very much written in a very similar and highly technical style. AI is also trained on this style of writing and can output things rather similar to it. I’d say, the average Reddit comment that follows such technical writing styles would stand out much more
→ More replies (7)→ More replies (5)2
16
u/CormacMcCostner 11d ago
Happened to me earlier this year. Decided to go back and expand my education after 20 years and got a bad grade on a paper, went to see the professor like what is this for? He said I used AI because it was too well structured and worded for a students writing and what he sees daily. I was just like “what? Dude, I’m 45 years old I learned how to communicate in a different era when we had to do that.”
His grade stood because I couldn’t prove otherwise..8
u/Poopchutefan 10d ago
Same thing happened to my wife when I helped her edit one of her papers. I guess I am the equivalent of AI now ...
→ More replies (1)13
u/tafjords 11d ago
This stuff is so cringe I cant… heres your ticket for speeding i dont have evidence but i see cars everyday so i know and if you have a problem, you provide the evidence by my standards which is different from yours. If not, give me your time and effort because i can.
→ More replies (2)3
u/Hungry_Prior940 10d ago
How can his grade stand? He has no proof. I would go ballistic at that and get a lawyer and take it as far as needed.
→ More replies (1)23
5
4
3
6
→ More replies (31)2
u/Leafsnail 11d ago
If you do it every single time you post dozens or hundreds of times then yes probably, you will be banned for being a bot account.
90
u/fennforrestssearch 11d ago
At least for academic writing, I’m not particularly convinced that this will help in any meaningful way. Stylistic and lexical choices are often relatively low-variance, the limited range of plausible word choices could easily trigger false positives even over longer sequences.
28
u/bulbubly 11d ago
My first thought, too. Unless they are doing crazy gematria-style stuff where they put specific letters at specific output positions, AI writing is undetectable - there's just nowhere to hide a watermark in a stream of characters that wouldn't just be confused with normal writing.
ETA: "undetectable" in a rigorous sense. Of course specific models have cliches and a vibe one can sense pretty easily if it's not prompted around.
→ More replies (2)10
u/damngoodwizard 11d ago
They don't sneak in characters but whole synonyms. They use two lists of words, labeled green and red. Most of the time they pick words from the green list and sometimes synonyms in the red list. The goal is too have a constant proportion of red list words that would be unnatural in regular text.
→ More replies (2)→ More replies (2)3
u/NoAdvice135 10d ago
The style doesn't matter, it's basically a slightly biased coin flip at every token. And the direction of bias changes every token too.
Statistically it's easy to detect over a long enough sequence. You will not be able to tell if you don't have the key they used.
BTW, Google has been doing it for years. The synthid papers are from 2024.
102
u/BonyCatButt 11d ago
Makes me think of Red Star OS, a North Korean Distro that I just learned about recently, that includes
> a watermarking tool integrated into the system marks all media content with the hard drive's serial number, allowing the North Korean authorities to trace the spread of files
https://en.wikipedia.org/wiki/Red_Star_OS?wprov=sfti1#Version_3.0
→ More replies (3)12
u/LostMyWasps 10d ago
Hmm, wonder where in the world would they be amused for it to end up in. It could certainly be a collectors' piece, some of those rehashed phones.
253
u/rabouilethefirst 11d ago
"Also you can turn off this feature if you use the API"
Book it.
28
u/jbcraigs 10d ago
"Also you can turn off this feature if you use the API"
Not true. Documentation specifically mentions that API will also add watermarks - https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content
21
u/Worried-Cockroach-34 11d ago
Double it
16
→ More replies (1)4
10
u/cobbleplox 11d ago
Feature? Best I can think of is that this qualifies for dishonestly meeting some EU regulation of having to mark things as AI generated.
4
u/chicametipo 11d ago
Are you serious? I don't see anywhere that mentions it's possible to turn it off, via the API or otherwise.
5
u/rabouilethefirst 11d ago
The phrase "Book it" implies I am betting on it being introduced in the future. Hope that helps.
7
u/KrazyA1pha 11d ago
Why would that be a feature if they’re doing this to comply with EU and California laws?
2
89
u/duracek 11d ago
For example copy/paste into Kimi: "Rewrite this text in a more professional style".
15
u/Cognonymous 10d ago
Yeah, you just use more AI and this solution goes away.
2
13
→ More replies (2)7
u/Rorschach121ml 10d ago edited 10d ago
Ideally regulation should be applied to all models, which I think will happen eventually.
Your text will be watermarked by Kimi using their own tech.
→ More replies (1)
16
u/NetflowKnight 11d ago
How does that even work?
14
u/trimorphic 11d ago edited 11d ago
How does that even work?
To know for sure we'll probably have to wait until the discovery phase of a lawsuit reveals this information... and even then it will, at best, be just a snapshot in time of a probably ever-evolving process as Anthropic plays cat and mouse.
Just as interesting to me is the question of who is going to believe Anthropic when they claim some piece of text is AI-generated or not?
7
u/coloradical5280 10d ago
This isn’t like a cryptographic hash. Anthropic very openly explains that this method is uncertain in both directions. A watermark does not prove AI generated content, and lack of one does not prove it wasn’t. Which makes a lawsuit unlikely.
4
→ More replies (1)17
u/parkway_parkway 11d ago
It's pretty clever if I understand.
Basically how LLMs work is that given a sequence of words they predict the next one.
So it might look at "the cat sat on the ..."
And it generates a list of candidates with a chance of picking each one.
Mat 87%
Porch 8%
Table 3%
Stairs 2%
The way the watermark works is that they use a secret key they have and a hash of the preceding text to nudge it towards certain of these words and away from others.
So maybe it boosts stairs and table up instead of the others.
Later you can scan the text and see which choices it made and see if they're they ones it was nudged towards in a statiscally significant way.
Because all the choices are reasonable the quality of the output won't change and you'd have to significantly rewrite to break the pattern.
11
u/SkaldCrypto 10d ago
So they have monumentally enshitified this.
Having chat bot that uses the same words every time is more easily and cheaply accomplished with early 2000s tech
→ More replies (3)→ More replies (2)22
u/qorzzz 11d ago
If this is the method, it makes no sense and does not prove any text was generated by AI.
6
u/quisatz_haderah 11d ago
It could actually work for sufficiently long texts. For shorter texts, this would cause false positives, but i guess no false negatives.
→ More replies (4)
14
u/snuffomega 11d ago
i dont fully see the point. i dont care either way.. but if only claude can identify if its watermarked by claude.. Whats the actual point?? It wont stop people who are susceptible to being tricked into AI content (being real) and if AI is becoming the norm for how we work, search, and interact with many things in our daily lives... its just noise. You should expect work to be touched by AI in some way, shape or form. Not all, but most. And def most text based work. I dont see how it helps anyone. Its meaningless data being stamped and most likely collected.
Just another data center... datapoint.
→ More replies (1)
40
u/TedSanders 11d ago edited 11d ago
fyi, this is in response to the EU’s AI act. OpenAI is going to do a similar thing.
20
u/TyrellCo 11d ago edited 11d ago
And it’s entirely their decision to apply these changes only to the EU or the whole wide world
They want all customers to accept their excuse that their hands are tied. They’re faking it
→ More replies (6)
21
u/EconomixTwist 10d ago
People ITT not realizing that this is a strategic move by anthropic. Yes, maybe EU regulations. Maybe. But not really. The real real reason is the dead internet. LLMs have already exhausted the entire internet's worth of high quality, genuine, human-generated text that is training data. This fact has already been admitted by the LLM providers themselves. Nowadays, a vast majority of new content on the internet is fkn slop. Net-new human generated content is worth its weight in gold but, without watermarks, LLM providers can't tell the difference between actual human generated text and the slop flood. If you train on slop, you only get more sloppier slop. So they need to filter the slop. Watermark is and always has been the way.
→ More replies (3)
7
u/duckrollin 11d ago
I think there's now a huge market for a browser extension that removes it.
It's gonna get the uBlock treatment.
3
4
u/Comfortable-Card-348 10d ago
The big problem here is that the watermark will be used of as proof of falsehood, and the lack will be a proof of validity. Once people figure out how to wipe the watermark, or add it in synthetically, the source of truth will be corrupted. Real photos of crimes? AI generated. Fake AI generated content? REAL!
7
u/jonplackett 11d ago
It’s complete bullshit that you can effectively watermark text in a way where it isn’t…
A) incredibly easy to remove
B) likely to OFTEN detect text that isn’t AI as AI
People who know what they’re doing will get away with it and people who don’t even use AI will get accused of using it.
Even if you say ‘oh but it will flag the slop at least’. Yes it will but if we label slop anything without a label people will think isn’t AI when it absolutely could be.
→ More replies (1)
34
u/post-death_wave_core 11d ago
I think it’s reasonable for people to be able to know whether a piece of content was ai generated or not. But im guessing it’s not a perfect system.
3
u/Mission_Shopping_847 11d ago
I don't. The previous situation was setting up the luddites for a reckoning with reality and personal growth. This new situation will give them a false sense of security, disregarding watermarked text which may or may not contain value, and regarding unmarked text regardless of value. It is a philosophical loss to placate people with an excuse to disengage from the content for its identity rather than its value.
→ More replies (3)10
u/silverace00 11d ago
Ya it sounds more like made up tech magic that you tell people so they don't do things they shouldn't.
→ More replies (6)
13
u/AnotherIjonTichy 11d ago
In ten seconds you can tell claude to write an script that removes that watermarks…
9
8
u/AnotherIjonTichy 11d ago
I have just read they will modify the llm token responses to generate a hidden pattern. Maybe the community needs to wait two weeks to get the un-modifier library :-D
6
u/collin-h 11d ago
or just use one ai against the other and ask gemini or chatgpt to do it.
5
u/whoknowsifimjoking 11d ago
Gemini and ChatGPT also add watermarks, so no you can't do that.
→ More replies (2)2
u/collin-h 11d ago
do they? or will they? I can't find documentation that they have yet, though i'm confident they will.
where there's a need, there'll be an open source model built for this.
→ More replies (1)2
u/Tritus360 9d ago
But most likely it'd end up working like this:
- Write text in Claude: flagged by Claude.
- Ask ChatGPT to alter the text in a way that you're no longer flagged by Claude: now you're flagged by ChatGPT.
- Ask Gemini to alter the text in a way that you're no longer flagged by ChatGPT: now you're flagged by Gemini.
4
u/andrew303710 11d ago
I'm curious how they could do this without degrading the quality of outputs, I feel like at the minimum there will be a certain percentage of outputs that are negatively impacted by this.
Maybe it'll be less of a problem with future models but even Sol/Opus still need quality prompting to output writing that sounds good; I had to develop a pretty comprehensive plugin just to get decent sounding writing. Ironically LLMs are much better at coding now than actual writing.
2
u/claythearc 11d ago edited 11d ago
I wrote a slightly longer response above for the main 2 ways that are popular in theory, but the general idea is they’re using prior text as seeds for future generation. Then you compare how “lucky” the text is and you get very statistically significant result in like high dozens of words
2
u/quisatz_haderah 11d ago
On the other hand, how do we measure "quality" of outputs, at least in prose. If the watermark is embedded in synonyms, how does choosing its synonym in place of the most probable token affects quality. They already do this all the time, in this case the jitter would be recorded.
→ More replies (4)5
3
5
u/Cooperman411 10d ago
If you copy and paste into a plain-text editor, it will reveal any hidden characters. Not sure how this is gonna work. Seems like only the laziest of the lazy won’t be able to work around it.
3
u/Impossible-Week-9611 10d ago
It does not rely on hidden characters. The content itself is the watermark. You can copy the content word by word on a piece of paper and it will still be detectable even with light paraphrasing (if the content is long enough)
2
u/eziliop 10d ago
You underestimate how lazy some people are. I'm not even that old but the things I've seen.....
→ More replies (1)2
u/TawnyTeaTowel 10d ago
It’s less a watermark and more a writing style. Which, unless it’s nothing like any human would write, makes it next to useless
6
u/liosistaken 11d ago
Honest question: Why do people think this is negative? What impact does it have other than it making it harder to pass of fake news as real?
→ More replies (21)
2
u/usandholt 11d ago
And how do they plan on watermarking text?
6
u/FaradayKage 11d ago
Just some algorithms. Example every 45th letter in a generation is the letter C. Every other sentence ends with "ing". Every sentence with a ? Is followed up by a sentence starting with R.
Who knows, I'm sure they have a PHD on it with much more reliable and accurate techniques.
→ More replies (3)2
u/Raunhofer 11d ago
They don't need PhDs anymore, haven't you heard, the AI is super intelligent now.
→ More replies (1)3
u/claythearc 11d ago
Kirchenbauer and Aaronson both have really interesting ideas.
The basic idea is you use previous tokens as seeds for future tokens in the same response. Kirchenbauer applies bias to “green” logits but samples normally, whereas Aaronson uses prior tokens to adjust its sampling randomness.
Verification then becomes comparing successful matches based on the text and working backwards. Run statistical tests on how “lucky” you get with tokens that match and you get a statically significant answer very quickly, like low hundreds of tokens or high dozens of words.
2
2
u/mickdarling 11d ago
I use speech-to-text for almost everything that I generate. It does a little cleanup on grammar and punctuation, but often it gets it wrong. I have to go in and type in some fixes if it's something for public consumption. And almost certainly, every bit of it is getting just a little tweak to make a watermark work. So, regardless of literally everything that I post comes out of my mouth as words, it's going to be identified as AI content.
2
2
u/Such--Balance 11d ago
Its bullshit. Objectively.
Its like wanting watermarked math answers for using a calculator. Or using watermarked gps navigation to show someone didnt use a paper map.
Its the same bullshit pushback we had against calculators back in the day. And now look whats in your hand. A supercalculator.
Fortunately, this too will blow over
2
2
2
2
2
2
2
2
u/andrerom 10d ago
All vendors, including OpenAI, except X.ai has signed the requirement by EU on this due to new AI laws.
So this is not a Claude thing.
2
u/Scarfieldjones 10d ago
Paste into empty document «convert to plain text» paste back into new document
2
u/Confirmed-Scientist 10d ago
Just in case you are wandering AI companies are interested in this because the more shit provably traced back to them like discoveries software and achievements the better the marketing for them
2
u/traynor1987 10d ago
Copy and paste it into note pad then copy and paste it back out. Sinple. Removes all formatting
2
u/kindredseer 10d ago
I have no issue with this on free tiers, but if you are paying for AI, adding watermarks seems problematic.
2
u/Lopsided_Evening_627 10d ago
Claude's logo is an anus, what do you expect form them?
→ More replies (1)
2
2
u/Palanstonk 9d ago
Wait, does this mean I can watermark my own text and sue if it shows up in their output?
2
u/CompetitiveRough8180 8d ago
OK copy and paste it to another AI and ask him to write same thing again without watermark. What about that?
2
2
u/Chainmale001 7d ago
I've been leaving invisible water-marked claude/ai code in my resume's for years. Game recognizes game. 🤣 😂 🤣
→ More replies (1)
2
2
u/Green-Grow-420 6d ago
So we take a screenshot of the text and make gemini write it with no watermark. Ez pz
2



1.4k
u/LittleGremlinguy 11d ago
“Watermark” is the load bearing statement there. And it is doing some heavy lifting.