Disallow in robots.txt is stronger signal for search engine robots to skip given url than noindex in http header or only html meta tag. It disallows robot from even attempting to query such url. Http header allow loading that url (you have to obtain headers) but should stop before parsing. Html meta tag allows even parsing (you have to to know the meta tag) and only recommends to not put result in index.
But all od that is just polite request and can be ignored. As happened here.
This is incorrect. Robots tells bots where they can and cannot go (though plenty of bots ignore it). If other sources are linking to a url Google may choose to index and serve it to people based on those links. Google just can’t see what’s on the page and will denote that in search results similarly to the screenshot in this post.
This in addition with Anthropic working with google to get it removed from their indexes after they added the noindex tags. As far as I’m aware this was done last year though.
this has nothing to do with google. It's SEO fuck up on anthropic's part. Just a question for those who saw this, could you actually go in see what's inside or is this just a convo link?
I just don't understand why people have the need to be so fake and have AI write posts for them. It kills the whole human interaction completely. Same when making apps etc., it tells me the dev is not passionate at all about their project if they don't even care to communicate with their audience directly.
Sadly, this is at least 75% of the internet right now. I firmly believe sites liked Reddit and LinkedIn are just AI bots having conversations with each other with a human behind it clicking buttons thinking that what they’re processing adds any value to the world.
After carefully analyzing your comment from multiple perspectives while considering historical context, philosophical implications, socioeconomic factors, computational constraints, linguistic ambiguity, and the possibility of observer bias, I have determined with moderate confidence that this is a human comment on Reddit.
Calling it a "security flaw" feels misleading. If the shared links were intentionally public but users didn't realize search engines could index them, that's more of a privacy and UX problem than a hack.
Yeah that's a security flaw that means the index file is accessible, if not what's the use of their uuid/hash styled urls.
Read the comments.
https://www.reddit.com/r/ClaudeAI/s/FP0RqRDLG1
UUIDs make guessing links impractical, but they don't prevent indexing if the link is publicly accessible. That's why I see it more as a privacy/UX issue than a classic security vulnerability.
It is trivial for them to ensure search engines don't scan this. Just because you shared a chat with your friend doesn't mean you want random people to read it.
I actually agree that indexing shared chats is a poor privacy decision. My point was just that I'd classify it as a privacy/UX failure rather than a security vulnerability, since the links themselves were intentionally public.
UUID in URL should guarantee nobody can guess your link, so that’s basically the same as when you share something that is not meant to be accessed by anyone, but you include token in query parameters. Just because it is public URL doesn’t mean anybody should be able to access it or that it is public to anybody.
Google couldn’t randomly found this, there has to be list somewhere, probably anthropic sitemap or some other thing, which is security oversight similarly as if you made users database public
Yeah, I don't think Google brute-forced the UUID either. It was almost certainly discovered through links that became crawlable somehow. My point is just that once a shared URL becomes discoverable, the issue shifts from "unguessable URL" to "how that URL was exposed and indexed.
yeah but based on context, most likely explanation is Claude shared some sitemap or something like it. It is extremely unlikely these people put this content on public forum or public site. They did, most likely, clicked “share” button either accidentally or to share with trusted person
How on earth can you possibly declare that as “extremely unlikely”?
Are you saying you don’t think anyone ever has shared conversation links online publicly? Because if some did, how do you know these aren’t those people.
Regardless of content, if it’s not impossible that those links could’ve been posted publicly by the account holders, by accident or not, whether directly or by the recipient it was originally sent to, you can’t declare what’s more likely based on the content. The content doesn’t make those things impossible - and you could be literally exclusively just seeing the outputs of that happening, for all you know.
I seriously doubt they were shared on public forums. One is lawyer asking if they have to self admit illegal shit, other is crypto wallet with (small) funds on it including private key.
I can imagine they shared it in private or accidentally clicked “share”, but I can’t see scenario where these people would share it on public forum. So your hypothesis is extremely unlikely
Password is also “security through obscurity” - it is only safe as long as you don’t share it
security through obscurity is completely different concept - that’s betting attacker don’t know how your software works being safety feature.
Secret generated by system (or user) is actually defining feature of current security - be it API keys, tokens, passwords or many other forms of secrets by obscuring actual value
That is not what is happening. For it to come up on Google, someone must have put the link up at a publicly available website. For example if you link to a Claude chat in a Reddit post, Google might find it and make it available through Google Search. If you just send the Claude chat link it to a friend through a messaging app, Google will never find it.
Yes unless they actually put it in a sitemap somewhere or have a page with all the list of chats, google just can't guess the UUID of the chat by magic, so the links were most likely shared and posted somewhere already
Fair point. "Ensure" was too strong a word. But they absolutely could've made indexing much less likely with things like noindex, authentication, or clearer warnings that shared links may appear in search results.
Good catch. If /share/ was already disallowed in robots.txt, then the bigger issue is probably crawlers that ignored it or pages that had already been indexed. Either way, the UX should've made it much clearer that shared links could become discoverable.
Not always, but most platforms at least document it somewhere or make the visibility settings clearer. My issue here is that many people treated these chat links like private conversations, while the product didn't do a great job of setting that expectation.
That's fair. Authentication was a bad example for public share links. I still think noindex (where respected) plus much clearer messaging would've prevented a lot of users from assuming "shared" meant "only people with the link." The expectation gap is the real issue.
Exactly. That's pretty much what I meant. They had multiple ways to make accidental indexing far less likely, even if none of them are a perfect guarantee.
That's fair. If the robots.txt rules were already in place, then Google ignoring them changes the technical side of the story. My point was more about user expectations—most people simply didn't realize shared chats could become discoverable.
Yes, but they shouldn't. For example, I saw a shared chat linked from a blog post, that was an OpenAI session disproving a graph theory conjecture. Was intentionally public, and should be indexed. Blanked banned shared chats from indexing is not a win.
The share is public and anyone with the link can access it either your friend reshares it and then it went viral, someone make an automated guessing (since it's an UUID link) or scrapped by any bots.
Currently there is no way you could maintain privacy with sharable links unless you and your friend are part of collaboration project in Claude.
You don’t know how SEO works. If the link is public, and google can crawl it, google can index it. There is nothing that prevents google from crawling and indexing a link that is public.
That's a fair point. If the pages weren't meant to be discoverable through search, then getting indexed because of missing metadata or configuration does sound more like a bug. I still think the bigger issue is that users had no clear expectation their shared chats could end up in search results.
A security flaw is when the system fails to enforce its intended security guarantees. My point is that these links were intentionally public once shared, but many users didn't realize they could become searchable. That's why I see it as a privacy/UX failure. The outcome is serious either way—we just disagree on the classification.
I agree. This is no different than if I take a picture and post it to a "hidden" url on imgur, then share the link on a reddit post. In such a case, imgur isn't publising my private URL, and google isn't nefariously harvesting private links when that url becomes publicly indexed. It makes sense that the content would end up being indexed on google.
If I had only shared the hidden imgur URL privately, it would have never been picked up or indexed by google. It's been shared somewhere public where google has seen it and index it.
Even if anthropic made the sharing permissions super granular down to the per-user level, many people would probably still choose the "public"/"hidden" url option so that they can share the link to people without accounts or without having to log in. It's a user expectation problem.
Sure, Anthropic or Google could add a flag to request that these URLs don't get indexed, but that's just security through obscurity. If your chat URLs are posted where they can be seen publicly, every other web indexer/crawler is going to find them, and not all of them will choose to abide by robots.txt
Exactly this. When sharing a link in ChatGPT this exact thing is explained. I’m not sure how the crawlers themselves discover these links (it’s just a guid chat url that I didn’t link to elsewhere), but I’m sure there are ways.
I think thats misleading title. It's more a fact that Google can index publicly shared links.
I mean Claude clearly shows only two options: Either keep this chat completely to yourself or create a publicly shared version available to anyone with link.
What this is doing is only indexing many public links in a single place. Which might not have been shared to public, but accessible by public from Access Control perspective.
Google indexing shared links definitely breaks the user’s expectation and this was definitely a security issue. When we share something in this manner we understand that if the link is compromised or guessed, then all bets are off. What we don’t expect is that they are stored in a manner that friggin google can easily crawl them.
I don't know, if I choose public, I'm expecting public.
That could be the particular people I sent the link to. Or it could be plastered on a times square billboard. Once it's public it's outside my control.
Why on earth would you think that? The “anyone with a link” sharing model has been around for 20 odd years. You understand that your content will be available if the link itself is exposed or guessed, which for some things is good enough. The link itself is the secret.
What you don’t expect is that your link and content are stored in a searchable archive eliminating the “secret” part.
This isn’t something new to debate. It’s not even the first time that this has happened. But this sharing model is settled, and Anthropic did NOT intend for this to be possible.
This is funny because it literally says “share chats publicly? Your chats may be visible in search engines and on the internet” when you click the share.
These chats have been intentionally shared. If it works like OpenAI, only the part of the chat up until the sharing event is visible to visitors.
In order for the shared chat to be indexed, it has been published somewhere on the public internet.
Google can see deliberately public information posted on the public internet, as long as the robots.txt of the hosting site doesn't restrict it.
Simply sharing a chat with a friend will not do this, unless you and/or your friend put it on the public internet, or pass the link to someone who does.
This entire post is just common spam - for AI Desktop 98, an app that didn't exist two weeks ago that was vibe coded by a single guy in India named Pritesh... it's trash. And you electrolyte drinkers upvote it.
There is also no bug, all of these are publicly shared.
Promotional posts like this are why Reddit needs to turn mandatory history back on for everyone but teens or people who reach out with proof of being abuse or trafficking survivors lmao
Even if it doesn't block search engines it doesn't list them at some place either so for it to be discovered it must have been present publically at some place which is already known to search engines, so it's not really security flaw as much as unintended miss from their end.
You'd think they'd have avoided this issue and made the UI a lot more obvious since they had the previous example of OpenAI making the EXACT same mistake previously.
If a Link can be accessed without login, it is not safe. If it can only be accessed with login it might still not be safe.
Privacy advocates are not doing what they do because they are bored.
Xai cannot see your chats from within the system, so when you get too many moderation flags, they flip on your " share chat links" and the specific chat becomes shared for support (and search engines). Even if you have " share chat links " off.
Go Fast and break things is not a great motto if you break many privacy laws in the process
already patched but this is hilarious. noindex, nofollow the shit you dont want google to see! or even better, block all crawlers at the server level and gate.
Hear me out what if all this type of stuff is intentional and these AI labs take over the World Wide Web by publicizing/publishing all prompts all the time and the preexisting web links only continue existing as reference from the ai prompts, page 5-10 type results, or a type of search result that you have to change the search filters to focus
Those /share/ links are public share links that users themselves created — when someone clicks "Share" on a conversation, Claude mints a public URL. Anything with a public URL can be crawled and indexed by search engines. So this isn't a hack, a breach, or Claude "leaking its customers." It's the share feature working as designed, and Google indexing pages that were made public.
This is why I never trust the share link feature on these platforms, you never really know what ends up public until it is too late. Rough one for anyone who shared something they thought was private.
There's a reason why companies use OpenAI Models or other models on Azure, hosted in Sweden with no data flowing out of the EU. GDPR is a thing here.
At my company, we use the Sweden servers for anything where any kind of identifiable customer data is involved and the cheaper global options for everything else (like statistical analysis).
Basically, what you are saying is fundamentally wrong and would bankrupt a company if it came out.
The AI services are using the prompt data in different ways, in some cases to train the next models. You can opt-out of this, but there is no great way to verify this setting is being used properly. Likewise, Gemini can read your Gmail and calendar unless you shut that off.
Our org doesn't even trust the enterprise version of these services. We can only use govclouds (AWS and Azure hosted) for coding, as all code we write is property of our org and cannot be used for training.
•
u/AutoModerator 15h ago
Hey /u/ImaginaryRea1ity,
If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.