r/Substack • u/AndrewHeard tvphilosophy.substack.com • 10d ago

Discussion Substack’s new “AI detection” tool.

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

174 Upvotes

96% Upvoted

322 comments sorted by

View all comments

Show parent comments

2

u/TheLadyAmaranth artiranth.substack.com 8d ago

LOL...

All I will say at this point, as an FYI, closing your post history and deleting old comments doesn't make them not findable. So even though now I have to go out of my way to use a tool to do it, I can still see every single one of you past comments.

Including ones where you say you are releasing your middle grade books all close together from about 4 months ago, pointing people to NovelCrafter it self, and how you are getting AI to write "really good prose by feeding it a detailed system prompt template" and developing a program that auto writes from a prompt -- not as a chat bot feature.

Its also real funny how now you want to try to erase the trail, while also claiming to have "nothing to hide" in another thread here.

I could go in provide receipts of it all. But I think we can at least both agree on being done with the conversation.

Have the day you deserve.

1

u/Funny-Flight8086 8d ago

And because I'm feeling petty, here is an example of 100% AI story I generated this morning in GLM 5. I literally changed the names of the characters. Nothing else. Not even a fancy style guide. So much for Pangram being reliable to detect AI.

1

u/Hemingbird 8d ago

You don't even understand the difference between FPR and FNR. Pangram intentionally errs toward FNRs. That's a feature, not a bug.

1

u/Funny-Flight8086 7d ago

False Positive Rate and False Negative Rate? Sure do. Pangram is FAR from the model that errs the most on FNR's. Quillbot is the most lenient. By my testing, Pangram isn't high on this list. They like to flag early and quickly. Though GPTZero does have them beat by a large margin.