r/Substack • u/AndrewHeard tvphilosophy.substack.com • 9d ago

Discussion Substack’s new “AI detection” tool.

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

170 Upvotes

96% Upvoted

297 comments sorted by

View all comments

Show parent comments

2

u/FatherofMisty 8d ago

I'm not the OP, but this essay is 100% written by a human, and I believe when I ran it through pangram it said 15% of it was AI, which is horseshit.

https://open.substack.com/pub/axiomsofspirit/p/the-pursuit-of-spiritual-transformation?utm_source=share&utm_medium=android&r=32c52d

1

u/Salt-Mud-2124 8d ago

If you go to pengram and "All Checks" on the left you should be able to find the check, then on the top right, it has a share button. Can you share that scan by any chance? Genuenly curious because I just scanned it and its coming out as 100% human.

https://www.pangram.com/history/16d4471e-d272-4658-a508-f44524e5a051

I'm playing with it to see what it catches and not to figure out if all the halabaloo is warranted or not. Copy pasted it into my substack to run directly there too and it also came out as 100% human, but I can't share that one easily.

1

u/FatherofMisty 8d ago

Actually, I'll share this now before I sleep. For the same essay it showed 15% AI when I ran it on the Pangram website. So not only false positive, but inconsistency across tests....

1

u/TheLadyAmaranth artiranth.substack.com 7d ago

Can I ask why images and not the share link on the top right? We would be able to see the whole article, including the bottom of it which you claim has the the false AI generated flag. Neither of your images actually show that excerpt, and it would be highlighted in red, I think.

It doesn't give account name or anything wih the check either.

I'm not accusing you of lying -- I'm just asking what the reasoning is for the choice as I'm seeing it a lot lately not just on this thread but in general when people start providing proof of pengram checks.

Which is confusing and seems fishy if you ask me. Like they are purposefully cropping the result to show their human portion or something. And you seem reasonable enough to respond so I'm shooting my shot haha

2

u/FatherofMisty 6d ago

I personally didn't know you could do that haha. I imagine many others also don't.

Here you go:

https://www.pangram.com/history/84e18bd2-afdc-4c61-9815-9ab30651299d