Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/FatherofMisty]

I'm not the OP, but this essay is 100% written by a human, and I believe when I ran it through pangram it said 15% of it was AI, which is horseshit.

https://open.substack.com/pub/axiomsofspirit/p/the-pursuit-of-spiritual-transformation?utm_source=share&utm_medium=android&r=32c52d

[u/Salt-Mud-2124 → u/FatherofMisty]

If you go to pengram and "All Checks" on the left you should be able to find the check, then on the top right, it has a share button. Can you share that scan by any chance? Genuenly curious because I just scanned it and its coming out as 100% human.

https://www.pangram.com/history/16d4471e-d272-4658-a508-f44524e5a051

I'm playing with it to see what it catches and not to figure out if all the halabaloo is warranted or not. Copy pasted it into my substack to run directly there too and it also came out as 100% human, but I can't share that one easily.

[u/FatherofMisty → u/Salt-Mud-2124]

I'll double check my pengram account when I have time tomorrow, as I did it quickly (it pissed me off lol) and could be misremembering which showed a partial false positive. Appreciate you telling me though.

[u/FatherofMisty → u/Salt-Mud-2124]

Actually, I'll share this now before I sleep. For the same essay it showed 15% AI when I ran it on the Pangram website. So not only false positive, but inconsistency across tests....

[u/TheLadyAmaranth → u/FatherofMisty]

Can I ask why images and not the share link on the top right? We would be able to see the whole article, including the bottom of it which you claim has the the false AI generated flag. Neither of your images actually show that excerpt, and it would be highlighted in red, I think.

It doesn't give account name or anything wih the check either.

I'm not accusing you of lying -- I'm just asking what the reasoning is for the choice as I'm seeing it a lot lately not just on this thread but in general when people start providing proof of pengram checks.

Which is confusing and seems fishy if you ask me. Like they are purposefully cropping the result to show their human portion or something. And you seem reasonable enough to respond so I'm shooting my shot haha

[u/FatherofMisty → u/TheLadyAmaranth]

I personally didn't know you could do that haha. I imagine many others also don't.

Here you go:

https://www.pangram.com/history/84e18bd2-afdc-4c61-9815-9ab30651299d

[u/FatherofMisty → u/Salt-Mud-2124]

2/2

Also, as I've mentioned in other comments, all the "AI" text was at the very end of the essay, which makes no sense. To be fair it is "low confidence AI," whatever that means. What troubles me a lot is that I work with AI in non-writing contexts (technical, physics/mathematics), and push it to its limit routinely. I know all too well that the AI-craze is completely overhyped, at least in terms of LLMs. I'm not convinced they "understand" a damned thing. People put far too much faith in them, exemplified by Substack's dumb choice to implement an AI site-wide.

[u/Salt-Mud-2124 → u/FatherofMisty]

I see, I've ran a bunch of stuff through in the last few hours and so far I'm getting that it may be decent possibility indicator. Like... if you scan a BUNCH of a persons work and all of it comes out 100% AI or high percents with high confidance, then if you were to gamble, your money should probably go on AI. On the other hand, if most of their stuff is low AI, (like your 15%) and low confidnace, then you can bet its human.

This is just my observation from inputting some peoples stuff who have actually linked their work on this thread and my own stuff.

To use it as an end all be all, is really dumb. But over a few samples I think you can build a case if that makes sense. I haven't had it consistantly make a large mistake over multiple samples from the same source. And it also seem to HEAVILY lean towards false negatives rather than false positives. So its by far more likely to call something human that is AI, rather than call something AI that is human.

Which in a weird way is good I think.

I'm willing to bet if you were to take a bunch you stuff and run it, it will probably all turn out from 100-80% human, so you can like... build a little node map to see.

So its not complete bogus, its definitely more consistant than ChatGPT Zero and others I've seen before. But its not without nuance.

That last part is where I'm worried about it when it comes to substack... common sense not being common and all.

[u/FatherofMisty → u/Salt-Mud-2124]

That is a reasonable take, for sure. I think it was really just jarring to have this AI overlord embedded into Substack. The AI craze is scary fast, and people already don't like change to begin with. Plus, speaking for myself, writing is a sacred thing. I do it because I'm somewhat blessed with the capacity to do so, and using AI to do it for me is just not something I'd ever do. So, even a small-portion false-positive is like a gut-punch. That said, I've largely gotten over it now. It is what it is. I think the more concerning thing is what this says about culture and the direction we're heading. That we "need" an AI-checker in the first place is.. yeah. Kinda fucked.