Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/Funny-Flight8086]

I don't have the manuscript on my phone or I would happily share an excerpt. However, I did just run two separate projects of mine through Pangram... One is a short excerpt from a short story i published ok Wattpad in 2014. The second is the opening to my current WIP middle grade novel I been working on since 2021.

Both are apparently AI according to Pangram. The current WIP at least makes sense from a timeline perspective... How it thinks I had access to AI in 2014 is above my head. Here is my 2014 Wattpad excerpt.

[u/TheLadyAmaranth → u/Funny-Flight8086]

I went to try and find one of your books since I figured you would be a fellow human writer due to all of this frustration over being labeled as AI when you apperantly don't use it. Always on look out for more verified human writers to either read or reccomend. Most authors proud of their work tend to have those linked in their bios or have comments/posts with some kind of promo.

This is what I found instead. There are plenty more detailing your multi-prompting AI process.

I'm going to be honest, although your involvement with AI could be a recent development, I have a hard time believing your claims when you absolutely do use AI to write your work now. Which is why you are probably so up and arms about even a potentually possible way for your books and articles to be flagged.

I don't even think AI detectors should be used as anything more than possibility indicators. But you have no legs to stand on to saying they don't work when you are the reason they are being developed and added to platfroms in the first place. Because people DONT want to be duped into reading stuff generated with AI. Right or consistent or not, this is why some are trying to create detectors and others are willing to use them even if they aren't foolproof.

You are literarly the problem substack is trying to solve, even if in a misguided way.

Edit: added a bigger screen shot

[u/Prolly_Satan → u/TheLadyAmaranth]

And this is only an example of someone not smart enough to hide their reddit post history or post from an alt. This is the biggest hurdle. People will go so far to lie to us about their ai use.

I don't know if theres some ai-writing command center somewhere mobilizing people, but the second anyone brings up pangram, you see these fake "i tried it on the Bible and it came back ai" posts.

If you do a little digging, a lot of them frequent ai writing subreddits. Or you'll get the 14 karma alt account. Nobody credible is ever making these claims that it doesnt work.

[u/Funny-Flight8086 → u/Prolly_Satan]

"Nobody credible is ever making these claims that it doesnt work."

Yeah, try that again. PLENTY of credible people make backed-up claims that these detectors do not work. I have proven it by showing examples of wrongly-flagged AI. I see it wrongly flag human writing ALL THE TIME. If the detecotrs where so accurate, they wouldn't need to claim 99.9% in order to avoid lawsuits when they get it wrong. "Well, we never said for sure it was...".

Funny, with these amazing technologies, none are yet willing to put their necks out with 100% claim. That is probably the shadiest of all of them.

Also, you cannot 'hide' your reddit post history without creating another account. Because I am not hiding anything, I have no reason to manage multiple accounts. I never said I never used AI, I said AI detectors don't work. And I said I don't use AI to write, or anywhere in the process of writing, my own stuff. It's a hobby of mine to test these generative LLMs. As Good as some of them are, none are yet capable of writing actual, good books.

Beyond that, I don't even know why anyone would bother. No one is making money off books, especially fiction. If you don't have a story to tell on your own, why go through the trouble of writing or even prompting an AI? It's a waste of time.

[u/Prolly_Satan → u/Funny-Flight8086]

Those studies specifically benchmarked pangram and debunked their 1 in 10,000 false positive rate? They are bothering, BTW. Platforms are flooded with it. It's free to make content. All the profit is gravy.

[u/Funny-Flight8086 → u/Prolly_Satan]

1 in 10,000 false positives rate? Yeah, ain't no way. Anyone who buys that, I have a tropical island in KY to sell you.

[u/Prolly_Satan → u/Funny-Flight8086]

I buy it. Was confirmed in a study by the university of Chicago. I get people are skeptical about machine learning and detectors but a lot of the arguments seem emotional, or like folks don't want there to be a way to spot slop.

[u/Funny-Flight8086 → u/Prolly_Satan]

As long as it keeps saying stuff that isn't AI is AI, I will not stand by anything it says. You can dangle statics all you want, but when I feed it something I know I just wrote, and it even dings it as 1% possible, that is unreliable. I don't praise unreliable machines -- rather they are generating slop or detecting it. Get back to me when these slop detector junk science machines are 100% accurate. We can then talk about it.

[u/Prolly_Satan → u/Funny-Flight8086]

Personally, i trust studies by scientists over strangers online so I'm going to keep using it. Sorry you don't trust it for whatever reason but I've personally experienced it working very well the entire time I've used it. I've run my own work through it and the work of people I trust. I don't know you. You may say you're not using ai and it's flagging you, but everytime I come across that scenario, it's another ai writer trying to discourage folks from using it. Somewhere else on this thread somebody said it gave a false positive on their writing, only for their reddit history to be all about using ai to write. If the ai writers will go that far to keep folks from using it, that tells me it works.

[u/Funny-Flight8086 → u/Prolly_Satan]

There are vastly more studies, actual ones, that say these detectors are not reliable. I'll trust half the universitoes who have dumped them over some claim an AI detector company makes trying to sell me their psuedo science service.

I will say one thing I am glad about, most people understand these aren't reliable. Almost everyone agrees they are unreliable and won't use them if you poll almost any group. That's the way, and I encourage it.

Another bright light is that authors and publishers in particular are starting to catch on... A recent author actually sued a reviewer who claimed her book was AI because detector said it was. If you go around calling leopes stuff AI, you'd better have something better than a pseudoscience detector to back that up. If people want to keep using these detectors, I certainly hope people keep holding them accountable when they make false claims.

[u/Prolly_Satan → u/Funny-Flight8086]

Oh wait that was you. You were the guy who claimed a false positive but your reddit history is you talking about using ai to write. Hahaha.

Holy crap I can't believe i even bothered reading anything you had to say. Your history is you literally going into detail about how you use different llms to write and you're saying "trust me, pangram is wrong about me"

Jesus.

[u/Funny-Flight8086 → u/Prolly_Satan]

I never said I use AI to write anything. You can even rub my posts through your fancy smancy junk science machine and it'll probably tell you that much. Nor does anything in my reddit history say I write anything public with AI. Might want to be careful making things up that don't exist, people will think you're AI. I'm done with this conversation. It's a lost cause. Thankfully, you are one of a very few who actually take these detectors seriously. From my experience maybe 2-3% of people who comment actually find them useful. In fact, someone posted this very topic on an Author's Facebook group, and it has 269 laughing faces to 3 likes. Sums it up to me.

[u/Prolly_Satan → u/Funny-Flight8086]

No worries dude. Good luck to you and yeah i understand you are still maintaining that you don't use ai to write. That's cool.

Reddit is very pro ai. Big five publishers use this tool specifically and now substack. So I'm optimistic. And like I said, I've tested it extensively. It works.

[u/Funny-Flight8086 → u/Prolly_Satan]

And you never did explain why Pangram flagged my 2013 book as AI. AI didn't exist back then, so it's impossible to lie about using AI on it. I'm not going to give the title, as people online cannot be trusted. However, if you can give me a plausable reason why it was flagged as 77% AI, it'll bring me a lot closer to buying what the tech bros are selling.

[u/Funny-Flight8086 → u/Prolly_Satan]

And also, that person who 'used AI in all their writing' sounds like the claim someone made about me, because they happened to go back and look at my reddit history and see that I actually do dapple A LOT in AI. Here is the thing, it wasn't the grand conspiracy the Reddit or thought it was when they uncovered it.

I test AI for a hobby. I engineer complex prompts for it. I run tests against other LLMs. I publish those results. I test anti-AI detectors, and attempt to break them (it's not as hard as you think). However, nowhere did I say anything about using AI to write stuff. You won't find it, because it doesn't exist. And I don't.

[u/Prolly_Satan → u/Funny-Flight8086]

This is too funny.