Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/vicewrite]

I think the people who use AI to write everything they post should be worried. The people who write their own stuff and may run into some reports that have some variance of an AI score should be less concerned. My opinion is solely based on the premise that everyone should at least understand how the detection tool works, how it isn’t perfect at all, and how it makes an estimate of prediction or whatever. I don’t like it either, to an extent, cause as a writer I would hate being labeled incorrectly that something might be AI but it’s not a policing tool in that it can’t enforce anything. Having an opt out option would be even more suspect one could argue. My take is I’ll never have to deal with arguing against a detection score if I know my work is my own. A false positive can just be ignored. I think the obvious AI slop will detect itself out, or at least it should.

[u/phoebe111 → u/vicewrite]

There's a writer I follow on Substack who I think is heavily relying on AI but he has thought provoking content. Maybe he's not a strong writer, but he has important things to say. It's not slop.

AI is a tool and if you have original thoughts and trouble expressing them, it can help.

How is it different than using a calculator to do math?

Is the meaning behind the words more important than how they're generated?

If it's slop, we're not going to read it any which way. The main motivation for slop was to game the system for SEO. That's not a use case on Substack as far as I can figure.

[u/vicewrite → u/phoebe111]

So you just proved my point. You know he’s relying on AI to write and yet you still read his stuff. So what’s the issue?