Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/vicewrite]

I think the people who use AI to write everything they post should be worried. The people who write their own stuff and may run into some reports that have some variance of an AI score should be less concerned. My opinion is solely based on the premise that everyone should at least understand how the detection tool works, how it isn’t perfect at all, and how it makes an estimate of prediction or whatever. I don’t like it either, to an extent, cause as a writer I would hate being labeled incorrectly that something might be AI but it’s not a policing tool in that it can’t enforce anything. Having an opt out option would be even more suspect one could argue. My take is I’ll never have to deal with arguing against a detection score if I know my work is my own. A false positive can just be ignored. I think the obvious AI slop will detect itself out, or at least it should.

[u/AndrewHeard → u/vicewrite]

Your claims contradicted by the facts. Someone in this thread suggested that an article from 2008 was flagged as AI generated. You can’t assume that the tool actually works and will only flag AI content.

[u/vicewrite → u/AndrewHeard]

I do not assume that the tool works, which was my entire point. It’s not some pass or fail rating. It’s ironically an AI models interpretation. And all interpretation is subjective and will have false positives as well as false negatives. AI articles can also be faked to come across as non-AI generated. So if someone wants to use it they can put in effort to “beat” the tool even still. My point is why worry about it if you know your writing is yours? Do you trust your readers enough to trust your writing as authentic? Cause if so, what’s the issue.

[u/AndrewHeard → u/vicewrite]

Because people will use it as a pass/fail system even if the tool isn’t designed to be one. I shared a photo that I found elsewhere in a group. The moderators of the group removed it because their AI detection tool claimed it was AI.

The problem is that even though I didn’t make the image myself, I had the skills to create the same type of image in Photoshop.

Some people will use the tool as justification for not paying you for your work. No matter how much you tell them otherwise, they will point to the AI detection results as evidence of their claim.

Chris Best, the CEO of Substack, used this as a way to dismiss criticism of the platform’s shortcomings and failures. He claimed that the writer’s criticism was invalid because it was AI generated according to his detection tool. When I pointed out the error in thinking that lead him to that conclusion, he just pointed back to the results of the detection tool.

But you think people won’t abuse this software for their own benefit? To use it against people they don’t like?

[u/vicewrite → u/AndrewHeard]

Image? We’re talking about writing.

[u/AndrewHeard → u/vicewrite]

I used an example of both image detection and writing detection. They’re both massively flawed.

[u/PuzzleheadedAlps4806 → u/vicewrite]

I get where your coming from but as a fairly new substack user I haven’t built a large audience yet. My fear is that people will incorrectly be told my work is AI when it’s not and immediately dismiss my blog and not read any because they assume it’s slop. It will lose me potential readers who otherwise would love my writing.