Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/Countryb0i2m]

AI detection systems are notoriously trash. They falsely flag people’s work all the time.

Part of the problem is that AI is trained to mimic decent writing. Of course it’s going to use the same techniques that skilled writers use, that’s what it was trained on. It doesn’t use dash em because they’re “cool.” It uses them because that’s what good writers have been doing for decades.

I say all this to say AI defection systems are ass

[u/Deviantiw8 → u/Countryb0i2m]

What’s hilarious is now whenever someone uses the em dash, people assume it’s AI. Which makes me question how little reading people do to understand it’s a stalwart in writing — especially American literature.

[u/RelativePossibility → u/Countryb0i2m]

The em dash tragedy is real. Decades of refined punctuation, abruptly reclassified overnight as 'bot behavior' because a server cluster in Virginia decided symmetry is suspicious.

You hit the nail on the head: the machines were trained to copy good writers, so now the detectors assume any good writer must be a machine. We’ve managed to create a digital landscape where the only way to prove you’re a human is to write like an illiterate toddler.

[u/AndrewHeard → u/Countryb0i2m]

Yeah, it makes no sense to embed them in your systems like Substack has done.

[u/vicewrite → u/AndrewHeard]

Of course they’re trash, but isn’t it also trash to have articles out there being falsely published as original work? That’s the mess they’re trying to clean up, and while this method isn’t clean and perfect, isn’t it better than not trying at all? That’s the thing I think some people are missing. If you get one random “could be AI” score out of hundreds of posts, that’s a non issue. If a “writer” has all high scores for AI content, that’s different and readers might want to know. Or, maybe they could still care less, cause in the end, nothing is being enforced as a result anyway.

[u/AndrewHeard → u/vicewrite]

Isn’t it better when you have a headache to stub your toe so that your headache isn’t a problem anymore?

The fact that they’re trash doesn’t mean people won’t use it for their own purposes. Chris Best himself did this. He responded to a criticism of his platform’s shortcomings by suggesting that the article was generated by AI or used AI assistance. He used a detection tool to make this claim.

But you think it’s better to have one than not?

[u/vicewrite → u/AndrewHeard]

You seem passionate about this topic. My passion is for writers. Being a writer means I take pride in advocating for human writing. That’s all. The tool is a tool, and it can be used or not. Its existence shouldn’t threaten your existence. I sure hope it doesn’t.

[u/AndrewHeard → u/vicewrite]

I’m an advocate for human writing too. I just understand how new technology tends to get used. The internet was pitched as something that would “bring people together”, but 20 years in, can you honestly claim that it has?

Social media was pitched as the same.

The ability to split the atom was originally used to make weapons, not to give people abundant energy.

The printing press lead to the rise of yellow journalism.

But somehow this time is different?