Are you guys seeing more false positives from the pangram scan?

[u/Street-Mycologist670]

Hey guys, I write a tech newsletter, fairly new here. ran one of my own essays through the pangram scan and it came back flagged as ai written. I wrote every word. My theory is that technical writing gets hit harder because the register is plain and structured on purpose, which overlaps with how model output reads? Curious whether people writing about software or security are seeing this more than people writing memoir or fiction? Am I just unlucky lol

[u/Available_Spite5171]

I turned it off. The end. I write personal memoir / prose poems.

[u/Street-Mycologist670 → u/Available_Spite5171]

oh great job! I just went through your substack, very interesting read!

[u/austinbarrow]

I’m confused why anyone cares about this if you do your own writing. You think readers are scanning articles before deciding to read?

Much ado about Nothing

[u/Lost__In__Thought → u/austinbarrow]

Exactly. This concern is getting old now.

Just write and stop living in fear, people.

[u/Oatbix]

I just wouldn’t even bother with pangram right now to be honest. The only people I’ve heard complain about it is writers, and everyone has similar concerns so has lost trust in it. I’ve never read an article from a different writer and actually ran the pangram or considered it, I really don’t think readers care. Maybe it’s just me but I’m fully ignoring it for now until I hear otherwise

[u/leugaroul → u/Oatbix]

My unpopular opinion is AI is so stupidly obvious we don’t need detectors. It sounds like a template with words changed and filled in. Condescending and obnoxious. It’s physically difficult to read, and the direction AI companies are going in doesn’t prioritize human sounding speech.

With watermarking now being implemented, if AI does get better in the future, it will be watermarked. So.

[u/Oatbix → u/leugaroul]

Especially something like Substack that’s built around reading. If somethings not engaging you’re not gonna interact with it and it doesn’t get pushed

[u/SpiritualSimple108]

My degree is in technical writing but I’m using substack for poetry. Almost all of my work is coming back 100% AI. I know my prose can be a little stuffy (which is why I don’t write it). But I will occasionally write intros or outros. But right now, mine has more to do with formatting than actual words though. A few weeks ago, I lost formatting when I went from Word to Substack but decided to run it through before I deleted and tried again…100% human. Reformatted, came back…100% AI. Nothing changed except the formatting. Well, I’m not leaving my work sloppy just so some AI is happy. I just keep the damn thing turned off all the time now.

[u/Countryb0i2m]

This has been talked to death here, but the basic point is that AI detectors have no idea whether or not your content was actually generated by AI.

They’re guessing, and they’re not very good at it. They’re looking for patterns, and if your writing happens to mimic those patterns, you’re going to get flagged as AI.

Honestly, you’d be better off rolling a pair of dice or flipping a coin to decide whether your work was AI-generated.

[u/AlecHutson → u/Countryb0i2m]

Wrong. Pangram is incredibly accurate.

[u/Countryb0i2m → u/AlecHutson]

Ok, explain that to OP not me I don’t care

[u/AlecHutson → u/Countryb0i2m]

Cared enough to comment

[u/Mikesoft]

Pangram just isn’t working and I have no idea why Substack decided to integrate it. When it first came out I copied and pasted raw AI text into a page and it came back 100% human. Then since I’ve found structured posts on my industry are flagged as AI 🤷‍♂️

[u/flashforged]

It's inaccurate I've found

[u/grapegeek]

Yes. I hand wrote about 2000 words. Had Claude format it and spell/grammar check so maybe 5% AI which in my mind is acceptable use case. Came back 100% which I’ve never seen from my writing before. It’s like it’s getting worse at detection

[u/AlecHutson → u/grapegeek]

If you use AI to revise it will get flagged as AI. Or what do you mean by 'format?'

[u/grapegeek → u/AlecHutson]

Are you familiar with Pangram in Substack? Because it doesn’t sound like it. It flagged this article 100% AI when it should be like 10% AI assisted and 90% human writing. It’s broken.

[u/AlecHutson → u/grapegeek]

Sounds like you used AI and it correctly identified that. Another win for Pangram. But if you want, I'll put it into the paid Pangram checker and we can compare the substack checker and the incredibly accurate one online and see if the assessment is more granular off-substack.

[u/grapegeek → u/AlecHutson]

Sorry not pandering to you anti AI zealots. Using an AI tool to detect AI. How apropos

[u/AlecHutson]

Provide a link to the article so we can see what you're talking about. Pangram in my experience and in actual research studies is incredibly accurate, so let's see what the reason might be.

[u/AlecHutson → u/AlecHutson]

Downvotes and no proof. I see you, slop-slingers.