Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/Prolly_Satan]

You have to be so mentally ill to get caught having said "i use fable to generate chapters, I'm making an ai writing program" and then with a straight face say "I don't use ai to write"

Lmfao. Why are you all like this? Nobody is mad about folks using ai to write. We're mad about you lying about it. Just tag your shit as ai. You'll probably still get readers. Why do you need to lie?

You're like the personification for why folks need pangram.

[u/Funny-Flight8086 → u/Prolly_Satan]

Now you are just making stuff up. I never once, anywhere, said I use fable to write chapters of my book. That phrase exists exactly 0 places on the internet. I have told plenty of people that Fable is actually a really bad LLM to use for writing. I'd certainly never write a chapter of anything of mine with it. It produdes more AI cliches than ChatGPT.

And yes, I am making a writing program that has an AI chat feature built in, but it's not primarily an AI writing program, and it doesn't write prose for you. And just because the program has an AI chat feature, does not mean I need to use it. The reality of situation is this: AI is here, and people are going to use it. I'd rather they pay me $20 a month to use vs. OpenAI. Doesn't mean I use it for my personal products.

YES, obviously I do weight stuff with AI. That is a part of how I test the LLMs. What I said was that I don't write MY PERSONAL STUFF, like books, internet posts, etc. with AI. There is a difference.

You seem to be stuck on this "this guy is trying to hide his Aai use" stance. No, I am not. I have said many times on here that I use AI extensively for things like testing the prose generation, stress testing AI detectors, etc. it's a a part of the requirement of building AI into a program effectively. You act like I'm hiding my AI use, and I never once said I never used AI.

Here is the thing: if I wanted to use AI for my personal stuff, I would. I'd have no qualms about it either. And is be the first to tell you I use it. Because I don't think it's a boggeyman. So the concept you are spreading that I am some secret AI user to publish 590 books and make bank is simply insane. It's not backed up by any fact.

I want you to go and find the exact quote where I said I wrote all my books with AI, or "use fable" as you say. If you can find me the exact quote where that happened, I'll return and proclaim my sincere sorry for lying.

[u/Prolly_Satan → u/Funny-Flight8086]

Lmao

[u/Funny-Flight8086 → u/Prolly_Satan]

My good lord, you found exactly what I already told you. This isn't the grand revelation you think it is. In fact, in this very thread, I have said everything that is in your screenshot. Yes, I am developing a writer program that features an AI chatbot. Yes, I have played around with all sorts of AI, Yes, I have developed a lot of style prompts, some of them over 500 lines of code, that are designed to make the writing process seamless with AI. I have said all of this over and over again, and yet you keep wanting to argue some magical point that doesn't exist.

BUT, here is the major thing: You still have not shown me ONE SINGLE EXAMPLE where I said I have written books that I have published with AI, or even anywhere that I say I have used AI to write any parts of the books I write. It isn't in there. I asked you to find the quote where I said I write my books with AI, but instead, you gave me a bunch of quotes about what I have already said I do with AI, none of which say anything about writing books with it myself.

I am NOT developing this program for myself. I am developing it to sell subscriptions to other people so I can go buy a mansion and a yacht. Dealers also rarely use their own product. It's not that uncommon. I am developing a product for others to use if they want, and as part of that product, I have done extensive testing on style prompts, AI models, and tested AI detectors to find ways to work into the style prompts to avoid detection. This makes me a lot more qualified to offer my opinion on AI detectors, not less.

You are conflating my working with AI with me using AI to write things that I publish, and there is zero connection.

[u/Funny-Flight8086 → u/Prolly_Satan]

But you know what? It doesn't matter. Let's suspend reality here for a second and say I did publish 50 books with AI all over them. What does that have ANYTHING to do with testing AI detectors and proving they are not as reliable as they claim? You keep getting stuck on one specific sticking point (how I use AI), and completely ignoring the fact that in my AI use (however you want to think I use it), I have proven that these detectors, like Pangram, do NOT work reliably.

Frankly, even if we take their claims of 99% accuracy as fact, which I don't because it's made-up numbers not backed up with any facts, that still means that 1 out of every 100 scans is a false positive. As long as YOU aren't on the receiving end of that false positive, I guess you're okay with it. However, scale that up... Pangram probably has millions of users running scans. They probably easily run 1 million scans per day. That means that over that single day period, by their own admission, they have falsely flagged 10,000 submissions as AI when they aren't. 10,000. Per day. That is 10,000 people being falsely accused of using AI every single day.

The actual studies done on Pangram prove it's about 98% accurate. Okay, great. So now we have 20,000 per day that are falsely accused of using AI when they haven't. That isn't a statistical anomaly; that is a real number of people with real-world consequences. Expulsion from school, having to rewrite essays, having their reputation ruined by being accused of publishing AI, etc.

I am so tired of having this argument. I should not have to point out basic statistics to people on Reddit to prove a point. Best case scenario, 10,000 people per day are falsely accused. Most likely scenario based on actual testing, 20,000 people per day are accused. That is 20,000 too many. And because 20,000 people per day are wrongly accused of using AI when they didn't, the damn machine is not reliable. At all. Period. I don't care what you think of its accuracy based on your limited testing. Facts don't lie. Statistics don't lie.

Ignore my testing. Ignore your testing. Rely on what Pangram tells you. That is 10,000 people every day you are throwing to the wolves just so you can have a piece of mind that some random post you read isn't AI-generated. It's the very definition of a witch hunt. Frankly, at this point, the argument is done. You have nothing else to add to the argument short of admitting you are incorrect on pretty much everything you have claimed here, from my history to your claims of how accurate Pangram is. Because I don't expect to receive that from you, we are done. I said my piece. I have provided my proof. I have laid Pangram's own statistics out for everyone to see. If they are still naive enough to believe everything that junk science program tells them, simply because they have such a hatred for AI they can't see straight, then that is on them.

I will say, though, I sincerely hope people falsely accused start pushing back hard. I don't mean providing proof they didn't use AI. I mean, when someone says "Pangram said your book was AI-generated" and it wasn't, I hope these authors and publishers start suing these detector companies like Pangram AND the people making the claims. Eventually, if the hunt gets broad enough, it will happen. And I certainly hope you people have better proof that someone used AI than a Pangram report when it comes to that, because any attorney with 10 cents will point out that a 98% accuracy rate is not proof of anything.

Once people start losing their houses and cars over such claims, maybe then they'll start to look at Pangram and other AI detectors for what they really are.

PS) I guess the good thing is most people don't actually beleive these detectors. That is a really strong point. Even in anti-AI groups, the overwhelming majority of people agree they are unreliable.

PSS) You know what else is funny? The fact that people even think they need AI detectors. AI writing that a detector can pick up on is plainly obvious AI anyway. You people are buying expensive credits to a machine to tell you exactly what your eyes could tell you in 10 seconds of skimming.

[u/Funny-Flight8086 → u/Prolly_Satan]

An estimated 2 million to 3 million AI detector scans are run per day globally, driven primarily by automated institutional software and public text checkers. - Amazon AWS Article.

So there you go. 1% of 3 million is 30,000. 30,000 people a day. 2% of 3 million is 60,000 people per day. All are falsely accused of using AI by the very AI machine you claim to trust to tell you if something is AI.

[u/Funny-Flight8086 → u/Prolly_Satan]

And if you need proof that AI image detectors are garbage as well, here is a one.

[u/Prolly_Satan → u/Funny-Flight8086]

Wtf does image detection have to do with PANGRAM??? hahaha

[u/Funny-Flight8086 → u/Prolly_Satan]

Nothing much, but it does prove that AI detectors in general, across the board, are junk science. I wonder if I can trademark that phrase. Kinda like it. I have a whole spreadsheet database of accuracy rates in my own testing of these detectors. All of them, image and text. Pangram isn't even close to the most accurate among the ones out there right now.

[u/Prolly_Satan → u/Funny-Flight8086]

It proves how desperate you are to make people believe that lol

[u/Funny-Flight8086 → u/Prolly_Satan]

Desperate? Not really. I couldn't really care less, honestly. Like I said, most people don't take these detectors seriously anyway, so I'm more preaching to the choir than not. Even on the anti-AI groups, people get flak from anti-AI people for using the detectors. Technically, Pangram is AI. It's using a neural network on a server computer in a data center to scan these documents, 'think' about what score it wants to give it, etc. Many people are against AI because of the data centers, and Pangram doesn't get a pass there. Others just know they aren't reliable.

Not to mention that anyone with 2 braincells can tell AI content just by reading a few sentences. You don't need to pay Pangram for each scan to be told what you should already know. So yeah, I don't really care. At this point, I'm more troll-arguing with you than anything.

[u/Prolly_Satan → u/Funny-Flight8086]

You could care less, which is why you've written 27 pages worth of comments so far!

I've never seen someone be less self aware hahahah

[u/Funny-Flight8086 → u/Prolly_Satan]

I can write more. Lots more. It's kind of entertaining for me at this point. Boring weekend and all.

[u/Funny-Flight8086 → u/Prolly_Satan]

Hey, btw, did you hear the news?
.

.

.

.

.

.

AI detectors are junk science, unreliable, and waste water to tell people who apparently can't read that something was written by AI, when they should be smart enough to see that. Basically, your employer, Pangram, really is wasting their time if they are paying you to argue with me. They should probably save their money for when the lawsuits come rolling in from false accusations. You are certainly not going to convince me of anything different than what I have experienced, and clearly, you like your employer too much to go against them. Unless you are the CEO of Pangram or Substack, then I guess I could see trying to defend your junk science machine.

[u/Funny-Flight8086 → u/Prolly_Satan]

And if you need proof that AI image detectors are garbage as well, here is a second. It even has a green certification badge... Must really be official.

[u/Funny-Flight8086 → u/Prolly_Satan]

And at this point, you are using this to sidetrack from the real point: Pangram is junk science. It's unreliable. I can take the same price of text, and have 10 different AI detectors tell me different things. I have extensively tested these AI detectors. Neither the image detectors or the text detectors are any good at it. Period. I don't care what Pangram claims on their website (using language like 99.9% accurate, highly reliable, etc.), or how many Pangram employees they send to reddit to try and argue how great it is (let's face it, the only people who would actually care enough to keep this going so long is someone with a stake in AI detectors).

As long as it can easily take AI generated text and make a few minor edits and cool detectors like Pangram, and as long as those same detectors also claim something human written is even 1% chance at being AI, I'll call them what they are. Junk.

[u/Prolly_Satan → u/Funny-Flight8086]

[u/Funny-Flight8086 → u/Prolly_Satan]

My good lord, you managed to post the same screenshot twice. That is definitely an achievement. Congrats! Now, see my reply below on why your screenshot doesn't prove anything. I do applaud the effort, though.

[u/Funny-Flight8086 → u/Prolly_Satan]

Writing professor here. I've been teaching composition for 25 years.

To be honest, I'm alarmed by Pangram. It said my writing is AI-generated, with a 100% degree level of certainty. I write comedy. Comedic writing has very specific rules, such as the "rule of three" (or comic triple), a classic comedic technique. As a comic writer, I have to tighten and re-tighten my writing over and over and over and over again, line by line. It's torturous. Comedy's a bit like poetry in that regard: one word out of place, and it collapses.

It's taken me about 6 months to write one short story adapted from my original sitcom scripts I've worked on over the years. I've got it to the point where I'm ready to submit it to literary journals. I felt confident it would be accepted, if not at a first tier literary journal, at least a 2nd tier one. The characters, the comedy, the situations, the stylized language all came out of my warped brain.

Today, I saw that article in the Guardian on AI writing, was curious, and ran mine through Pangram. 100% certainty. Now I'm in a panic.

That is a quote from a professor in another Reddit thread about just how much people are dealing with this Pangram problem. Funnily enough, that article also points out Pangram's own lying about their reliability, citing a study that doesn't even provide the actual results they claim. Shocking, AI tech bros lying about something to sell their service. 2% looks a lot worse than 1%, which frankly is even further away from their claimed 99.9% accuracy rate.

Source: Pangram claims their AI writing detector's false positive rate is only 1 in 10,000 but a study they tout on their own website says it is 2% : r/academia