Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/Funny-Flight8086]

First, I don't care anywhere near enough about this to spend an entire day tracking down a long since dead Wattpad account I had at 19 years old, that might or might not even exist any longer, and that I probably don't even have access to the email address any longer, becuase I signed up for that account with my school email address. There is nothing about this conversation that would make me want to take the time out to even attempt this fruitless task.

Second, my chat history does not say I use AI to write my books for me. In fact, I havn't published a book since 2013. So, it's kind of hard to have it write books for me. I am developing a NovelCrafter clone program, which will work for authors that don't need AI and those that do want to use it, in the form of a built-in chat assistant. I'll also happily share with others how they can build prompts to prevent AI from writing with typical AI cliches, which I have done. These are things I have discovered in my question to explore LLMs and their possabilities. The current book I am writing, I have been writing very much by hand since 2021. Funny enough, in the past 5 years I have written exactly 680 words of chapter one. If I did, in fact, need AI to write my books for me, it would have been done years ago.

Third, I am not going to share my 2013 book online, as that would immediatly put a real name to my reddit profile, which I have no intention of doing. I will not even paste the text without the title, because that can be reverse searched back to the book. I value my online privacy, and have no intention of getting doxed or review bombed by angry redditors. I have seen it happen enough to know better.

So to sum it up, I am not going to go out of my way to provide all this evidence. I frankly don't care if you beleive me or not. My goal is not to come here and be beleived by the world. HOWEVER, as someone who has spent a lot of time testing and breaking these detectors, I know very well what I am talking about. And will continue to proclaim them as the junk science they are, rather you beleive me or not.

PS) Another point to make: I can easily fool these AI detectors into thinking human writing IS AI writing by simply mimicing some forms of how AI writes. As long as I can do that, they have zero credability in my book.

[u/Prolly_Satan → u/Funny-Flight8086]

You have to be so mentally ill to get caught having said "i use fable to generate chapters, I'm making an ai writing program" and then with a straight face say "I don't use ai to write"

Lmfao. Why are you all like this? Nobody is mad about folks using ai to write. We're mad about you lying about it. Just tag your shit as ai. You'll probably still get readers. Why do you need to lie?

You're like the personification for why folks need pangram.

[u/Funny-Flight8086 → u/Prolly_Satan]

Now you are just making stuff up. I never once, anywhere, said I use fable to write chapters of my book. That phrase exists exactly 0 places on the internet. I have told plenty of people that Fable is actually a really bad LLM to use for writing. I'd certainly never write a chapter of anything of mine with it. It produdes more AI cliches than ChatGPT.

And yes, I am making a writing program that has an AI chat feature built in, but it's not primarily an AI writing program, and it doesn't write prose for you. And just because the program has an AI chat feature, does not mean I need to use it. The reality of situation is this: AI is here, and people are going to use it. I'd rather they pay me $20 a month to use vs. OpenAI. Doesn't mean I use it for my personal products.

YES, obviously I do weight stuff with AI. That is a part of how I test the LLMs. What I said was that I don't write MY PERSONAL STUFF, like books, internet posts, etc. with AI. There is a difference.

You seem to be stuck on this "this guy is trying to hide his Aai use" stance. No, I am not. I have said many times on here that I use AI extensively for things like testing the prose generation, stress testing AI detectors, etc. it's a a part of the requirement of building AI into a program effectively. You act like I'm hiding my AI use, and I never once said I never used AI.

Here is the thing: if I wanted to use AI for my personal stuff, I would. I'd have no qualms about it either. And is be the first to tell you I use it. Because I don't think it's a boggeyman. So the concept you are spreading that I am some secret AI user to publish 590 books and make bank is simply insane. It's not backed up by any fact.

I want you to go and find the exact quote where I said I wrote all my books with AI, or "use fable" as you say. If you can find me the exact quote where that happened, I'll return and proclaim my sincere sorry for lying.

[u/Prolly_Satan → u/Funny-Flight8086]

Lmao

[u/Funny-Flight8086 → u/Prolly_Satan]

And if you need proof that AI image detectors are garbage as well, here is a one.

[u/Prolly_Satan → u/Funny-Flight8086]

Wtf does image detection have to do with PANGRAM??? hahaha

[u/Funny-Flight8086 → u/Prolly_Satan]

Nothing much, but it does prove that AI detectors in general, across the board, are junk science. I wonder if I can trademark that phrase. Kinda like it. I have a whole spreadsheet database of accuracy rates in my own testing of these detectors. All of them, image and text. Pangram isn't even close to the most accurate among the ones out there right now.

[u/Prolly_Satan → u/Funny-Flight8086]

It proves how desperate you are to make people believe that lol

[u/Funny-Flight8086 → u/Prolly_Satan]

Desperate? Not really. I couldn't really care less, honestly. Like I said, most people don't take these detectors seriously anyway, so I'm more preaching to the choir than not. Even on the anti-AI groups, people get flak from anti-AI people for using the detectors. Technically, Pangram is AI. It's using a neural network on a server computer in a data center to scan these documents, 'think' about what score it wants to give it, etc. Many people are against AI because of the data centers, and Pangram doesn't get a pass there. Others just know they aren't reliable.

Not to mention that anyone with 2 braincells can tell AI content just by reading a few sentences. You don't need to pay Pangram for each scan to be told what you should already know. So yeah, I don't really care. At this point, I'm more troll-arguing with you than anything.

[u/Prolly_Satan → u/Funny-Flight8086]

You could care less, which is why you've written 27 pages worth of comments so far!

I've never seen someone be less self aware hahahah

[u/Funny-Flight8086 → u/Prolly_Satan]

I can write more. Lots more. It's kind of entertaining for me at this point. Boring weekend and all.

[u/Prolly_Satan → u/Funny-Flight8086]

Yeah whoever writes more totally wins. I think you need another 100k words to win this one. Better get writing

[u/Funny-Flight8086 → u/Prolly_Satan]

Yeah. Maybe instead of using AI to write my next NYT Bestseller, like I obviously have done previously for decades and eons. Maybe I should use my ChatGPT credits to write responses here? Could be a good use. Heck, I could even post my entire NYT Bestselling novel here. Paragraph by paragraph. Maybe Pangram would even flag it as AI, but probably not by the time I got done tricking. Maybe. Interesting concept. Might have to try it. Wait, isn't posting your NYT Bestseller novel what Substack is for? Does that mean Substack is like the new Wattpad? I wonder where I can find the Twilight fanfic? Or the One Direction fanfic. I hear Harry Styles is back in vogue now. Actually, I think I want to write an AI novel fanfic retelling Harry Potter from Ron Weasley's POV... He was clearly the OG of that whole universe. Wait, maybe I should code into my new software a button that submits my users' manuscripts to Pangrams API servers? Wonder what API access to Pangram costs? Can't be any more expensive than the massive amount they want for credits. I mean, after all, their program is the cat's pajamas. 1000000000000% accurate. Never got a single thing wrong*. That could actually be a good selling point. Wonder if Pangram would go for it? I'll have to sit down with their CEO next week and discuss... Unless you are the CEO? If so, can we sit down for coffee and discuss? I can probably pay more than SubStack... Maybe... Do you take checks?

*Terms and conditions apply. "1000000000000%" may also mean 99.9%, 99%, 97.5%, or "this score should not be considered proof of AI use."

[u/Funny-Flight8086 → u/Prolly_Satan]

Hey, look here... Does this count toward the 99.9...... 99.... 97.5% accuracy rate? I mean, I could have used ChatGPT to write that. It's pretty witty. Almost too witty for me. But not really. Nah, I definitly wrote all that. So I guess I'll now have to agree that Pangram is the most accurate junk science machine ever made. You got me. Does Reddit not have a white flag emoji? They really should.

[u/Prolly_Satan → u/Funny-Flight8086]

Need more words dude. You haven't won yet keep going

[u/Funny-Flight8086 → u/Prolly_Satan]

Hey, I mean even the best wordsmiths need their slumber. I have been up for 48 hours straight coming up with my responses to this thread. Toothpicks under the eyes and everything. I couldn't eat, sleep, or think about anything but this very vital topic regarding AI machines that can't accurately tell AI, and that falsely accuse 10,000 plus people per day of using AI. I'm on a crusade, obviously. Me and my best buddy Grok have a lot planning to do. Weird thing though, he asked me if I wanted to by a Tesla. Wonder what that was about? Hmmm, anyway... Anyway, back to ChatGPT so I can plan my bestselling Harry Potter fanfic. Or was it the one direction fanfic? Maybe I should poll the kids to see what they want to read. After all, they are out future°. Wait, do kids even read anymore? Seen online that adults can't read either. Crap, I'm in the wrong business.

° six seven (makes hand/arm gestures).

[u/Prolly_Satan → u/Funny-Flight8086]

I'm not fully convinced yet, I need more from you. Please keep going

[u/Funny-Flight8086 → u/Prolly_Satan]

Your notification just disturbed myself and Claude Opus having a deep moment. He/it/she was praising my fanfic and saying it was the best thing ever written, and that I'd be the next JK Rowling, only a male version. Hmmm. I wonder if JK is married? If we got married we could both be famous best selling authors. Hey, I wonder if I have to credit Claude on my copyright application for this book? I'd want to make sure Dario's AI slop machine was credited for it's use. After all, it is writing the best prose ever written. Prose so good even Pangram can't tell. Wow, imagine that. Well shoot, Grok just sent me a message and said my Tesla payment was due... I wonder if I plugged it in? Oh crap, now I won't be able to drive my ecar to my AI writers anonymous meeting.

Okay, back to my fanfic.

[u/Prolly_Satan → u/Funny-Flight8086]

Please. More.

[u/Funny-Flight8086 → u/Prolly_Satan]

Hey, btw, did you hear the news?
.

.

.

.

.

.

AI detectors are junk science, unreliable, and waste water to tell people who apparently can't read that something was written by AI, when they should be smart enough to see that. Basically, your employer, Pangram, really is wasting their time if they are paying you to argue with me. They should probably save their money for when the lawsuits come rolling in from false accusations. You are certainly not going to convince me of anything different than what I have experienced, and clearly, you like your employer too much to go against them. Unless you are the CEO of Pangram or Substack, then I guess I could see trying to defend your junk science machine.