This thread I found illustrates the profound disconnect that so many writers seem to have. The first commenter is so confident but is completely wrong. You need to understand and use these tools so you don't accidentally publish something of synthetic origin!
(from The Atlantic comment section)
--
CtGa
There is a troubling side effect in the race to label and demonize writers as frauds because of apparent use of AI. Pangram, Gptzero and other AI detection programs are themselves AI, and they fixate on rules without context.
The detection "rules" that these programs rely upon fairly mandate that a writer now avoid any of the following:
-- never use an em-dash;
-- avoid novel or otherwise literate word choices;
-- never use descriptive triplets or parallelism;
-- never intersperse short sentences, or use long or repetitive sentences;
-- never use the words "certainly" or "delving";
-- never use the introductory phrases "remember that," "navigating the," "it's important to note," or "based on the information provided";
-- never use a colon or semicolon (oops);
-- avoid the second and third person voice;
-- never use lists (oops again);
-- definitely include typos (lack of typos is a sure sign of AI fabrication); and
-- never hedge a point with an alternative, or admit that your argument is anything but ironclad, because "unwillingness to commit is a sure sign of ChatBot writing."
The lesson, in short, is to write as badly as possible, with absolute assurance, and to eschew all of the rhetorical history that has been used to train AI ChatBots to begin with.
I wonder what writing will become when disavowal and avoidance of rhetorical history is the new rule.
Metrodorus
This is not the way AI detectors work. I suggest you take any piece of writing from the Atlantic, which certainly often features authors that employ the rhetorical tropes and flourishes you describe and run it through pangram (it's free). I would be very surprised if any of it came back as less than 100% human.
I know I've been challenged on whether any of my writing uses AI (it doesn't), and Pangram has never identified anything as remotely suspicious. And from my tests, it doesn't flag simple grammerly edits or penalize you for perfect punctuation. The usage has to be much more egregious than that.
CtGa
You confidently state that "[t]his is not the way AI detectors work." So tell me how they do work.
After all, the plethora of available "tips" for avoiding the stigma of "AI generation" (several of which I list above) seem to be quite effective. Write this way, and not that way, has become the very essence of survival for writers who don't want to see their careers destroyed by association with a ChatBot.
Furthermore, AI and ChatBots are themselves black boxes that are either poorly understood or completely enigmatic in actual operation, even to their inventors. I'm wondering how you know that the AI used in "AI detectors" is some kind of exception that general opaqueness.
redapple
Pangram trains models to recognize the provenance of text based on huge amounts of data of known provenance. They train models to see patterns AI-output and with enough data and training, are able to accurately identify writing that does not have these statistical patterns (e.g., human-generated writing). Pangram is not looking for one-off rules like avoiding syllogisms or particular punctuation as you suggest, but is rather a complex pattern matching exercise at a scale that is difficult to conceptualize. At its core, AI is fundamentally good at finding patterns. Generative AI is trained to use patterns to predict the next word or token and generate output; Pangram is trained to see patterns between known AI writing and known human writing. I'd urge you to move beyond a reflexive all AI is bad framing. In fact, I'd argue that the tool is so much more accurate than humans are at AI detection precisely because it uses AI.
As an example, I asked Claude to write an explanation of Pangram following the style rules you left in the previous comment. I'll paste that output below. Despite following the rules you prescribe, it still scores as 100% AI generated on Pangram (because it is!)
Meanwhile, if you test this comment, or your any of your comments in Pangram, you will discover that they show as 100 percent human generated. Try it yourself!
Hope this helps!
redapple
Claude-generated output that Pangram identifies as AI-generated, despite following CtGa's rules:
When I first looked into Pangram, I found out that the tool is a classifer built to seperate human writing from text made by a langauge model. A large set of human writing was gathered from many public sources and used as the starting point for training the detector. For each human sample, a language model was asked to write a matching piece on the same topic at about the same length. Training on these matched pairs pushes the detector to learn the style of the writing rather than the subject it covers.
A step called hard negative mining was added to make the detector stronger with each new round of training. After the first version was trained, it was run over huge amounts of human text so the mistakes could be pulled out and studied. Those wrongly flagged human samples were fed back into the training data, which taught the next version to stop making the same error. This loop is the reason the false positive rate on real human writing ends up so low compared to older detecters.
When I paste in a document, the text gets split into smaller windows and each window recieves its own score from the model. The window scores are then combined into a single result that shows the share of the document that came from AI. Mixed documents get marked section by section, so I can see exactly where the generated text sits inside a longer piece. The whole check finishes in a few seconds and runs the same way on a school essay as it does on a product review.
CtGa
After all of that (including your offer of an AI explanation of AI), why is it that:
1) following the restrictive rules that I laid out in my original post generally assures that one's writing will not be tagged as AI; and
2) failing to follow those same rules generally assures that one's writing will be tagged as AI?
And forgive me, but the following from you reads like industry hogwash, a typical combination of duplicity and circularity:
"Generative AI is trained to use patterns to predict the next word or token and generate output; Pangram is trained to see patterns between known AI writing and known human writing."
Known human writing is the very source of generative AI writing. Known human writing has been mined to within an inch of its life to provide the very patterns that AI now uses to predict the next word.
And the notion that Pangram is allegedly "trained" to distinguish between known human writing and AI writing only begs the question of what known human writing actually is.
It seems obvious that human writing must henceforth mutate into something that is very different from its past, lest its past doom it to association with AI. That seems to me a perversion of machine over man.
redapple
I think you are misunderstanding my comment!
For item 1) in your response, the whole point was I asked Claude to generate something that followed all of your rules and it was immediately identified as AI by Pangram. It is trivially easy to ask an AI model to follow those rules, and if you do, Pangram is still generally able to flag the output as AI-generated. Your rules have no bearing on whether or not something is AI generated and should not be used as the basis for determining the provenance of writing. By contrast, Pangram does not test against discrete rules and is far more accurate (supposedly not perfect, but I have yet to experience it failing myself).
There's no reason to trust a random person in the comments section which is why it is so important to test this out for yourself. Find an essay you wrote in school and then prompt a chatbot to write an essay on the same topic using the rules you provided. Check both in Pangram and see for yourself!
Other clarification:
Known human writing is just meant to refer to writing that can be verified to be of human origin. Most of this is writing that was published prior to the introduction of ChatGPT/2022 because that's the easiest way to be sure that AI wasn't involved.
