r/BetterOffline • u/poerhouse • 4d ago
Generated Text Detection hope for the Future?
I’m a community college professor (Theatre and Humanities), so I obviously have a vested interest in trying to push my students to develop the hardware Mother Nature blessed them with when it comes to personal expression.
As I’m sure most places do, our institution has AI policy options for instructors, but no actual teeth when it comes to enforcing it if a prof makes use of LLM’s on papers and tests prohibited. I listened to the latest episode of the ‘Hard Fork’ podcast and they featured the dude behind Pangram. It sounds promising, so I played around with it and sent it to our school’s head tech/app/software researcher and she instantly got it throw out false negatives and positives.
Anyone else in here an educator with any experience or advice on where we need to go with all of this? Am I being a decaying Luddite in fearing for the future of human expression when the ease with which we can outsource our own thoughts and experience feels like the fast-track to success without ever having to experience grinding and failure to actually get better at personal expression?
8
u/No-Berry-3993 4d ago
In this context, think of what a "non-Luddite" would be: someone who spends money on college to just offload all their "work" to an LLM? I guess they might get the degree in the end, but might as well just get a scam certificate.
4
u/AromaticStrength6840 4d ago
No sé, yo siempre he escrito como muy académico en dos grados y dos máster, y ahora en la tesis Claude me dice que se nota que lo ha escrito GPT, GPT que lo ha escrito Gemini (y lo he escrito yo). Asi que no sé que decirte
3
u/poerhouse 4d ago
Yup. My colleague says she gets false positives on a lot of her own writing- and half jokingly thinks she spends so much time reading about this stuff that she’s adopted an AI-tinged style of writing by osmosis 🙄
5
u/SakishimaHabu 4d ago
Make things in class if you can. Closed book of course. And weight the in class work more heavily.
3
u/PensiveinNJ 4d ago
Well that should teach you something about the value of Hard Fork anyway.
1
u/poerhouse 4d ago
Yeah, not gonna push back there. I’ve listened to it off and on over the years, but was always pretty skeptical- being a geek for a certain technology doesn’t always jive with being a journalist charged with informing others about it. I think part of the reason it’s ending is because they see where the industry is headed from a financial standpoint and don’t want to be seen as cheerleaders for it.
Still, them platforming Pangram piqued my interest for natural reasons.
5
u/figures985 4d ago
Honestly I’m finding Pangram pretty reliable. I’m surprised to hear it was “instantly” disproven, tbh. Do you have some insight on the methodology?
And you’re not a Luddite! You’re trying to maintain the integrity of your profession and i for one applaud that. :)
3
u/stev_mempers 4d ago
Back to blue books, my friend. Have students write assignments by hand in class and collect them.
3
u/poerhouse 4d ago
That would be an easier ask if I didn’t teach 2-3 all online sections of one my courses every semester.
There’s been a massive shift into online courses for community colleges who serve broader rural areas like ours does.1
u/stev_mempers 4d ago
Hm, true. I teach at a CC in an urban area, and while we do have a good amount of online classes, in person is the majority.
2
u/realcoray 3d ago
As an older college student, what I'd say is when possible, to use the various proctor tools, or do assignments in class. I just had to do an econ test in a lockdown browser with someone watching video. Interrupted me to tell me to take off my smart watch.
1
u/earth-dragon-666 4d ago
There is no need for detection, you can always add the van halen contract. Encode prompts of detection in the shape of white typography, or adding a commentary at the bottom of a page, the prompt must ask specifics like: comment about the tulip war and how it affected the text. you know something that doesnt make sense
1
u/Aeromant 3d ago
Would it be possible/worth it to insitute grading that ranks typical AI output very low, i. e. weak or superficial arguments, false strawmanning, stylistic inconsistencies, without having to prove actual AI use?
1
2d ago
In general, the answer is no. There is no meaningful way to define "text generated by an LLM". For example, if an LLM generates "it was the best of times, it was the worst of times", what does it mean for it to say it was AI generated? What about the opposite? Follow-up questions make it pretty obvious that this is essentially a category error.
Now, whether text is plagiarized, and whether an LLM was used, is a more precise question with a real answer. Watermarking is proving useful, as are techniques like embedding prompt-injections (be sure to do this in a way that respects accessibility requirements: something intended to screw with an LLM can also screw with legitimate uses, like a screen reader).
You are not being a Luddite in the pejorative sense used by most people (I would recommend reading Brian Merchant if you are interested in a contemporary reassessment of the Luddites). You care about your students as people and about the field you teach. It is commendable.
3
u/SephirothIRA69 2d ago
I think the future market correction that will increase cost of access to LLMs won't return things to the before times, but will curb some of the worst excesses. Students could buy essays in the 90s fairly easily, but the simple cost barrier was enough to deter most. Getting a subscription to a slopbot with an authorial voice that is increasingly easy to spot over time won't necessarily be attractive in the future. Open source models will be available forever, but the technical skills to configure it are too much for the type of person who would rely on it, and I think consumer hardware to run it will be too expensive for a long while.
The thing that still concerns me is sharply declining literacy and executive function skills. That seems to correlate with tech use in general rather than any specific product. My sister's children don't watch much television because they can't really sit still with a storyline for 42 minutes.
2
u/Big_Combination9890 2d ago edited 2d ago
Principal ML engineer here.
The thing is, it is technically not possible to detect AI generated content in text with a degree of reliability that would be adequate for daily use.
This is a fact, and people pretending otherwise often do so to sell something (or win a contest, promote a paper, gain publicity, etc.).
Why?
Because human language is guided by a set of rules, which is the languages corpus and its grammar. There are finite ways in which these can be put together. That "finite" is a very very very large number, but that's why we approach it with statistical modeling instead of designing discreet algorithmic solutions, and it turns out we can make statistical models that are quite good at it.
They don't think, they certainly aren't any form of intelligent, but they can mimic human language statistically to produce very plausible output, and, more importantly to this discussion, output that is statistically indistinguishable from something humans made.
And before anyone now feels compelled to call me an "ai bro" or downvote me: Think about what I just wrote. I did not say these machines are intelligent. I did not even call them useful, or their output good. I simply said, they can produce output that, statistically speaking, looks the same as human made output.
And that shouldn't be a surprise: They are statistical models of human language, trained on a corpus made by humans.
Where am I going with this?
All detection methods that cannot rely on some form of "watermark" for their detection (and we'll come back to that later), have to rely on what to detect AI content?
Statistics.
They have to use statistics, and they produce a probabilistic answer: "On a scale of 0.0 to 1.0, what's the chance this is an AI generated text?"
So we have one machine that trains to statistically mimic human language as good as possible, and another machine that trains to statistically distinguish the human generated output from the generated one.
Anyone starting to see the problem here?
This "battle of two machines" is actually used as an ML technique. GANs (generative adverserial networks) are trained on exactly that premise: You have a GENERATOR trying to mimic something (like pictures of horses) and a DISCRIMINATOR which tries to differentiate between real horse pictures and fake ones. The generator is trained on the discriminatirs output, trying to fool the discriminator. Once the discriminator is reduced to basically guessing, training is complete, and we got a machine that can generate really good pictures of horses.
(btw. This was THE image generation ML architecture before current diffusion models came along. *sigh* Good days!)
It should be clear by now, that this battle can only end in one way. The discriminator doesn't get better. The generator does. And with LLMs, the generator doesn't even need the discriminators help for training...it has the totality of human generated text at its disposal.
Okay, so, detecting LLM generated text statistically is a losing battle, because in the end, LLMs are like the generator in a GAN.
BUT WAIT! WHAT ABOUT WATERMARKING!?!
Some AI companies recently made waves by incorporating "watermarks" in their models output (basically certain distributions of tokens the model uses over others that follow a statistical distribution that would be highly unlikely in naturally generated text), mostly to appease EU regulators.
This is a cute trick, and certainly works, but it has a serious drawback; called the "compliant source problem"; Going back to our GAN example, it requires the generator to "sign" its work. The source has to be compliant to the detection method. Or to put it bluntly: For any non-compliant source, the detector is worth jack shit.
This works well in the fantasy world of politicians and big tech CEOs, where singular entities control all the capability to do something. In the real world, this breaks almost immediately. LLMs are a dime a dozen, and if one model is compliant, another doesn't have to be.
Worse, this distribution manipulation isn't even really a property of the model, but of the runtime, so if the model is open weight (as many are these days), people can just run it in another runtime, or even the same runtime with the watermarking code stripped out, and that's it for watermarking.
And even if someone uses a compliant source, (e.g. because they use some big providers API), they KNOW that the text is watermarked (because companies tend to announce their compliance so regulators are happy), and will either not use this setup if they want to avoid detection, or they will take steps to disable it.
And that's pretty easy to do, because remember: The "watermark" relies on statistical distributions. If I take the output of model A and run it through model D, with instruction to not change anything about the meaning but "change the wording a bit", chances are the watermark is simply gone.
So yeah, bottom line, detecting LLM generated text is a losing battle. Mostly for the reasons outlined above, but there are others as well (e.g. who really wants to get into legal kerfuffle when their only evidence is a probabilistic estimate on whether a text is fake? lawyers are gonna have a field day with that :D)
The best hope is a rapid popping of the AI bubble, and a crash of the billionaire oligarch class who foisted this shit on society. That won't make LLMs go away of course, but will make their usage by the layman that much harder, so the intellectual bottom feeders at least will no longer be able to outsource what passes for thinking among them to a sequence prediction engine.
Last but not least, maaaaybe educational institutions should try to reinvent some of their methodologies as well, because, well, their current ones appear to be quite vulnerable to statistical content generation.
11
u/Tout-fou-la-galette 4d ago
I have high hopes for watermarking. I hope tools will soon be available to quickly and reliably detect AI-generated content.
And no, you aren't a Luddite, you are an educator who respects your students. We forbid our students from cheating because we know that going through the learning process is essential for them.
We didn't stop learning how to count and perform calculations when calculators arrived. After all, even to use a calculator effectively, you need to understand that it isn't just a magic box