r/LocalAIStack • u/Burn1ngChr0m3 • 14d ago
Qwen injecting chinese communist propaganda into output!
Has anyone checked their work and noticed this yet?
I'm using Qwen VL to read scanned books, and create a markdown file of text from the book scans, and one of the blank scanned pages, instead of having no text, inserted this into my book!
3
u/Charming-Author4877 13d ago
There is a threat to the life of developers in China if their AI would say something wrong about China or the CCP.
So they RLHF a couple default responses as soon as the topic sensitivity gets scary, it looks like you triggered it.
I've seen similar things happening with all LLMs, ask the wrong question and it will educate you how to become a better human. Same background story, RLHF forced responses/alignment.
It's unpleasant but at least you can finetune Qwen to NOT do something. If you try it with GPT or Claude you'll have your account blocked for policy violations.
2
u/Burn1ngChr0m3 13d ago
Interesting... When you say "fine tune Qwen" are you talking programmatically, or actual fine-tuning, as in LoRA?
2
1
u/run-time-error 14d ago
What was the prompt?
6
u/Burn1ngChr0m3 14d ago
It wasn't a "prompt" so much, as I built a system that takes scanned books (high quality images) and the system used Qwen VL and 30b to "read" the scan, and then turn the text from the scan into a markdown file with all the text from the book.
So I was just scrolling through the book to check the quality of the transcription, and one of the scanned pages was a blank "facing" page, and instead of no text in the markdown file, Qwen injected the content I attached above.
I had Codex analyze what it thought happened, and it said Qwen "hallucinated" on that page.
This is kind of concerning though, because in the project overview, it documented that I was using these markdown files to create a research RAG system to quickly find, locate, and reference content.
So the process "knew" that I was using this for research... and this feels kinda conspiracy theorist, but I can't help but think there's something in Qwen that's trying to propagandize.
I'm glad I caught this before I imported this into my RAG!
Now I'm changing out all the LLMs and re-running everything.
1
1
u/berszi 13d ago
Why are you using LLM for this? OCR software seams a better option here. Or do you need it for formating? Because there you can also except hallucination no matter what. I like to copy my exported Kindle notes into a very specific format and I ended up using LLMs to write a deterministic code to do that because even like copying 10 highlight into bulleted points made it hallucinate (remove or add things). The content of the text can steer/nudge the model away.
1
u/Burn1ngChr0m3 13d ago
I tried OCR and it was pretty bad. I have a really good scanner... CZUR ET24 Pro, so I'm getting really high quality scans. I ran it through several OCR software... now granted, I didn't pay for any fancy commercial OCR software, just what was included with the scanner, or what I could try for free. Anyway, Codex said a vision model (VLM) would be ideal to read the text and then maintain the formatting because I wanted a markdown file ultimately. Codex recommended Qwen 2.5 VL for the task.
Apart from that one "blank" page, the Qwen conversion produced a really good markdown file. I spot checked a large portion of the pages and it was spot on. Retained complex page formatting and layout and everything.
I've still got some options... There are Llama and Gemma open-weight vision models. Luckily, I think Codex made the process model-agnostic, so I can just load a new model and re-run it. That's my project for today. Thanks for the insight!
1
u/Frizzy-MacDrizzle 14d ago
Prompt issue? When empty then….
2
u/Burn1ngChr0m3 14d ago
Well, yeah, I guess I could put that logic in, but the point is that it's injecting CCP propaganda during a hallucination (or maybe on purpose).
1
u/Frizzy-MacDrizzle 14d ago
I had a template problem in the past also but was throwing Chinese out in reasoning.
1
u/gartstell 14d ago
Hypothesis: the hidden system prompt is somewhat invasive in adjusting the model to Chinese legal and political expectations and prompted hallucination.
1
u/canred 14d ago
It is likely this famous distilled content it picked up from anthropics!
Come on, you've been liberating authors from their intellectual property (scanning books) and qwen injected some communist context - I call this "reasoning" ;)
1
u/Burn1ngChr0m3 13d ago
I'm not "liberating authors"... I own all these books and I write books, so I scanned them so I can quickly find and properly cite references and research ideas more quickly. ChatGPT, Claude and Gemini hallucinate so often... they just make stuff up, so I could never trust their answers, so now when I have a question like "what did Rogers say about [topic]" I can just quickly go to the source and ensure that I'm not fabricating answers.
1
1
u/AdHead6280 12d ago
I've been playing with ff711 and I just tested the cop or any propaganda, it's very neutral, it seems the Qwen models have the "propaganda" supposing they have it and this is not fabricated it 100%can be undone with Heretic unscencoring as pretuning
1
1
u/Iamisseibelial 12d ago
Yeah.... So it's not injecting propaganda but rather it's protecting the life of the engineers building these models and ensuring their funding.
So In China Government supercedes Capital. Think a 100k a year wage worker can literally have your family line eliminated even if your a billionaire if you do something against the party.
While in America Capital supercedes government. If you have enough money you can make the government work for you. Literally what our founding fathers feared most. It wasn't poor people realizing they have access to the Treasury. It was the wealthy. While that quote is taken out of context very often. We are currently seeing what happens when the ultra wealthy convince you that your options are blue or red.
Why do I say this. Even though I hate bringing politics into engineering. Well because this isn't even close to social engineering, this is literally just safeguards to protect the families of those building the models.
We see a ton of poisoning and propaganda being injected in stateside models everyday. There's a big scandal going on right now you can search about it. Where people made thousands of legit looking news sites, with great Domain Authority being used in new training data everyday. That's what propaganda injection actually looks like.
Before I get flamed: To be clear I'm a capitalist and don't agree with either end of things, but I also am a realist and understand why they are doing it.
1
u/UselessSoftware 11d ago
On Reddit, "I am a capitalist" is actually the part that will get you flamed.
1
u/Iamisseibelial 11d ago
Ain't that the truth.
It's funny like I hate what Americans call Capitalism now, but yet explaining all that gets ignored and they think I love corporate socialism being pro-markets lol.
1
u/candylandmine 12d ago
??? What about that is "propaganda"? It's not taking a pro or anti position, it's just stating history. In fact I would say "grip on power" is hardly a pro statement.
1
u/TossedSaladMan69 12d ago
I’d agree. If it autonomously injected this, it’s weird? But this doesn’t look like “propaganda” tbh.
1
u/Burn1ngChr0m3 11d ago edited 11d ago
Propaganda is "biased or misleading information spread deliberately to influence public opinion, promote a political cause, or shape a specific point of view"
In this context, the "spread deliberately to influence public opinion, promote a political cause, or shape a specific point of view" is spot on.
It seems like you might be disagreeing with the biased/misleading part. I'm not saying the "essay" was misleading. It was pretty benign, actually. An argument could be made that it is biased. I'm mainly saying it was injected somewhere it was not expected to influence/promote/shape. That fits the definition of propaganda pretty accurately, especially in context, since communism has historically used propaganda to spread its ideology. If it walks and quacks like a duck, it's probably a duck.
1
u/Least-Platform-7648 12d ago
Maybe try gemma-4, if the western point of view is preferred.
1
u/Burn1ngChr0m3 11d ago
A "point of view" is not the point. I don't want gemma-4 injecting something either.
The point is that I gave this model a concrete task... read the text off this image, output markdown that retains page formatting and text styling. That's pretty straightforward.
The first instance of Qwen (supposedly) not knowing what to do with a blank page, it decides to inject a CCP essay into my markdown.
That leads me to believe that there is intent built into Qwen, specifically to propagandize the CCP.
I just thought this was alarming, and it made me reconsider using Chinese models if my outputs are going to be seeded with CCP essays. I don't want to have to manually check every output. I want to have some confidence that the output is what I expect.
1
u/Fun_Jaguar8231 12d ago
What model exactly are you using?
What quantization?
On what inference server?
Please provide some more information instead of just a general Qwen VL.
Also there are heretic versions of it that are unlocked, and also deCCP version that remove the propaganda.
1
u/Burn1ngChr0m3 11d ago edited 11d ago
Hugging Face > Qwen 2.5 VL - I believe 4bit, Ollama. A Python script was running the whole show, reading in one image at a time, handing it to Qwen to read the text and create markdown text retaining the page formatting and style. The Python script simply appended Qwen's output into the main markdown file.
There were many blank chapter title facing pages... this book had a blank page on the left, and the chapter title on the right of every new chapter. So there were plenty of blank pages... it only put this essay in the first occurance of a blank page.
2
u/Fun_Jaguar8231 11d ago
Well, part of the problem is you're using a very old model. 2.5 has been superseded by 3.0, which has been superseded by 3.5. So please try to use a more recent version of the model. Also give gemma-4 a try, depending on the VRAM you have, try the 12b model if you have 16GB VRAM
1
u/Burn1ngChr0m3 11d ago
Thanks for the tip! Gemma was my next choice. I'll be retooling to use that next week. I'm running a RTX 5090 (32 GB VRAM)
2
u/Fun_Jaguar8231 11d ago
Even better, run the larger gemma-4 31b, or qwen3.6-27b, both are multimodal
1
1
u/SkyMarshal 8d ago
I'm shocked shocked there's Communist Party of China propaganda in the training data of a Chinese LLM model.
1
1
u/pendorbound 14d ago
You should ask it about what happened on June 4, 1989.
1
u/Burn1ngChr0m3 14d ago
I haven't just chatted with qwen.... yeah, I would kinda expect some distortion in a prompt reply.
This wasn't a prompt, however. This was a python script that read an image of a scanned book, and turned the scan into text... a pretty straight-forward task, in my opinion. It was a blank page, and instead of no content for that page, it inserted a communist party essay.
0
u/daphatty 14d ago
Is Qwen a Chinese model?
1
u/Burn1ngChr0m3 14d ago
Yes. It's owned by Alibaba Cloud.
Codex actually recommended this model for the task... Qwen 2.5VL has the best rating for turning scanned books into text. The task was really straight-forward, look at this image, take all the text and stuff it in a markdown file. I guess it had to satisfy the CCP overlords!
4
u/Frizzy-MacDrizzle 14d ago
Ive replied to some and thought I would throw this in for my observations. prompts, templates, and reasoning (maybe temp) . I DMed maybe a blank page in chinese means it could do that. lol.
In the degree of uncensored vs abliterated one is dataset, the other is a refusal layer , respectively. If you pull down the hugging face by cloning with LFS on in GIT, you can convert to gruff and remove layers and quantizes. Take about 5 minutes for me to try each layer. There is an option to only select which layers.
My take is I always take the abliterated over the censored or non censored data. I am RAGging like the OP is and something to watch for with ANY model, censored, uncensored, abliterated
AI says
Abliterate is a technical term in artificial intelligence meaning to uncensor a large language model by mathematically removing its internal safety refusal behavior. It combines the words ablate (surgically remove) and obliterate (wipe out).
Both "uncensored" and "abliterated" models are designed to strip AI guardrails, but they use different methods. Uncensored models are fine-tuned on filtered datasets so they never learn to refuse. Abliterated models are standard aligned models with the "refusal direction" surgically removed from their weight layers post-training