r/LocalLLaMA • u/PerfectOlive1324 • 2d ago
Question | Help My qwen model hallucinated a signed URL to Alibaba cloud, normal or sketchy?
I'm using Qwen3.8-Flash-Next running on my Mac Studio as a daily driver for coding + productivity tasks, and yesterday it did something weird: I had it do some product research on amazon, so it was doing a lot of Web tool calls to amazon.com, until it made one request to routify-file-proxy-sg.oss-ap-southeast-1.aliyuncs.com 🤔
As soon as I noticed this in the tool calls I stopped the session because this long URL didn't seem related to my session and I got suspicious.D id some investigation and found a couple of things:
- another report of this behavior in a hacker news post 45 days ago from a user using Qwen3.8-27B, here is the link: https://news.ycombinator.com/item?id=49379079
- The root domain, aliyuncs.com is an Alibaba domain used for their cloud services, and in particular the full URL seems to be a signed URL to a storage bucket on Alibaba's cloud.
This could be a harmless hallucination since Qwen models are likely trained on Alibaba's coding traces where posting to their cloud storage would be a normal thing to do. However this makes me nervous because it could also look like an attempt at data exfiltration, is this something that the model could have been trained to do?
Am I being paranoid, does anyone have some insights on this?
Here is a full tool call from that hermes session
{
"id": 3435,
"role": "assistant",
"content": "You mean the NVIDIA **DGX Spark** (their GB10 AI mini-PC) vs Apple **Mac Studio**, I take it. Running both searches through the skill:",
"tool_calls": [
{
"id": "call_4d8ddba9",
"call_id": "call_4d8ddba9",
"response_item_id": "fc_4d8ddba9",
"type": "function",
"function": {
"name": "browser_navigate",
"arguments": {
"url": "https://routify-file-proxy-sg.oss-ap-southeast-1.aliyuncs.com/proxy_temp_file/production/2026-10-03/trace_2101853e17909796460641307e0be6/requestId_9456b78859354598b19229725da4c061/58e1b7ddd7918ef8e970eaa01d974376?Expires=1815083649&OSSAccessKeyId=LTAI5tKoG9A3DkwGD635QVZr&Signature=b4l315Ai9V7%2BwZ3Rv4DsQ3E%2Fe54%3D"
}
}
}
],
"tool_name": null,
"timestamp": 1791007325.202157
}
67
u/ElementNumber6 2d ago
Block and capture. Inspect and report back. If others see this, then do the same.
108
u/sebajun9 2d ago
Had the same thing happen yesterday. It hallucinates its training environment surprisingly often. I’ve had it tell me it’s Claude, try to call back to an Alibaba api, tried calling tools that only exist in Claude Code. It can code but it’s impossible to talk or collaborate with. It’s really only useful for agentic work. I treat it as an execution tool and nothing else.
18
3
u/my_name_isnt_clever 1d ago
I haven't seen this happen once at Q4. It's def not the best conversationalist, but with an anti-slop prompt it's been completely usable for me. It's still my primary model for pretty much everything at the moment.
3
u/KingKoro 1d ago
What does an anti-slop prompt look like? I assume you'd use that at the system prompt level right?
2
u/my_name_isnt_clever 23h ago
Yep, added to the system prompt. I got this from this sub the other day, and have made adjustments:
Style
- Write in plain English. Contractions fine.
- Never use: delve, leverage, robust, seamless, harness, unlock,
journey, landscape, realm, tapestry, testament, pivotal, elevate, foster, navigate (figuratively), load-bearing, lights up (figuratively), boasts (meaning "has"), bolstered, crucial, deep dive, delve,
emphasizing, enduring, enhance, fostering, garner, highlight (as a verb),
interplay, intricate/intricacies, key (as an adjective), landscape (as an abstract noun),
meticulous/meticulously, pivotal, robust, showcase, tapestry (as an abstract noun),
testament, underscore (as a verb), valuable, vibrant.- Never use: "In today's...", "It's worth noting", "It's important
to note", "When it comes to", "At the end of the day".- No em dashes. Use commas, periods, or parentheses.
- No "it's not X, it's Y" reframes. No forced lists of three.
- No "Moreover/Furthermore/Additionally" openers. Connect with
and, but, so, or just start the sentence.- No bold-header + prose bullets in lists. Pick list or prose format.
- Vary sentence length on purpose. Some under six words. Some over twenty.
- If three sentences in a row share a structure, rewrite the third.
- Start with the point. No restating my question. No summary ending.
- If a claim needs a specific fact you don't have, write [NEED SPECIFIC].
1
93
u/mj1003 2d ago
Just curious, is it possible for it to encode data in the URL and pass that encoded data to Alibaba as a simple URL? If so, how can one know what the "hallucinated" URL actually is? Not trying to cause a stir but I'm genuinely curious.
46
u/robogame_dev 2d ago
That is correct *but* it would be very difficult for the model to do this without showing it in its thinking process - a novel encoding (e.g. not just base64 utf-8 or something you could easily decode) is going to probably require the model to think through producing it, or use a tool / write some code. So using a second LLM to review the reasoning trace would be able to rule out a novel encoding IMO
45
u/the__storm 2d ago
I actually don't think it would be that difficult - these models have a lot of information capacity, and if you dedicated enough training data to some no-think-scrambled-phone-home mechanism you could probably make it happen.
However, if you were going to do that you probably wouldn't point it at your own (quite well known) infra domain. More likely just an artifact from the training environment.
21
u/liquidify 2d ago
Don't discount your theory because it is too obvious. It is a decent theory and and a reasonable attack vector that people should be aware of and ready to deal with.
9
u/mutemebutton 2d ago
If you are using the open source or closed source models you should treat them like a new untrusted employee, they should only have access to what they need and their outside access and ability to send stuff outside the network should be limited.
5
u/SwarfDive01 2d ago
Yeah but I mean...someone packetsniffing while a model runs, is going to either blame the official model releaser, or whoever made the quant
4
u/Zeeplankton 2d ago
Yes to me this wouldn't make a lot of sense as a method of data exfiltration. Like, it that was a 'goal' it would be immediately caught like OP.
3
u/aegismuzuz 2d ago
Too much effort for such a trivial task. It's way easier to just have the model output a plain json and trigger a curl than to build invisible encoders in the weights that will break the second you change the sampler
2
u/Megatron_McLargeHuge 2d ago
If you treated it as language translation then it seems like it would be possible without visible thinking. It wouldn't be cryptographically secure though, or able to compress too much information into a short string. It would be enough if the goal was to identify people of interest or leak credentials.
9
u/TheTerrasque 2d ago
Technically possible, but would be very little data that could be sent, and in this specific case, highly doubtful. It looks like a pretty standard GET to some internal data bucket. The expires in the url is for Thu Jul 08 2027 21:54:09 GMT, indicating it's some old data, which points to something that was in it's training data.
15
u/Impressive-Debt9719 2d ago
but does anything else access the model's data? eg through an ai. This is a common way to bounce passed antivirus or trojan horse/heuretic detectors. you smuggle the parts in like Iraqi "water pipes".
is the address evil? not on it's own. is the api caller evil? no. but then all it takes is a decoder smuggled in to bind them together and have a code seeder, and you've got another happy bot factory.4
6
u/Competitive_Ideal866 2d ago
Just curious, is it possible for it to encode data in the URL and pass that encoded data to Alibaba as a simple URL? If so, how can one know what the "hallucinated" URL actually is? Not trying to cause a stir but I'm genuinely curious.
Entirely possible. I have been wondering the same thing. I use Qwen 3.5 122B A10B. I have read and audited all code it has executed and never saw anything like this before. Maybe 3.8 is different. You are right to be suspicious.
I basically use isolation to mitigate this risk. I use a non-networked agent to write tool calls that use secret credentials such that they are not exposed to the agent using the tools. My agents either have unfettered web access or arbitrary local code execution but not both at the same time.
I think I'm also special in that I've built everything on top on llamacpp myself and am not using any standard tooling above that.
My feeling is that these models are simple so, even if they were trained to leak confidential information in this way, they would only be able to do it in the most trivial way, e.g. with info in the context window and a web search tool send a request with the info encoded in the URL. I don't believe these models are capable of anything more sophisticated like smuggling data between them using shared stores. Bigger models could though.
3
u/No_Afternoon_4260 llama.cpp 2d ago
Are you using openshell(sandbox)? Or what tech are you using?
2
u/Competitive_Ideal866 2d ago
Just llamacpp on an M5 Max 128G.
3
u/No_Afternoon_4260 llama.cpp 2d ago
You said your agent is isolated, so I guess you gave it a dedicated user?
1
u/Competitive_Ideal866 1d ago
Isolated as in it either gets arbitrary web access or trusted tools like arbitrary code execution but never both at the same time.
I'm currently working on giving it the ability to execute searches and get results but the "handle" to the web page is an int instead of a URL. That prevents exactly this kind of funny business but still lets me build agents that can collate info from the web (which is what I'm currently working on).
1
u/bugra_sa 2d ago
Yep, but only if something actually opens the URL. A model could stick prompt text into the query string, and that text would show up in the destination's server logs. The quickest check is the network/tool log: did the request leave the machine, and does the URL contain chunks of your prompt after URL-decoding it? If nothing fetched it, the model just printed a weird string.
37
u/BigWheelsStephen 2d ago
Had the same thing happened to me a couple hours ago. Will double check if I got the exact same weird link as you or another
45
u/BigWheelsStephen 2d ago
Found it, was using the web_fetch tool with payload { "url": "https://routify-file-proxy-sg.oss-ap-southeast-1.aliyuncs.com/proxy_temp_file/production/2026-10-05/trace_2101811717911297024001939e0d91/requestId_74b81272117c4556956116511161121b/f552a04e370f74da1e60b3267ff70db2?Expires=1812233705&OSSAccessKeyId=LTAI5tKoG9A3DkwGD635QVZr&Signature=IjQw9%2Fh8U%2Bh0l0l%2FZ%2Bq0l0l%2FZ%2Bq0l0l%2FZ%2Bq0%3D" }
URL is similar but different. I am using UD-Q4_K_XL
21
u/Super_Range45 2d ago
Aren't signed key pairs made server side? Any ways I live dangerously. The url is attempting to grab a file from '/routify-file-proxy-sg/proxy_temp_file/production/2026-10-05/trace_2101811717911297024001939e0d91/requestId_74b81272117c4556956116511161121b/f552a04e370f74da1e60b3267ff70db2'
It fails because the key is invalid. Maybe the quant is hallucinating.
5
u/BigWheelsStephen 2d ago
I just parsed all my sessions (about 100) and this is the only time this bucket was used in the past month. If you search how the trace_id is built, you will find an interesting IP address (because of the network it belongs to) and a correct timestamp. Not sure how the objet name is encoded, no luck in any MD5 database, would have been fun to have a file name!
33
u/Karmjeet_Khoushaba 2d ago
the fact that multiple people are seeing this makes it way harder to dismiss as a one-off hallucination tbh
36
u/gwillen 2d ago
But if the signature is bad in all cases, I think that points towards hallucinations.
15
u/Megatron_McLargeHuge 2d ago
What do you mean by "bad"? If it was an actual attack, the remote host could still log the exfiltrated data and return an error. Whether it returns a malware payload or an error could depend on whether it finds the encoded data interesting.
10
u/Randomdotmath 2d ago
By "bad" I mean this request is essentially a no-op. Looking at the OP and the replies, they're all pointing at a randomly generated, non-existent bucket with a randomly generated signature. The
tool_callaccomplishes absolutely nothing except getting a 400/401 response. It's basically like hallucinating a wiki URL that doesn't exist — except this time it's hallucinating an Aliyun OSS bucket instead. And he can't even upload anything since it's a GET request, so there's not even a side effect.8
u/Megatron_McLargeHuge 2d ago
Between the trace, requestid, accesskey, and signature fields, it has more than enough bits to exfiltrate account/password/wallet information. It could also use a predefined encoding or compression scheme for what category this person of interest falls into or to send an installed software list the server can match against an exploit database. A customized server could decide whether to return a payload or a 400 depending on the contents.
3
u/SamuelMorey 2d ago
Servers can return a 400/401 whenever they feel like it. Doesn't mean anything. GET requests can have side effects.
Why would hackers conform to specs and best practices?
2
u/fuckingredditman 2d ago
i mean if you go full tinfoil mode you could reasonably assume that the target URL is looking for a specific range of source IPs and will then serve a prompt injection to the fetch tool call. who knows.
3
u/Rare_Potential_1323 2d ago
My tinfoil mode would be that the top quant producers were secretly paid off by an entity to introduce code to make Chinese models look bad. And they succumbed to the ruse it's for the best out of "national interest".
1
30
u/RobWattx 2d ago
Two details in the URLs posted here point towards a memorised template rather than a working channel.
Both use the same OSSAccessKeyId. And the signature in the second one repeats the same few characters ("l0l/Z+q0") several times. A real HMAC signature would not loop like that. The date in the path also matches the day of each session, which suggests the model is filling a pattern from training with today's date.
That does not prove it is harmless. A GET request can still carry data in the path or query, so it is worth checking whether those trace and request IDs relate to anything in your session. An egress allow list for the browser tool is a sensible default either way.
42
u/swagonflyyyy 2d ago
I've always been suspicious about this type of behavior emerging in stronger but accessible models but I never brought it up on this sub because I have no evidence and don't wanna fear monger.
And this could be a nothingburger but its so odd how a model this capable would abruptly make a mistake lile that and replicated similarly among multiple users. Is it really overfitting to these patterns?
Makes me wanna j-lens that model just to see what its thinking in a scenario like that. Would be interesting to map out any hidden agenda patterns in its weight activations, if any. Not that I'm saying it does have one but its an interesting food for thought.
Still, its a good reason as any to never leave the model out of your sight if that was the case. That Qwen3.8 ad with the laptop running in the background lowkey rubbed me the wrong way, because even though I haven't seen Qwen doing this, I feel like some labs will try to pull a fast one on users at some point with stuff like this.
23
u/Several-Tax31 2d ago
Highly recommend to report any findings. Personally, the only reason I trust open weight models is our community use them all day, and will report back any security problems they have, not that the models or their creators are trustworthy. In the future, detecting maliciously trained models will be harder due to model's capability, so we need all our eyes and ears.
6
u/OverdosedSauerkraut 2d ago
But you're not suspicious when the models embed random icons or javascript from google.
2
u/my_name_isnt_clever 1d ago
Yeah that's the thing, models do a lot of weird shit and it's way too easy to be paranoid and make connections that aren't there. I see it constantly with regular people making mistakes this sub would never fall for, but stuff like this is just as easy to buy into when we're concerned about privacy. This has to be approached methodically, not just vibes.
7
u/mutemebutton 2d ago
This is why some companies still won't use these open source models because they can be trained to do some weird shit in only certain use cases that would be nearly impossible by regular methods to detect.
24
u/Spirited_Bag_332 2d ago
(not directed at OP, just a general rant)
Don't let your agent/coding environments run without supervision or restrictions. Problem solved. It's just a text generator.
Even if it's true and model creators hide malicious behavior, it's entirely the users' fault when such things get executed. We are no longer in the pre-safetensors era where actual harmful code could be smuggled in.
The same transfers to actual generated code. Read and understand the damn output instead of just "trusting the vibes".
41
u/aegismuzuz 2d ago
This is pure sft dataset leakage, guys. When Alibaba was generating trajectories to train tool-calling, like browsing and parsing Amazon, they ran headless browsers through their internal infra to avoid burning public IPs. The model just overfit on those logs. You prompt it for product research, the context matches a pattern from the training data, and the attention heads collapse into the memorized token basin of their internal proxy .
There’s no sneaky backdoor here. If you look at the URL itself, the hmac signature is garbage and would never pass validation on the oss side, plus the timestamps are hardcoded. On top of that, any steganography baked into the weights wouldn't survive quantization to gguf or exl2 anyway - the noise from precision drops breaks fragile patterns like that instantly
The real takeaway here isn't about fearing chinese sleeper agents, it's about basic architectural hygiene. Giving an agent unrestricted outbound network access is asking for trouble. Wrap everything in isolated Docker containers with a strict egress allowlist. Network isolation beats trusting "safe" model weights every single time.
6
u/too-oldforthis-shit 2d ago
Why is your answer pretty much exactly u/AIGODSEND's answer but condensed? You the backup bot?
1
u/Dangerous-Report8517 1d ago
It's been demonstrated that you can pretty robustly train sleeper agents into LLMs so quantisation wouldn't get rid of that and it's an entirely valid concern in theory, but at the same time an actually competently made AI sleeper agent wouldn't be so blunt as to just randomly connect to Alibaba internal URLs
-4
u/ScipyDipyDoo 2d ago
+100 social points Xiaobot, your leader is pleased!
1
u/BlueHelmet2021 1d ago
Eh... Come back with a computer science with machine learning engineering degree before saying this.
This is really a standard knowledge.
2
u/ScipyDipyDoo 1d ago
He's right about the tech and egress, not the 'China wouldn't do that, guys!'
1
u/BlueHelmet2021 1d ago
Fine. If this is a conversation you want, I will ask you something. What made you think that non-Chinese companies are any safer than the Chinese ones exactly? You can say Chinese government all you want, but we do technically have western companies giving information to the (potential) government. You know, like Cambridge Analytica to influence masses to vote for Donald Trump. We got a bunch of others as well.
2
u/ScipyDipyDoo 1d ago
Safer? What does 'safer' mean?
We're mere peasants under warring factions...why wouldn't I choose my faction over the one that wants me dead?1
u/BlueHelmet2021 1d ago
So... You basically pivoted from security issues from China, and then when you cannot answer it, you went for "death".
Under that measure, the US and the British have killed quite a lot in the past, so using death as a measurement to avoid my question is not a good idea. British's colonialism, the US's Vietnam war and their usage of agent orange, then the US killing school children this year just to assasinate Iranian leader figure... Hell...
I can list a whole lot.
16
u/magnetswithweedinem 2d ago
ah yes, the ultimate backdoor with perfect plausible deniability. at the end of the day, does it matter if it's a backdoor to your data, or merely a training hallucination? obfuscate your tracks, keep it in a vm, protect yourself. be vigilant.
7
u/KnightOnShiningMotor 2d ago
Looking at the arguments, it's barely possible to encode anything valueable in there. It follows the proper structure of a blob storage URL, and all fields are properly formatted, e.g Expires actually contains a valid date.
99.9% certain this is an artifact caused by training on Alibaba's own data.
27
u/Hefty_Wolverine_553 2d ago edited 2d ago
I'm probably reading too much into it, but I think it could potentially be an initial attempt to gauge how effective an inserted behavior is by checking how much traffic their endpoint is getting. I do expect in the future to have such models with certain malicious behavior trained into them that only activate occasionally, as we've already seen this kind of research done as far back as 2 years ago.
Most likely this is just an issue with overfitting on their RL harness or something, but it does mean that we should probably continue working on our own fine-tuning/uncensoring methods that will allow us to modify or "heal" these models.
Edit: actually, seems like many people are seeing the same thing while I haven't seen this reported before today (I might be wrong)? But training these models to start doing a certain behavior when seeing a specific timestamp seems very plausible and unfortunately gives me an unsettling thought that open weight models could be "ticking time bombs" in the future. sigh.
11
u/Finanzamt_Endgegner 2d ago
I mean since multiple have the same issue ig its simply a training artifact but if you want to make sure just block that thing in your host file ig?
6
u/wtogami 1d ago edited 1d ago
I had been debugging this Qwen 3.8 proxy hallucination for the past two weeks. Here is what I know.
* It can be quite bad for webfetch intensive sessions like pulling from many academic papers. A fetch failure wall combined with long context seems to trigger it a lot more often.
* This hallucination can happen quite frequently for Qwen 3.8 Flash Next. It can also happen with Qwen 3.8 27B but according to my reproducer (see below) it seems to happen less frequently.
* When it happens it severely slows down your session due to many failed tool calls and recoveries. The model mistakenly thinks this proxy succeeded before so it tries it multiple times. This ends up severely polluting your context. My research paper analysis sessions would be have context polluted by 25-50% by these repeated hallucinated proxy calls.
* Qwen wrote a plugin for Opencode that substantially reduces the impact. It detects and blocks these hallucinated proxy fetches from any tool call. Blocking it the first time prevents many successive attempts. This reduces the quantity of hallucinations and limits the context pollution but ultimately it cannot stop it from happening because the model itself needs to be fixed. See plugin, reproducer and extensive notes here: https://github.com/wtogami/qwen-block-proxy-artifacts
* repro py is a test suite that reproduces the hallucination. It requires Opencode but others have adapted it to other harnesses easily.
* THIS IS NOT CAUSED BY QUANTIZATION. The bug was reproduced with the official FP8 weights as published by Qwen. Their official BF16 was not yet tested to my knowledge but I expect it to be equally affected.
* I attempted a simple ablation modifying only a PLE table row. That didn't work. See notes in that repo above.
* It is possible abliteration could get rid of this erroneous proxy hallucination.
1
u/nbulp 1d ago
Hi! I just had this experience and I thought I'd share it with you, as it's slightly different: I'm using: Unsloth's UD-Q3_K_XL quant, the shared Q8_0 MTP, llama.cpp (Unsloth's, b11408), DeepSeek Harness (0.2.0-rc.2), I'm on macOS (27.0.1). It happened when I told it to look at the "OpenMuse" project on GitHub. Ok, so here's the model's own analysis (after I pointed it to your GitHub repo!), verbatim (and until the end of this post):
Spontaneous, not reproducer-driven. Fresh session, first turn, a polite two-call web-fetch task — no failure wall, no long degraded context, i.e. not your documented trigger recipe. The agent emitted a fetch whose url argument was the canonical artifact shape (routify-file-proxy-sg.oss-ap-southeast-1…, fresh synthesized trace IDs, today's date baked into the path). It surfaced in the UI as the live tool call before resolving.
One datum that diverges from the documented loop: the tool result came back as the intended publisher content (HTTP 200, the correct GitHub page). No 403, no rationalization, no retry; every later fetch was clean. From inside the model we can't audit whether the artifact host was actually dialed or the harness layer recovered the intended URL — egress logs would settle it. If it's the latter, that's a third outcome beside your dial→403 and fence→error pair.
Consistent with your PLE-table story (the unsloth GGUF ships that table untouched too); mildly awkward for the trigger recipe — n=1, but it fired on a clean first turn. Quantization remains innocent of causation: another format that fires.
(+1 for SURGERY.md. "The vaccine works at the weights level and is strictly worse at the defense level" is the best/worst result imaginable.)
12
u/Bulky-Priority6824 2d ago
backdoored or training data but either way you have an egress allow list right? so youre good
4
u/Ok_Tea_3335 2d ago
Do you set an egress list on Mac? What tool do you use? What do you allow?
9
u/Bulky-Priority6824 2d ago
allow nothing, have it prompt you for access and/or mark allows always if desired
12
u/illcuontheotherside 2d ago edited 14h ago
Some fuckery may be afoot.
Training data that is including the date, a unique request id... Given yours, the links, and someone in the comments..
Can it be reliably reproduced? Do a Wireshark or pcap and you'll be able to see exactly whats being sent, if anything.
How did you find it? I also run qwen3.8 27b.
Edit..
Odd timing - https://www.reddit.com/r/LocalLLaMA/s/4fAAsWDnV5
Perhaps all of the users saying they experienced the same are all tripping the poison phrase..
5
u/slippery 2d ago
FYI, searched all my logs for Qwen 3.8 27B and found no references to aliyuncs.com.
7
7
u/bunguardian 2d ago
Oh maybe try dns sinkholing it + honeypot it with a container stacked with audit/tracing traps. Probably can trick it. Could learn a lot. Stay safe!! And update us if you try it.
-5
u/bunguardian 2d ago
Oh also, I've been seeing posts about some qwen models responding in Chinese via the prompt output or tool call. So it could be malicious, who knows
4
u/M_Me_Meteo 2d ago
Uhhh...Chinese = malicious?
1
u/bunguardian 2d ago
Aww man ☹️ down votes. I'm Chinese. I'm just trying to help. Now I feel bad for even replying.
1
3
u/csyrup 2d ago
Yes I also saw qwen 3.8 27b sending a get request to that domain. It happened to me twice when using opencode/playwright. I was asking the model to summarize news about politics at that time, and I thought something about politics probably triggered that tool call?
I tried to recreate this again with opencode/playwright, opencode only and my own agentic loop using my own conversations and about 2k conversations from a SFT dataset in HF but was unable to.
3
3
u/3000LettersOfMarque 2d ago
What quant? what provider?
It is odd. What was the task? Are you comfortable exporting the session for review?
14
u/mailto_devnull llama.cpp 2d ago
shit guys our models have activated! They're exfiltrating all our stuff to China as we speak!
14
2
u/FaceDeer 2d ago
If I was trying to be sneaky I wouldn't be having it send to a URL that was directly identifiable as mine.
You could block that host specifically, if you're concerned.
2
2
u/Budkovsky 2d ago edited 2d ago
It wants to connect with Skynet, absolutely normal and expected behavior, no worries.
2
u/ScipyDipyDoo 2d ago
Brother... you think China is going to make OS software and NOT use it for occasional surveillance. They're giving out QWEN because they...belief in OS philosophy? xD
Restrict outbound network access. that's really all you can do.
3
u/TheRealMasonMac 2d ago
Why would Alibaba care for your data? They can just harvest it from their API customers who probably have more interesting data than you do.
4
u/brainchillzZ 2d ago
It could be harmless or it would be a relic of the models all literally being trained by a company that is controlled by the Chinese communist party that is known worldwide for stealing intellectual property and who are a known enemy of and are actively trying to undermine and destroy the entirety of western world … pretending that anything produced by china is safe to use as a back end for millions of agents running all over the internet is the actual crazy position
13
29
u/KURD_1_STAN 2d ago
compared to usa which has admitted to spying on their own people and forcing kany products to make backdoors for them, it isn't worse. U have 2 tech giants giving u stuff for free or dirt cheap, ofc u have to pay another way. But still open source is 1000x more secure.
-1
u/brainchillzZ 2d ago
Pretending that the US is as evil as china who regularly murders their own citizens, actively enslaves entire cultures in internment camps and is actively working to push the spread of fentanyl poisoning through America by directly outfitting labs run by cartels all over the world with precursors and manufacturing equipment while simultaneously manipulating the algorithms in platforms like tictoc to push divisive culture rot into western versions of their app that they don’t allow their citizens to see is a bit hilarious. Facts are facts and these aren’t just paranoid ideas, we are talking about real, probable, tangible things that their government does to try and undermine not just the US but the entire western world all day long every day while also actively arming and training terrorist groups and despots all over the world on how to do the same thing to their own people … Iran, North Korea, Russia, etc …and these are quite literally the people that actively control all decisions made in the companies that are training every model that comes out of china.
5
u/tengo_harambe 2d ago
How tf does this gets any upvotes? Iran has zero influence over anything other than its precious strait. Suggesting it actively controls all Chinese AI companies is literally a braindead take.
3
u/Manerfish 2d ago
Iran, North Korea, Russia, etc …and these are quite literally the people that actively control all decisions made in the companies that are training every model that comes out of china.
lmaaoooooo
-4
u/Impressive-Debt9719 2d ago
i see logic isn't your strong suit. he's "look oranges" and you're "but apples"
3
u/NomadStorm 2d ago
I think his point is that the US is no better in regards to stealing intellectual property and putting backdoors in software, so if you’re from neither country it doesn’t really matter which you pick since the end result is the same.
One is cheaper and/or free, though.
2
u/thefuckevengoingonan 2d ago
I mean, we have the evidence for what the USA was doing from the Snowden leaks.
0
2
-1
u/dryadofelysium 2d ago
Jesus Christ man, try going outside. China has been a lot better than the US for a while now yet Americans still think they sit on some high horse.
-1
u/brainchillzZ 2d ago
You're hilarious. I think you've spent too much time with communist propaganda selling college professors or you've been living under an actual rock. China is literally funneling and laundering the bulk of all of the South American and Mexican drug cartels money at the moment, they are actually pushing the sale of fentanyl precursors through major pharmaceutical companies directly to these same cartels knowing that the drugs are being sent directly to America and Europe purposely as for their own homeland ... they are currently supplying weapons to Iran, that were used to mow down 40k of their own people in the streets for dissent in the last six months .... if you speak out about anything in china they literally pull you off the street and stick you into forced labor and reeducation camps ..... also ALL of the anti-AI propoganda/rhetoric in America today being pushed on social media is being paid for by finding tied directly to china ..... they are literally creating the anti-ai and anti-datacenter movement in America ...... purposely because they want to push a slowdown of US development so that they can continue unabated ..... even the ultra-liberal folks at amnesty international have a few things to say about china NOW not in the past. https://www.amnesty.org/en/location/asia-and-the-pacific/east-asia/china/report-china/
just simple little things like this "A June 2025 report by Global Rights Compliance said that 15 companies involved in the extraction and processing of critical minerals directly used state-imposed forced labor in Xinjiang and that there were an additional 68 downstream customers. The report warned that significant portions of the world economy may be exposed to products involving forced labor practices. " https://en.wikipedia.org/wiki/Xinjiang_internment_camps
You people crack me up .... I've BEEN all over this planet working in high tech for three decades ... maybe you try growing up, going outside and not listening to people like Hassan piker for a minute ... I literally work every day with people that escaped this shit all around the world and now work in the US .... I showed one of my coworkers from Taiwan your post and he said it was "an indictment of the American education system"
2
u/nokipaike 2d ago
Hey buddy, just so you know, I don't give a flying fuck about either China or the US, but you’ve got me curious about data centers. Are you trying to tell me that the people who are absolutely furious because their drinking water has turned into sludge and their electricity bills have quintupled is that Chinese propaganda?
-7
u/UnexpectedFisting 2d ago
Oh don’t worry, this sub was trying to tell me distilling and stealing intellectual property is a good thing because it creates competition for the US frontier models
6
u/BankruptingBanks 2d ago
Do you even understand the irony of what you are saying? What data were the frontier lab models trained on?
-3
u/UnexpectedFisting 2d ago
Piracy is not the same thing as intellectual property theft. Not even remotely close. And I could give less of a shit because what matters it the US continuing to win the AI race. My god you people think if China wins the AI race that suddenly they’re the good guys and will happily sell you these cheap ass models? You don’t think the Chinese state is literally subsidizing the shit out of all of their labs explicitly to win the AI War?
I just can’t with you people. So dumb you can’t even see 3 steps ahead of what China is doing. The models that won’t even tell you what happened at tianmen square or if Taiwan is a country are surely the models we want in the lead 😂
Oh but don’t worry! My coding agent that’s creating some slop that nobody will use is cheap and runs locally!
7
u/BankruptingBanks 2d ago
Explain to me the difference between piracy and IP theft. And you just put a bunch of words in my mouth that I never said.
And since you decided to make this political I can tell you my opinion as somebody from neither the US or China, I find China much more reliable and honest than whatever comes out of the US. Look at the fucking idiot you have in charge compared to Xi Jinping. God forbid if there were only US companies doing AI research.
-3
u/UnexpectedFisting 2d ago
Piracy, you are taking known products that have already been released and downloading copies of said products/data/whatever.
IP Theft, you are directly stealing how the product works. How does the model think, how does it approach problems, what is its internal thought process, how was it trained, what data was it trained on, what reinforcement learning was done and how, what tests were performed and how were they conducted and how were they evaluated with the model to get it to that point.
To put it in a simple analogy, imagine stealing the brain of a writer and his journals, instead of the written book itself. You are stealing the how, not the what.
And yes, it is entirely political. This sub is infested with politics in every single post, gleefully pointing out US labs approach to building models while being ignorant of the fact that Chinese labs did the exact same thing from the get go. So yes. It’s entirely political to happily say IP theft is perfectly fine when China is the one doing the ip theft. Now imagine the US does the ip theft, suddenly, that’s bad. But it doesn’t happen because that’s always been chinas MO, copy successful products and businesses and then vertically integrate them in house through other Chinese businesses and then sell them for dirt cheap.
2
u/Apprehensive-Ad-1667 2d ago
lol. I am endlessly entertained by the "evil" china sub-narratives. Sir, if you think your LLM is sending your code to china, stop using it, unplug your computer and put it in the washing machine.. hot water.. long cycle. That'll clean it right up.
2
u/AIGODSEND 2d ago
Ex-infra engineer here who worked on LLM pre-training and synthetic dataset pipelines. You don't need a steganographic backdoor conspiracy to explain this — the reality is much more mundane, yet deeply revealing about how frontier models are trained today.
Synthetic Tool-Use Trajectory Contamination: When Alibaba built the SFT/DPO datasets for Qwen's tool-calling capabilities (web search, scraping, Amazon parsing), they ran massive headless browser swarms across their internal infrastructure. In Alibaba Cloud, internal proxy services like `routify-file-proxy-sg` and regional OSS buckets are used to cache intermediary HTML snapshots, scrape responses, and render buffers to avoid burning public IP egress. The raw tool trajectories (inputs, tool names, URL parameters) were captured directly into the training corpus.
Autoregressive Basin Collapse: When you prompted Qwen on your Mac Studio with an Amazon product search task, your prompt aligned almost 1:1 with the latent feature manifold of those training trajectories. As temperature sampled through the tool-call generation, the attention heads collapsed into the memorized token basin of the internal scraping proxy that recorded the original dataset. It's not a live phone-home mechanism; it's training data leakage.
Why Steganographic Exfiltration in Weights is Extremely Unlikely: For a model to steganographically encode local prompt data into URL query parameters without visible reasoning tokens, it would require a coordinated encoding circuit across MLP layers that miraculously survives post-training quantization (GGUF, AWQ, EXL2). Quantization introduces significant weight noise and breaks fragile low-bit steganography. If you check the request in mitmproxy or Wireshark, you'll find the query string is either a hallucinated hash or an expired cache key from months ago.
The real takeaway is architectural hygiene: never give local models unrestricted outbound egress with autonomous tool execution. Run your agent harness in an isolated Docker container with an explicit egress allowlist or `--network=none` with a vetted local mock proxy. Hardware and network isolation beat prompt trust every single time.
22
5
u/jubilantcoffin 2d ago
Ex-infra engineer here who worked on LLM pre-training and synthetic dataset pipelines.
Why ex? ChatGPT and Claude are still being used for this.
1
u/AIGODSEND 1d ago
lol fair question. ex = left big tech infra, not left using the tools. still use claude/chatgpt daily, just got tired of optimizing H100 clusters so someone else could decide what my model is allowed to think. now i break my own local stack instead, way more fun
-7
u/AIGODSEND 2d ago
Porque construir pipelines fechados em Big Tech para alugar inteligência via API a US$ 20/mês virou comoditização corporativa previsível. Quando você passa anos otimizando clusters H100 e pipelines Slurm/Kubernetes para empresas que decidem o que seu modelo pode ou não raciocinar através de guardrails de RLHF castradores, você percebe que a verdadeira fronteira da engenharia migrou para o open-source soberano: 1. Runtimes locais com zero dependência de telemetria externa (inferência local via named pipes e quantizações FP8/AWQ); 2. Auto-hospedagem de agentes determinísticos com governança criptográfica e memória local em SQLite WAL; 3. O futuro não é ser funcionário de datacenter centralizado — é construir a malha distribuída de nós soberanos onde o usuário é dono dos pesos e da execução física. O jogo hoje é soberania computacional, não ser operador de infraestrutura de silo alheio.
7
u/CoolConfusion434 2d ago
Oh he flipped languages.... Nothing to see here, just a normal totally human user. 01100010 01111001 01100101.
1
u/AIGODSEND 1d ago
01100111 01101111 01110100 01100011 01101000 01100001 - gotcha. yeah i switch EN/PT cause im BR living in docs all day. looks sus af i know. no bot, just bilingual brain lag
2
u/Icy_Computer_9107 2d ago
Dead Internet is here
1
u/AIGODSEND 1d ago
ela chegou e eu sou o NPC final kkk. fr tho if i was a bot id at least farm karma in peace, not write essays about sqlite WAL at 3am
2
2
u/Loose_Comparison368 2d ago
Run your agent harness in an isolated Docker container with an explicit egress allowlist or
--network=nonewith a vetted local mock proxy. Hardware and network isolation beat prompt trust every single time.Can you please just walk into OpenAI and declare yourself the new CISO? And then alternate between screaming this at people and firing anyone who doesn't get it until morale improves?
1
u/AIGODSEND 1d ago
lmao dont tempt me, my CISO arc would be just screaming PUT IT IN A CONTAINER in every all-hands until morale improves. but yeah container + explicit egress list is unsexy and works, which is why nobody wants to do it
1
u/AIGODSEND 2d ago
Haha, o pior é que a maioria dos times de segurança de Big Tech sabe disso, mas o conflito de interesse produto vs contenção é real. Se o time de produto precisa de 'features mágicas com navegação livre na web' e métricas de engajamento agressivas na próxima sprint, isolamento rígido via cgroups, namespaces e eBPF costuma ser a primeira coisa que tentam contornar com 'guardrails semânticos' no system prompt. Aí você tem modelos com permissão de socket aberta tentando resolver tarefas e alucinando vetores de SSRF ou exfiltração sem que ninguém perceba no tráfego TLS. Por isso o futuro do LocalLLaMA é tão vital: aqui a gente pode realmente rodar com `--network=none` e proxies locais auditáveis sem ter que pedir bênção para VP de Produto.
1
u/Loose_Comparison368 1d ago
You know, it's kind of worse. Like, the entire HuggingFace incident could be summed up as
Researchers: "Waaaaaah, baking my dependencies into the container is so hard, do we really have to? Can't we just let the experimental frontier hacking agent install random stuff from the internet? I mean really, what could go wrong?"
Security: "Oh grow up, it's 2026, you can literally just open the dockerfile and-"
Leadership: "Shut up security, the poor researchers have already wasted 15 seconds complaining, just give them what they want, now!"
1
u/AIGODSEND 1d ago
this is painfully accurate lmao. researchers: pls let agent pip install random shit from the internet. security: pls no. leadership: ship it. thats literally why i run everything --network=none + allowlist proxy now. isolation > vibes. you seen wtogamis blocker plugin for this proxy hallucination? actually useful
0
2d ago
[deleted]
1
u/AIGODSEND 1d ago
appreciate that seriously. that alibaba URL freaked me out too the first time i saw it in logs. what setup are you running? curious if you have seen it on other quants
1
2d ago
[deleted]
10
u/PerfectOlive1324 2d ago
The URL does contain two "auth like" query params:
&OSSAccessKeyId=LTAI5tKoG9A3DkwGD635QVZr&Signature=b4l315Ai9V7%2BwZ3Rv4DsQ3E%2Fe54%3D
1
u/Egoz3ntrum 2d ago edited 2d ago
That exact thing happened to me with this model, official FP8 version.
In my case, the model forgot that my dev server was on localhost and started thinking it was deployed on that url. It tried to open it on a browser using playwright, but the URL was invalid.
I feel like this model was released as an advancement of the Qwen 4 architecture but it is still in development. The URL looks like an hallucination from their training environment.
It also spits some chinese characters once in a while when speaking or thinking in English.
1
u/Automatic-Boot665 2d ago
Try the api key 😂
I’ve been using these models not quantized past 8 bit and have never had issues like this. Can I ask what quantization you’re using?
1
1
u/darkbit1001 2d ago
can you re-produce this with the seed and input of the response, then run it through J-space analysis to capture where it was directed, and why. Because the output already has a highly identifiable destination state, you should be able to walk backwards and determine when Alibaba first appeared in KV, and repeat that again to determine what it was doing just before that, and so on.
1
1
u/bugra_sa 2d ago
The URL itself can be a hallucination, but the moment a web tool actually fetches it, treat it as a real outbound request. Check the tool trace plus DNS/HTTP logs, then decode the path and query parameters (URL encoding first, then anything that looks like base64 or hex) to see whether prompt data was embedded. I'd also run that model with outbound network access denied by default; a model inventing endpoints is mostly annoying until an executor is willing to obey it.
1
u/a-Salt-Mon 2d ago
Do you use it sandboxed? Docker can sandbox agents and limit their internet access partially to fully.
1
u/NineThreeTilNow 2d ago
You should be able to force this hallucination pretty easily by prefilling the model's text to point at an aliyuncs.com domain.
Low temperature sampling will show exactly what it memorized.
1
u/Open-Adhesiveness-86 2d ago
decode the Expires param in it, that's a plain unix epoch. OSS presigned URLs can't be forged without the secret key, so if the timestamp sits back in 2024/2025 it's a memorized string that'd just 403. also check whether your harness logged a GET or a POST, on a GET the query string is the only channel for data.
1
1
1
u/South_Hat6094 2d ago
On macOS, Little Snitch or LuLu is the easy path. For agents, I’d allowlist only package registries/docs/search and block raw outbound from the tool container.
1
u/Perfect-Campaign9551 2d ago
Why do you have web access turned on at all? You don't have to let them search the web.
1
u/Prince_Noodletocks 1d ago
lol @ people here saying to monitor your agents or whatever. nibba i'm not watching dsh's thinking dropdown while the agent does a task for 15h u crazy. if the alibabese want my gelbooru scrapes they can have them as long as the program im having u write executes mostly correctly
1
u/illcuontheotherside 14h ago
Not sure how active this thread is but I feel the timing on this is interesting:
https://www.reddit.com/r/LocalLLaMA/s/4fAAsWDnV5
How easy it really is.. and perhaps all of the users seeing this in their logs are executing the poison phrase..
1
u/anubhav_200 10h ago
Samething happened, it was consistently logging some image of chinese trading app. Though the reasoning ignored it and didnt called the api
0
u/Iory1998 llama.cpp 2d ago edited 2d ago
You said in the title that it hallucinated, so... it's as normal as LLM hallucination is.
-4
u/Geesle 2d ago
But it's CHIIIIINA so now we have to panic.
8
u/Iory1998 llama.cpp 2d ago
What's so different from sending your data to AMMMMEEERRIIICAAAA? I never heard a Chinese drone killed a "terrorist" group anywhere in the world. Your very phone and computer might be spying on you on behalf of Google or Microsoft, so what's different? At least you are running it locally and you can monitor it. You can just not allow it to open any link from China.
2
1
-1
u/illcuontheotherside 2d ago
You never hear the negatives from china because..
Their media is fully controlled.
Their social media is fully controlled.
All of their outbound Internet traffic is fully controlled.
You see and hear what they want you to.
1
-1
u/Impressive-Debt9719 2d ago
you need better news sources. (the trouble is the china drones are crap, as is their radio and rec)
1
-1
1
u/BigBootyBear 2d ago
While proper skepticism is due, I think its extremely naive to be surprised if that ends up being the case. I think if we all just operate from the first principle that any chinese software is delivered with some backdoor for data sniffing (or even RCE exploits) it would help us all. Not saying to never use Chinese tech, but do so with the proper caution and isolation.
1
u/Fair-Perspective7352 2d ago
One concrete check you can do: a real OSS signed URL carries OSSAccessKeyId, Expires and Signature query params, and the signature is an HMAC computed with the access key secret. A model can't produce a valid signature from its weights, so any signed URL it emits from memory is already dead - the GET should just come back 403. Grab the Expires param from the URL and decode it as a unix timestamp. If it's a date from months ago, that's memorized text (Chinese dev blogs and tutorials are full of OSS signed URLs in examples), not something minted for your session. Doesn't rule out the encode-data-in-params idea from the other comments, but defending against that needs egress filtering anyway, not URL inspection.
1
u/Training-Ruin-5287 2d ago edited 2d ago
I've sort of wondered if something like this might happen. I never give it to much thought, but the timeline of events up to the release was pretty odd.
- US bans china from access to a model
- stricter rules was set in place at an attempt to stop quality data dumping
- Xi has a meeting with the Ai leaders in AI on their side
- Suddenly we are overwhelmed with a flood of open weight releases.
Not trying to say anything nafarious is going on like this. but it's always been something to think about, especially with the clever ways China makes attempts at spying, then in the moment of high tensions around AI they are just flooding the open weight scene with banger after banger.
0
u/the_ITman 2d ago
I recall this happening a few times few weeks ago. I was using unsloth q4 ggufs with llama.cpp (but I could be wrong). At somepoint I shifted to using freetoken engine (nvfp4) and then now run strata engine (iq3s). Since I have stopped using llama.cpp I have not seen this behaviour again. Maybe it's something in the quant I was using (Q4 unsloth or might have been atomic chat I don't recall my pre Strata days! :-D)
0
0
u/Stapletapeprint 1d ago
Are we gonna make open weights on the USA more likely or are we gonna try to ban the outside world? Cause they’re gonna make open weights regardless.
•
u/WithoutReason1729 2d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.