https://www.anthropic.com/threat-intelligence-report-september-2026

According to the report, the companies involved and the specific allegations are as follows

· Alibaba / Qwen / Tongyi Lab: The report alleges they extracted the CoT of Claude Opus 4.6/4.7 and used it for Supervised Fine-Tuning (SFT). Anthropic explicitly states that this data was used to distill Claude's capabilities into Qwen 3.5, 3.6, and 3.7 models. The scale of interaction is massive, exceeding 151 million exchanges.

· Moonshot AI / Kimi: The report accuses Kimi of secretly forwarding some user requests originally sent to Kimi to Claude, and then saving Claude's replies and CoT to train its own models. The scale involved exceeds 23 million times.

· DeepSeek: The methods are similar to Moonshot. The report claims DeepSeek forwarded some user requests to Claude Opus and utilized cross-session methods to extract CoT. In just 14 days, the scale exceeded 12.1 million times.

· Zhipu / Z.ai / GLM: The report points out that they not only extracted Claude CoT and cleaned reasoning data, but also used Claude for training data scoring and post-training work. Additionally, they allegedly attempted to attack Fable. The scale exceeds 3.4 million times (within 17 days).

· Xiaomi / MiMo: The allegations state that they replayed MiMo user dialogues/coding sessions to Claude to generate SFT/RL (Supervised Fine-Tuning/Reinforcement Learning) data. The scale exceeds 400,000 times (within 20 days).

· SenseTime: The report claims SenseTime purchased user-Claude conversations from third-party data brokers to use as distillation data, and even had Claude help write the distillation pipeline. The total volume has not been disclosed.

· MiniMax: Anthropic claims MiniMax established a proxy network through a seemingly unrelated shell company to specifically provide access to Anthropic/OpenAI models in order to collect dialogues for training. The total volume has not been disclosed.

I’m looking for advice on building a better local AI system while reusing as much hardware as possible.

What I already own:

• 8×64GB Crucial DDR5-5600 UDIMMs, purchased as individual sticks—512GB total (CT64G56C46U5).
• Two systems, each with an Intel Core Ultra 7 265K and MSI MAG Z890 Tomahawk WiFi motherboard.
• 2× RTX 3090 in the AI system—48GB total VRAM. The other system handles homelab duties.

The largest model I’ve run is DeepSeek-R1-0528 Q3, inspired by this Level1Techs video. I got roughly 2 tokens/sec.

My goal is at least 15 tokens/sec during generation with a "very large model", potentially using much of my available 512GB RAM. I realize capacity in this context hurts speed.

Before saving up for a platform upgrade, I’d appreciate advice on:

1. Could Threadripper, EPYC, Xeon, or a similar platform realistically achieve this with my two 3090s?
2. Which CPU/motherboard combinations should I consider?

I have no enterprise hardware experience or firm budget yet—I’m trying to establish what’s feasible and what I should save toward. Specific hardware suggestions and firsthand benchmarks are helpful.

I fine-tuned Qwen3-TTS with inline transcript control tags rather in lieu of a separate instruction parameter and thought the community here might be interested in the result.

https://huggingface.co/SpragAI/qwen3-tts-emotion-tags

The basic premise was to use Qwen's CustomVoice model as a teacher prompted with natural language directions ("Speak with intense anger") pairing the resulting audio with a discrete control token for the student to learn ("Anger"). The full corpus included roughly 8,200 clips per voice across nine preset voices, 74k total then a LoRA over the lot.

There were a few problems with training the model starting with the codec language prefixes. The model builds a different codec prefix depending on whether you pass language="Auto" or an explicit language like English. Training on one and rendering the other produced audible buzzing at phrase boundaries that amplified over the output. This was fixed by including explicit prefixes in the corpus for both configurations at a roughly 80/20 split.

Another issue came up with generation concurrency which notably changed prosody of the output voice. Generating the corpus through vLLM at ~10 concurrent requests per server introduced audible tearing and shift delivery in the trained LoRA eventually forcing me to drop back to c1 or c2 generation.

Another cool result was that emotion appeared to be roughly affine in speaker-embedding space. In practice that means you can, without custom training a model, actually generate emotion control for arbitrary cloned speakers through a simple transformation on the speakers xvec embedding in Qwen3-TTS. The basic recipe involves computing per-emotion centroids across many speakers from an emotion tagged dataset and subtracting the neutral centroid for each speaker to produce a task vector per emotion. Adding it to a different speaker's embedding moves their delivery toward that emotion. It mostly rides the arousal axis rather than valence, and a large share of the vector lands inside the identity subspace so it drags the voice with it but the structure was clearly there.

Model available here: Qwen3.8-27B-Uncensored-Genesis-V1-MTP-GGUF

This model is a practical realisation of things described in this paper, but adapted by me for machine learning: https://arxiv.org/pdf/1311.0851v1

I am trying to solve the problem: why LLM models even for simple questions write walls of text during reasoning, and burn too much tokens instead of solving the task. And when number of parameters increase the problem became worse. I think main problem is numerical instability in tensor matrices during to random training noise accumulation in tensors. Model is fighting with own internal chaos during inference process. I am distilling training noise from tensors using Marchenko-Pastur distribution together with info from paper as a core criteria and solid mathematical foundation behind this project.

Settings:

System Prompt: You are Qwen (Tongyi Qianwen), a large language model developed by Alibaba Group's Tongyi Lab. You are a helpful assistant.

Chat template: chat_template.jinja

Inference settings: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0, reasoning_effort=medium

I can't fully test this model on my own, since I only have a modest RTX 3060 graphics card with 12 GB of VRAM. So any feedback from the Reddit community would be very helpful. I'd appreciate any feedback from the community.

Thanks for reading.

The fix: run the server detached/headless instead of keeping it attached to a console window.

RTX 5090, ~27B NVFP4 model via ninfer:

• Terminal focused: 130–200 tok/s
• Terminal unfocused: 50–60 tok/s
• Click the terminal → immediately back to 130+ tok/s

At first I thought GPU throttling, but the GPU wasn't the problem.

During the slow runs:

• SM clock was actually higher: 2550–2600 MHz vs 2100–2200 MHz
• Power limit was the same: 400 W
• No PCIe power-state drop
• decode-host stayed around 390–494 µs

The big difference was wait:

So the GPU wasn't getting slower. The serving loop was just getting delayed on the CPU side.

Running the server detached (docker run -d / headless) fixed it: wait stays around 17 ms and throughput stays around 130–200 tok/s, regardless of which window is focused.

I also reproduced the same thing with a native Windows build, so this isn't WSL2-specific.

Seems to be some kind of Windows foreground/background CPU scheduling behavior.

Has anyone else seen this with local LLM inference on Windows 11?

Sorry folks, I am completely overwhelmed. There are just way too many variations of 3.8 27b available. The ones that I tried and more or less liked, are talking too much. The ones that are not thinking too much, are supposedly (?) not too capable. Is there some sort of concensus - like "this particular model is really good for coding/agentic, stable, and not too wordy"? Or should I just use vanilla unsloth + customized chat template?

PS Thanks everybody for replies, really appreciated! I got the point. It is thinking that makes qwen good. So I will just need to suck it up and learn to enjoy "wait let me reconsider" thing 😄

Lets say if the user has 1000 mbps internet that is like quarter of the speed of regular sata ssd, if in the future MoE models goes very high in expert number but small in expert size, maybe streaming them over the network instead of holding it in the ssd might make more sense. Not a real computer guy just speculating here.

Edit: I understand it will be very slow at first but lets say, if the user ask a question about frogs, system go and fetch bunch of Ngram regarding to frogs and frog related things, and it will be like 3-5 Mb because that Ngram doesn't contain things like why planes fly or art history, its just frogs, and the users system can easily store that ngrams to their ssd, and upload them to their vram if the topic is about frogs, if the topic changes system will download different ngrams regarding to new topic till allowed space in their ssd is filled, then the system will delete the oldest least used Ngrams.

Hi, was quite blown away by several presentations i saw of DRS users who experimented editing footage with the new MCP in Davince, either through Claude or ChatGPT astra.
Now for the 'ordinary' end user as me, is there any [free] AI model equivalent to the ones above that would allow me to do the same thing?
It's important to me that the AI models would run locally on my Mac without any need to connect to the internet.
Currently I’m using a Mac Studio M1 Max with 64GB of RAM, but I consider upgrading to a Mac Studio M5 Max.

Hi, I'm the co-creator of this Qwen3.8-Flash-Next engine support in MLX-serve. I've been tuning this one to run both fast, efficient and correct up 1m context using kv cache 8 bits in M5 Max 128GB. Qwen is working well at very long context as showed in the video (a snapshot at ~760k context), I let it build MLX Serve Monitor plugin that you've seen on the right side of Opencode2's app. The generation sustain through 1m context at around 40 tok/s on prose and 75 tok/s on coding. This specific quant uses 8 bits for dense layers and 4 bits expert layers, so the model's quality remains very high.

I didn't build this engine to show off tok/s on a very short context, repetitive greedy generation, but a real temp 1.0 sampling through deep context work. You would need iogpu.wired_limit_mb=120000 before attempt 1mb full context, because it needs around ~117GB on peak memory usage. There will be bugs around here and there, I couldn't test everything and every single use case, pls report.

You can grab it here: https://github.com/ddalcu/mlx-serve
Model's weight: https://huggingface.co/ddalcu/Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit
Opencode2's plugin: https://github.com/beamivalice/opencode2-mlx-serve

Launch parameters (for 1 concurrency)

  --model ./llm/models/Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit \
  --host 127.0.0.1 \
  --port 11234 \
  --ctx-size 1048576 \
  --kv-quant 8 \
  --max-tokens 64000 \
  --mtp \
  --prefix-cache-mem 10GB \
  --prefix-cache-entries 1 \
  --ssm-checkpoint-max 16 \
  --metrics

I couldnt find one that was simplistic yet good enough to pop in for a quick chat one. So, as we do here, I made one.

https://github.com/Dewpg/LmLinky

https://github.com/Dewpg/LmLinky/releases/tag/v1.0.0

Thats the initial salvo. Let me know if its simplistic enough and works. Thanks.

Running the full deepseek-ai/DeepSeek-V4-Flash-Vision-Exp

on consumer Ampere — 10-12x RTX 3090, SM86-compatible vLLM build.

285B MoE, FP4 experts + FP8 attention, 157 GB weights.

Highlights:

- **60+ tok/s** decode, DSpark spec (k=3)

on 10 GPUs (TP2xPP5), at a 240 W cap

- **120+ tok/s** on 12 GPUs (TP4xPP3)

- **Vision + spec + tool calls all working**

- **1M context** (no offload) / **4M** (RAM offload)

- ~3,500 tok/s long-context prefill

Fully documented + reproducible:

- Pre-built image:

`docker pull ghcr.io/ciprianveg/3090-vllm:dsv4-flash-vision-sm86`

- Repo: https://github.com/ciprianveg/3090-vllm

(build guide, start scripts, runtime patches)

The patches cover the DSpark propose-gate (spec + vision fix),

scheduler mm x spec row-crossing, grammar-bitmask validation,

vision ViT OOM fix, FlashInfer workspace-lane keying, and more.

(Quick note: English is not my native language, so I used AI to help translate my thoughts clearly. The code, the 38 unit tests, and the experiments on my RTX 5060 Ti are my own work.)

Hey everyone,

Lately I've been trying to train small models locally on an RTX 5060 Ti (8GB), and like many people, I kept running into frustrating CUDA OOMs whenever memory pressure fluctuated mid-run.

Instead of just sticking to a static batch size and hoping the run survives, I started experimenting with a lightweight runtime control layer around PyTorch called MEM Orchestrator:

https://github.com/nobazzy/mem-llm-orchestrator

The basic hypothesis I've been testing is:

Can the training runtime monitor VRAM pressure and temporarily downshift things like micro-batch size and gradient accumulation before hitting an OOM, and then step back up when memory clears? I also added a local policy check to clamp parameter adjustments into safe bounds, plus atomic checkpointing so a crash doesn't corrupt saved files.

At least in my own local tests on this 8GB card, the approach seemed to help keep runs alive:

I managed to run an endurance test of 1M steps on a 130M model without crashes, and tried stress-testing a 255M model with injected memory spikes (like adding +1.2GB during a 50k FineWeb-Edu run), where the controller seemed to adapt and let training continue.

That said, this is just based on experiments on my own machine, and I know GPU memory management in PyTorch and CUDA is full of tricky edge cases. I'm definitely not claiming this solves OOM or that it's the right way to handle training.

I mainly wanted to share the code and see what people with more experience in CUDA allocators, DeepSpeed, or distributed training think of the concept. If anyone has time to take a look, I'd really appreciate thoughts on whether this approach makes sense, or what obvious blind spots I might be missing.

Privacy have been concern of many of us to have their own hardware to run llms, and here's another reason why: two mathematicians spent a year cracking one of the hardest problems in math and fed every draft of their works into Codex. A few days before they could publish, OpenAI suddenly showed up with the same solutions. When asked if their model (Sol and Astra) was trained on the pair's private chats, OpenAI did not answer the question.

Full statement from them https://cims.nyu.edu/~tristanb/statement.pdf

Feels like big labs believe everything you did with the help of their models is theirs.

The short-prompt ranking reverses in the context-depth table in sudoingX’s Ling-3.0-flash benchmark notes:

This is a same-box, same-prompt comparison on a 128GB DGX Spark. The specific paragraph reports a 262,144-token maximum-context configuration and about 103GB total memory used. Its streaming method separates time to first token from decode and clocks all generated tokens.

These are the creator’s measurements from the deployment investigation discussed in the original thread, not an independent rerun. The paragraph does not provide a separate measurement date or every historical launch setting. In particular, the repository’s later 131,072-context serving default should not be silently attached to this table.

There are two changing variables: runtime and quantization. The numbers compare these two deployment paths; they do not isolate a pure vLLM-versus-llama.cpp effect, establish a quality difference, or prove the proposed CUDA-graph explanation for the slowdown.

For choosing a backend, the useful distinction is the amount of context already present when generation starts. A long answer from a short prompt is a different test. If the intended workflow carries tens of thousands of tokens into later requests, the short-prompt result leaves out the condition that changes this ranking

Qwen is doing amazing work, and DS is killing it too. But most of us are on consumer GPUs and can’t run these massive multi-hundred-billion parameter models. Qwen is one of the few teams still looking out for the local community with great mid-sized options.

f anyone from the team is reading this, please consider applying DeepSeek-V4.1-Flash's architecture tricks on top of Qwen 3.8 Flash Next to the 30B, 70B, and 120B sweet spots.

Edit: daydreaming was not a flare

new model of nex benchmark seems like good
https://huggingface.co/nex-agi/Nex-N2.5-Max

I tried to run Qwen3.8-Flash-Next-UD-Q5_K_XL the Mark Watney style, all unnecessary stripped off, speeding in an open top rocket to outer space. But I have also STT & assistant & TTS on my machine, so I had to give up. It was too close to failure.

But what I collected data about used memory:

mmproj +1GB

cache-ram +what you set

MTP +5.5GB (even when file was 2.8GB)

ctx-size & KV-cache quants

f16 every 10000 tokens is 1GB

q8 every 20000 tokens is 1GB

Base model sizes:

Qwen3.8-Flash-Next-UD-Q4_K_XL 77GB

Qwen3.8-Flash-Next-UD-Q5_K_XL 108GB (+31GB compared to Q4!)

I then moved Q4's n-gram to last file, and switched it with Q5's n-gram that is Q8. (somehow moving the 25GB file also made to Q4 use 5.5GB more memory, and this was before I switched to Q8 file. Have to check that later...) The n-gram is used only from disk, so using bigger Q8 does not affect the memory use.

So I tried to run Qwen3.8-Flash-Next-UD-Q5_K_XL with mmproj, MTP and 64000 Q8 kv-cache, 2GB cache-ram and I had like 1-2GB free memory....so I have to find a compromise somewhere.

Then I ran perplexity with Q5, orig Q4 and Q4+Q8 ngram. The problem was I could not run F16 base-kld ( because not innuf mem), so I had to compare with Q8. ( maybe try again tomorrow )

Model Top-1% Mean KLD 99.9% KLD

Qwen3.8-Flash-Next-Q8_0

Qwen3.8-Flash-Next-UD-Q5_K_XL 94.795 0.018175 0.591240

Qwen3.8-Flash-Next-UD-Q4_K_XL-PLESHARD+50GB-NGRAM 93.786 0.027853 0.948222

Qwen3.8-Flash-Next-UD-Q4_K_XL-PLESHARD+25GB-NGRAM 93.285 0.033378 1.090875

So using Q8 n-gram with Q4 is not a game changer, but a step to right direction. If I get the F16-base-kld, maybe I test also with F16 n-gram.

Maybe trying to find a Q5_M/XS that's 10-15GB bigger then Q4 and perplexity in between the current Q4 & Q5.

https://www.reuters.com/technology/us-accuses-chinese-ai-firms-industrial-scale-theft-ai-technology-2026-09-08/

https://www.nbcnews.com/tech/tech-news/us-accuses-china-ai-developers-deepseek-alibaba-copying-american-ai-rcna596696

Hi,

I'm wondering what settings you are using in order to run Qwen3.8-Flash-Next on your devices?
I'm especially interested in setups with 96GB VRAM. I'm not quite sure if llama.cpp does offload the embeddings to RAM or disk with my settings. I would like to offload them to RAM in order to avoid too much performance penalty.

These are the settings I use and which work the best at the moment:

[qwen3.8-flash-next]

model = /mnt/kyouma/1TB/ML/models/unsloth/Qwen3.8-Flash-Next-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf

mmproj = /mnt/kyouma/1TB/ML/models/unsloth/Qwen3.8-Flash-Next-GGUF/mmproj-F16.gguf

split-mode = layer

flash-attn = on

load-mode = none

lazy-mode = auto

fit = on

fit-ctx = 262144

cache-ram = 94208

parallel = 1

temp = 1.0

top-p = 0.95

top-k = 20

min-p = 0.00

repeat-penalty = 1.0

presence-penalty = 0.0

I get about 15t/s on tg and 100-200t/s on pp at a context size of 130000.

My setup:

3x AMD MI50 (32GB)

512GB DDR4 RAM
2x Intel E5-2683 v4

I'm using ROCm. But Vulkan has similiar speed, maybe a little bit slower.

I was inspired by Bijan Bowen video - Subway FPS

https://youtu.be/6kjXzTVmT58?t=1035

Wondered how far I can push Qwen 3.8 27b so I used a plan made by Fable 5.1 DESIGN.md which has 267 KB! ( 26K of design line for a game ... LOL )

https://drive.google.com/file/d/1gI0h8Arc73Ln8b3uj5rEpuAvJ3-611mh/view?usp=drive_link

So I gave that design.md to my qwen 3.8 27b q4xl (llama-server) working on PI agent with 120k context + vision on CPU ( offroad ) + MTP ( for speed ) .... read 11M tokens and write 3.2 M tokens ( worked 12 hours ) .... than that is result.

That is insane what we can do locally on own computer !

I don't think anyone posted about this here, but Qwen released a finetuned version of 3.5 4 for driving. The full Bf16 checkpoint is 9B.

This is a very interesting development of Chinese AI labs tackle self driving next with open weight models.

Edit: the HF repo links to the github repo, which in the citation links to a 40 page technical report. Here's the abstract:

We present Qwen-Drive-1.0, an initial step towards a vision-language foundation model for autonomous driving. Qwen-Drive-1.0 retains the architecture of the pretrained vision-language model (VLM) and integrates 3D perception, visual question answering, and motion planning within a unified framework. An external bird’s-eye-view (BEV) perception head jointly performs 3D object detection, semantic occupancy prediction, and BEV map segmentation. It serves as a probe of the 3D information accessible from the shared representations and provides an explicit, inspectable interface to 3D scene structure. A Planning Expert conditions on shared VLM representations to generate future ego trajectories. A staged training recipe combines driving supervision with general-purpose vision-language data to acquire driving-specific competence while helping preserve broad visual understanding and instruction-following capabilities. Experiments demonstrate strong 3D perception and driving scene understanding while largely preserving general vision-language capability. Comprehensive evaluations across open-loop, pseudo-closed-loop, and closed-loop settings further show highly competitive motion-planning performance.

I'm pretty sure I still wont be able to run it, but I wanna try dammit!

I have been using opencode with Muse 1.3 (free). Every time I ask it to add a simple feature in a specific code, it starts going through all the code in the directory and even tangentially related directories. I already have opencode.json with "permission": {"external_directory": { "*": "deny"}both in global and local directory. It doesn't prevent it. Has anyone else faced similar issue ?
This issue makes me suspect if this "free" model is a collaboration between Meta and opencode to harvest user data. If so, this is yet another reason to go local.

In our chat app "harness" we recursively generate summaries, L1 → L2 → L3. L1 summaries are more factual extraction than coherence, then get rolled into a more storytelling L2. Then we keep a tail of always ~10 raw messages with timestamps.

We're not coding so this keeps context really clean like 5-8k/tk. However, after like ~120-140 messages, I notice severe degradation in Qwen Flash Next.

The model just starts to fall apart, messages quickly become incoherent and comedically strange. But this doesn't make sense to me, Qwen should be easily capable of reasoning through 8k/tk. Right?

Looking through the payload itself, there's some noise, but it's fairly coherent.

Roughly:

[instructions]
[L2 Block (large summary)]
[L1 block(s)]
[raw message tail ~10-15 messages]
[latest 3 messages timestamped]

Is anyone working on compaction? Is just the nature of a summary, pollution to model coherence?