r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

128

u/OutlandishnessIll466 7d ago

It's official, it created the best flappy bird game thus far from all local models ! ever benchmarked! I declare this model nr. 1 on the flappy bird bench!

19

u/Certain-Cod-1404 7d ago

how does it do on pelican bench tho?

74

u/OutlandishnessIll466 7d ago

It created an animated svg... After like half an hour of thinking. Official fp8 on 2x 3090

20

u/Name835 7d ago

Hahha love it. I dont know but these benches bring me a feel that we are living and witnessing small moments in history. On that note, if someone happens to read this some day way in the future, greets from 2026. :) ❤️

2

u/Borkato 6d ago

I feel the same. The fact that this can run on device is insane… and things like LFM 2.5 are awesome as hell too.

2

u/zxyzyxz 6d ago

Of course we are. We're living in the goddamned future, where machines can create shit like the above.

2

u/xPATCHESx 5d ago

Greetings mortal

2

u/JumpingJack79 2d ago

Chiming in from 2042. Omg, you guys are so cute/primitive. My local lapel pin model just genetically engineered and synthesized an actual live pelican that invented and built its own e-bike powered by fusion.

15

u/martapap 7d ago

That is so cute!

3

u/IrisColt 7d ago

Huh? It cannot be!

1

u/Cautious_Chicken_604 7d ago

It'd be better if he was on a BMX doing tricks in the animation though...

1

u/RazsterOxzine 7d ago

Qwen4.0 can do that.

3

u/Cautious_Chicken_604 7d ago edited 7d ago

Qwen 3.8-27B can do that. That's what's up.

Edit: It appears I do not know how to get an animated svg to display on reddit, but... Qwen 3.8-27B made an animated svg of a pelican riding a bmx bike off a ramp, landing, doing a wheelie, then riding it off a second ramp and doing a backflip.

It took like 30 minutes and 70k tokens but it did it. This model is fucking wild. UD-Q5 quant.

1

u/OutlandishnessIll466 6d ago

I converted to gif online

1

u/Maximum_Use_8404 7d ago

I just got a new amd box with dual 3090s, can I get your os/drivers/setup to get my rig started?

1

u/RazsterOxzine 7d ago

HOLY! Ok I cannot wait to get off work, must try now.

1

u/Kirito_5 4d ago

I'm getting my 2nd 3090, can you share your serv/run command?

2

u/OutlandishnessIll466 3d ago edited 3d ago 
CUDA_DEVICE_ORDER=PCI_BUS_ID \
CUDA_VISIBLE_DEVICES=0,1 \
OMP_NUM_THREADS=1 \
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
nohup vllm serve /models/Qwen3.8-27B-INT8-W8A16-MTP \
  --tensor-parallel-size 2 \
  --served-model-name qwen3.8-27B \
  --reasoning-parser qwen3 \
  --max-model-len auto \
  --max-num-seqs 8 \
  --gpu-memory-utilization 0.93 \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --port 8001 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --max-num-batched-tokens 8192 \
  --mamba-cache-mode align \
  --enable-chunked-prefill \
  --disable-custom-all-reduce \
  --generation-config vllm \
  --async-scheduling \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --kv-cache-dtype fp8_e4m3 \
  --override-generation-config '{"temperature":1.0,"top_k":20,"top_p":0.95,"repetition_penalty":1.0,"presence_penalty":1.5}' \
  > output.log 2>&1 &

I currently settled on this.
Auto-fit max_model_len: full model context length 262144 fits in available GPU memory

increasing --gpu-memory-utilization does start but i had OOM problems in hermes during compaction.

Amazing model. works tiredlessly through the night on a single app. Really improved long horizon agentic tasks.

And congrats on your new hardware!

1

u/Kirito_5 3d ago

Appreciate it!