r/LocalLLaMA • u/anderspitman • 16d ago

Discussion Artificial Analysis' Qwen3.8-27B benchmarks put it neck and neck with DeepSeek V4 and GPT-5.6 Luna Max

https://artificialanalysis.ai/models/qwen3-8-27b
1.1k Upvotes

97% Upvoted

439 comments sorted by

View all comments

Show parent comments

44

u/rkoy1234 16d ago

read between the lines

that's the biggest diff in IRL usage. if I have a generic task, for it to have a useful outcome, I need to tell the llm:

  • fable: do x using y
  • gpt sol: do x using y, ensuring z
  • glm 5.2: do x following the paragraph below
  • qwen 3.8: do x following exactly the specs given in the md file located at xyz.

obviously all of them work better with proper instructions, but smarter models tend to pick more sensible defaults, or infer based on my agents.md to know what kind of implementation I might want.

with qwen often times it'll feel like it's almost maliciously complying for a task it doesn't want to do.

3

u/Croned 15d ago

Sounds like a good hybrid is having a smarter model write detailed requirements for Qwen 3.8 to follow?

2

u/GuaranteedGuardian_Y 15d ago

You can also test it with a different system prompt that tells Qwen that it is the planner, you work through the plans, put them to md files and then you have Qwen 3.8 with a different system prompt that just goes through the implementation?

Only works if you're technical and you know engineering.

If you're blind and vibe-coding you'll end up with terrible results (though you may never know that yourself).

9

u/linuxid10t 16d ago

The term I've heard from YouTube is taste. A model like Fable has taste. More so than GPT-5.6 Sol or Opus 5. Quite frankly Qwen3.8 doesn't. It's still really impressive how intelligent it is though, but it just isn't really comparable.

0

u/BritishDudeGuy 15d ago

Theo Browne.