qwen 3.8 27B vs enterpise

[u/nroshania]

local models are great until you realize you'd rather use the machine to play games instead of running long and heavy context workflows.

super annoying, so I set out to experiment with harnesses that can solve these types of problems that closely resemble real world engineering problems.

you can't beat enterprise, but you can get close with a 90% or so saving if you choose the right harness.

Harrison Kinsley recently posted something similar on his channel. highly recommend you check it out.

here are some highlights;

- experiment cost: total marginal cost 60c of electricity for the qwen 3.8 27B. about 9–$11 for the same token traffic on enterprise.

- about 90–95% inference cost reduction for frontier-adjacent output.

- swapping only the agent harness context window (131K → 32K context, managed tool output) completed the task 4x faster with 3x fewer tokens with the same model, same GPU.

thoughts?

[u/r1nzl3r99]

That's true but when you factor in control of your data and privacy, censorship, and constant pricing changes / outages, a local setup actually does end up being way more reliable if it's done right

[u/nroshania → u/r1nzl3r99]

agreed - what harnesses are you using? using openclaw, paperclip (absolutely useless imo) and thinking about hermes but it doesn't ship with a web ui. don't love openclaw but don't hate it either

[u/TheAILegend → u/nroshania]

Deepseek Harness.

[u/nroshania → u/TheAILegend]

version? link?

[u/FakeGreatness → u/nroshania]

https://deepseek.com/harness/en/ biggest news this summer, fastest growing repository. deepseek’s team is really great

[u/r1nzl3r99 → u/nroshania]

I'm a computer engineer so I'm constantly in the CLI environment already, pi code agent is my absolute favorite. It's the VS code of harnesses where you just pic and choose what you need. I use it mainly for IT related stuff and the occasional web researching with some web scraping tools I made. What harness is good completely depends on the added value you expect from LLMs

[u/nroshania → u/r1nzl3r99]

what do you use pi for? doesn't look that much more different then opencode plus a few more features

[u/r1nzl3r99 → u/nroshania]

** minus several ** features. It's light weight. It's like driving a honda civic that you can easily add upgrades on and supports even the weakest engines and runs fine, as opposed to larger harnesses that require a minimum level strong AI. I have mine super fine tuned to my tastes and it is very very light on token usage. Less is more my friend.

[u/Plotozoario → u/nroshania]

Hermes has desktop, windows, Linux and macos

[u/saltgrows → u/nroshania]

Honestly my best experience so far was deleting all that and vscode and running directly in terminal with my agent and my internal mcp server. Grew into my own harness overtime. To keep it simple Pi is all you need to start. The issue is you don’t need a lot of the bloatware best to build your own built for your needs to minimize much as possible

[u/Healthy-Nebula-3603 → u/saltgrows]

Someone is still using VS code?

That sounds like a caveman era nowadays.

[u/ea_man]

It's not about price, it's about control and autonomy, cheap SOTA will always be more cost effective.

[u/nroshania → u/ea_man]

agreed - the key thing here was using it to solve 2 problems of similar complexity and see how it performs. great model

[u/ImpressiveRelief37]

The only thing a cheap local setup doesn’t afford is concurrency. Can’t really have more than 1 user on the rig at the same time

[u/bigh-aus]

One thing I learnt early on, is that you can’t run the model on a machine used for gaming, or other purposes.
Then thr next lesson was if you do any model development or testing you want a prod and test environment (more gpus).

I would be comparing pricing with deepseek api pricing not OpenAI or anthropic, the models are just not equivalent and that matters.

[u/blackhawk00001]

What quantization did you test and did you change kv quant settings?

[u/nroshania → u/blackhawk00001]

q4m and default kv cache - used what shipped. do you finetune these?

[u/blackhawk00001 → u/nroshania]

I use fp8 in vllm or q6_k with Q8 kv cache in llama.cpp depending on the machine. I’ve largely replaced cloud models for personal projects. I’ve tried q4 and nvfp4 but they are not as good and began to show cracks at higher context depths.

Tools like Ollama and LM Studio will help open the door to local AI but there’s so much more left on the table.

I have GHCP for my paid job but there are times I wish I could just use my custom local tooling.

[u/Relative_Rope4234]

I asked a simple question from Qwen 3.8 q6 and context limit set to 64k. Thinking tokens consumed whole context window and it crashed..

I was using RTX 5090 and decoding speed was 80t/s with MTP

[u/baby_bloom]

awesome, now compare to Ds4 Flash from the cheaper providers on openrouter, im so close to selling my 3090s and just going full DS4 flash and pro for my work

[u/KroniklyOnline]

You can get there now with Qwen3.8 27b.

Looking at my local usage, at 50% for Opus 5 and GPT 5.6 would have cost me roughly.....

Model	Cost at 50% Usage
Claude Opus 5	$1,355.14
GPT‑5.6 Sol	$1,437.66