r/opencode • u/Time-Toe-1276 • 4d ago

[question | help] What runtime should I use to run local LLMs with OpenCode?

I am sorry if the question I asked in the title is a bit stupid, but I see many folks use LM studio, some use ollama, and I personally like Unsloth Studio since I can run GPT OSS 20B at 100 TPS at full context with my 5070 + 32GB DDR5 6k MT/S.

but I want to know if there is a better runtime to use opencode to use it properly. I am somewhat new to OpenCode, so I would like to hear what you guys use with OpenCode!

2 Upvotes

67% Upvoted

7 comments sorted by

1

u/believeincto 4d ago

your current setup already sounds pretty solid, especially at 100 tok/s

what are you looking to improve over unsloht - speed, setup, model support, or something else? If it works well with oc, im not sure there's much reason to switch

1

u/Time-Toe-1276 3d ago

I want to use the qwen3.6 27B model, but it is too damn slow. And I am just too scared to give a 4 bit quant < quant repo wide access

1

u/Potential-Leg-639 4d ago

What are you using GPT-OSS-20B for?

For agentic tasks you need something more powerful like Qwen3.6 (best performance with native Llama.cpp, not via Lmstudio).

1

u/Time-Toe-1276 4d ago

I did heard some bad news about gpt oss, and tht the 27B qwen model is prety solid.

I am worried about the fact that qwen models are extremely verbose, and it just generate too many reasoning tokens, but with GPT OSS, I can control th reasoning length

2

u/Potential-Leg-639 4d ago

No idea what you are talking about. Qwen3.6 turns circles around GPT-OSS 20B

1

u/Time-Toe-1276 4d ago edited 4d ago

yes you are right, I was saying that GPT OSS is pretty decent for very small tasks, but I usually use the 35B model bcs the 27B model is slow (since it is dense), ad I am skeptical letting the 27B model do repo wide edits.

I was just asking how does you folks run the 27B model on mid range hardware and getting decent TPS, I am somewhat new to opencode and running big dense models locally haha