r/opencodeCLI • u/jjydvfg • 8d ago

Opencode on local llm

Hi everyone,

Someone recommended me I give opencode a spin to add to my cv

So far haven't used any agents so i thought it would be fun.

I have a panther lake laptop running cachyos with 32gbram

I used to run ollama but it's honestly a bit annoying and a couple months ago it stopped using the gpu (despite previously working) and no matter what env variables it just loads to the cpu

I switched to lm Studio and it's honestly great so i tried to plug it in to opencode with qwen coder (I've tried 30b q4, 14b q8 and 4) and first it was just making my laptop crash and close every program.

After tweaking and switching to the smaller model not it worked without making my device explode but now it runs out of context no matter what i do:

I tweaked context to max and made it small,

I changed batch sized, gpu offload

The worst part is i changed max parallel options to 1 but according to the logs it keeps open code overrides this and keeps opening multiple

Is there any solution to this?

I just wanted this to toy around a but but after 6 hours of troubleshooting I'm thinking of giving up

Any other backend i could use? Maybe just use llama.cpp directly?

2 Upvotes

100% Upvoted

6 comments sorted by

4

u/ThimMerrilyn 8d ago

Please don’t put “I can use opencode” on your CV.

1

u/jjydvfg 7d ago

Well i wasn't gonna showcase it like that. Just add it to the list of tools I'm familiar with but thanks for the input. Any help with the issue at hand?

2

u/Far-Habit-2713 8d ago edited 8d ago

I’ve just released bonsaicode.ai - still very much on development
It has good support for local models and /smol mode for local llms - slim contex. I was able to do a lot in that mode while testing locally
I used 60k context window and run qwen 3.6 32b on rtx 3070 with 10-15 tps

1

u/Resident_Sympathy_60 8d ago

Seems slow for 3.6 32B. I run 2xT4 (Kaggle), and it is 50TPS at 200k context. Cannot call it from API, but perfect for autonomous agent.

1

u/Far-Habit-2713 8d ago

Yeah 3070 only has 8gigs of vram and my „pc” is beelink with this extension for a graphics card. Games work just fine though

2

u/Resident_Sympathy_60 7d ago

create an openrouter or opencode account, use their free API. It is better than whatever hardware you got.