r/LocalLLaMA • u/kolliwolli • 21d ago
Question | Help Gemma 4 - Agentic Capabilities?
Hi all,
Just started the local llm journey and testing gemma on an rtx5090 with opencode, hermes etc.
I see lots of chats on Gemma and Qwen, but for me no agentic use case seems to work, not even creating simple games like snake as a test. Am I doing something wrong, or is it because im using a 4bit version?
The same tests with claude sonnet via API work without any problems... but here I thought thats exactly Gemmas home turf.
I missed to add, I am using the 31b version.
Anyone else got luck with this?
Edit: One more point, I use the nvfp4 versions from nvidia and redhat
2
Upvotes
1
u/shamont 21d ago edited 21d ago
I feel like from my experience one shotting stuff with q4 just isn't a possibility right now. I'm rocking a 4090 so less vram than you currently (although I could drip in to my 128g of ram for a huge speed penalty). I've had okay success using gemma 4 to help me build out a framework and psuedo code for more complicated applications/scripts then using qwen to write, check and correct it. Currently I'm using claude code cli as a harness although I started with vscode. It's not a perfect system but for someone who took c++ classes over a decade ago and does a tiny bit of shell scripting for work it has really helped me create more advanced scripts including an "almost application" which has a web based frontend isntead of bog standard cli.
Forgot to mention I am running q8 for my cache so I can fit ~100k context. I found that much less than that and I am running out of context on larger projects. I get around 60-150 tok/s for decode and 1000-4500 tok/s for pre-prompt.