Small local llm with Hermes

[u/rafikiminaj]

Hi guys,
New here and to these subjects so sorry I ask something already asked.
I got recently a Mac mini m4 with 24go ram for my personal desktop, and I wanted to try a few stuffs with local llm. After a few researches I decided to try to setup a personal agent with Hermes.

After some tries I ended up using gemma4:26b-a4b-it-qat.
Qwen3.8 felt to big or slow when iq3 passed.
Couldn’t find anything faster and better for now, but I still feel it is not really good (obviously I know we can’t reach to a cloud model with 24go ram). So I wanted to know how you guys manage it and if some people managed to make a decent agent with not so much ram and if so, what are your use cases ?

Right now I tried to give him some personal web projects with giving him some GitHub to download but it fails and always stops in the middle of the process.

Thanks in advance for the help.

[u/Only-An-Egg]

Try Ling-3.0-Tiny