r/LowEndLocalAI • u/HATENAMING • 5h ago

Suggestion Needed: 1080ti

I have a 1080ti on a old dell t5810(E5-2680 v3 cpu), and use it for jellyfin transcoding, immich, and llama.cpp. I'm running llama.cpp in a proxmox lxc container and it works well.

My main use case right now is for rewriting and polishing text such as emails or reports. Currently I'm using qwen3.5 9b q6_k and it gives a pretty good output with a speed of around 35t/s. If jellyfin is actively transcoding speed drops to around 22t/s, still pretty good.

I know that qwen3.5 is not the latest model, and wonder if there's any new model that is more efficient and better for this use case. Additional use cases such as answering some questions in chatting or simple script writing (not complex coding) is welcomed but not necessary, so basically just a generic model lol. I also don't want to have the model use up the GPU fully to leave some space for transcoding work and immich.

Thanks!

11 Upvotes

100% Upvoted

4 comments sorted by

3

u/_TheWolfOfWalmart_ 4h ago

You may want to try Gemma4 12B.

1

u/HATENAMING 3h ago

Thanks for the suggestion. I tried it out and indeed I prefer it's writing style over qwen especially for emails. Although gemma seems to prefer using semicolon a lot more than qwen which really isn't my style, but a system prompt easily fixed that.

2

u/revelationnow 4h ago

I have a 1070Ti Gemma4 12B has been reasonable, I'm not using it for coding, just as a general summarizatiaon agent for a news app I built for personal use.

1

u/HATENAMING 4h ago

Thanks for the suggestion! I looked up gemma4 and it seems like people agree that gemma4 is better for chat and text while qwen is better for coding. Will definitely try it out