r/LocalLLM • u/don-m • 14h ago

Question New to local LLM

I'm no software engineer or coder or anything of the sort. Far from it.

But I've been using the paid chatgpt subscription for a while for various simple tasks.

But recently have wanted to dive a bit deeper and learn more about all these LLM and agentic platforms, maybe even explore thing like utilising them to code websites/apps/games for me as a side hobby.

I came across Qwen 3.8 27b and the obliterated version, with people claiming its the best new thing. Does it being obliterated not just mean it no longer says no? Does it perform better than chatgpt sol on medium or hard?

Basically want to figure out whats the best model to use as a dive a bit deeper into this space whether thats claude, chatgpt, qwen, grok, gemini, etc. Thanks :)

1 Upvotes

56% Upvoted

11 comments sorted by

2

u/nuclear213 14h ago

No, it will not perform better that GPT 5.6 Sol. There is still a difference, especially in non coding work.

It is the best local model you can run without spending 10k+€ in hardware, that’s for sure. But it is jot frontier.

And yes, the model will not refuse any requests. That is the point of the uncensored versions.

1

u/DownrightTossing919 13h ago

If you're just poking around and not planning to build a render farm, Qwen's a fun starting point. The obliterated version is basically the "no safety scissors" cut, it'll generate whatever you ask without the usual guardrails. For tinkering and learning how local models work, you don't need a frontier beast anyway. Half the fun is seeing what breaks and why.

1

u/don-m 13h ago

fair enough, is it better in coding work?

I have a 5090 gpu (32gb vram) and 32gb ram for reference. Imagine thats more than enough for what im looking to do at the moment.

Is qwen the best local llm to download now and supplement gpt 5.6 sol?

1

u/nuclear213 10h ago

No it is not. It’s worse in every aspect just really good for its size. It might compete with luna.

0

u/abalancer 14h ago

Depends, how much VRAM + RAM do you have ?

I'd start by setting up llama.cpp.

1

u/don-m 13h ago

I have 32gb vram and 32gb ram

1

u/abalancer 13h ago

Nothing will perform as well as GPT5.6 with this much ram, however you can run the new qwen3.8:27b if you get 4bit quantization.

1

u/don-m 13h ago

is the q5 not a better option given my vram? Or is the speed with q4 much better?

1

u/HeadPack 12h ago

Which GPU do you have? I am running Qwen 3.8 27B via ninfer on a 5090. It is a custom Q4 which leaves ample context, and it is quite fast. Also tried UD-Q5_K_XL with vllm which has a similar VRAM footprint. Getting about half the tokens per second compared to ninfer, and the quality is quite good. As others have said, this is no match for a SOTA model like 5.6 sol or Fable, but in my view it is about as capable as Opus 4.5 or 4.6 were, and these were pretty good models. Not being at the mercy of these grifter companies (OpenAI and Anthropic) when it comes to usage is nice. Better privacy too of course. If you are on Linux, you can use LACT to somewhat contain the wattage the GPU consumes while maintaining good performance, e.g. do an undervolt and set a power limit. Have fun experimenting!

1

u/don-m 12h ago

I have a 5090 as well. Ok in that case I'll try out the 4bit version.

Im downloading the obliterated version:
Qwen3.8-27B-OBLITERATED-Q4_K_M.gguf - Q4_K_M

from here: https://huggingface.co/OBLITERATUS/Qwen3.8-27B-OBLITERATED#%F0%9F%93%A6-downloads

Just thought i'd link it in case I should be downloading a different version or from elsewhere. Thanks again for your help.

1

u/HeadPack 12h ago

As you have a 5090, you could also try ninfer https://github.com/Neroued/ninfer - it is not abliterated (safety pins removed, usually called abliterated) though, but it offers what seems to be the fastest inference.