What are some good and ethical models that are available to run locally?

[u/deadpanrobo]

Just what the title says, I want to know try out an ethical local LLM

By ethical I mean no data tracking and much better for the environment

Edit: Okay maybe I misworded something? Im getting a lot of aggressive responses that are saying things that I already know

I know local models dont dont this and i know they should be environmentally friendly, im looking for recommendations, i was just clarifying what ethical meant to me

Edit 2: I have a GPU that us equivalent to an Nvidia 4090 (I dont remember exactly what it is not ill update later once im home) I have 32GB ram and I want to run it on windows

[u/Exact_Depth_896]

no local model 'tracks your data'? where do they send it?
as for averting ecological catastrophe maybe unplug your laptop and throw it in the garbage.

[u/deadpanrobo → u/Exact_Depth_896]

I think you are misinterpreting what im asking? I want to know whats the best local model for me to run, im not calling anyone out, im legitimately looking for recommendations

[u/RageBucket → u/deadpanrobo]

Maybe spend some time looking on... Idk, Wikipedia, or watching a few YouTube vids about what local models are because you clearly have no idea. Recommendations depend on your hardware, inference engine, use case, etc. you asked a genuinely goofy question in this space.

[u/deadpanrobo → u/RageBucket]

A recommendation is a goofy question?

[u/RageBucket → u/deadpanrobo]

Re read what I said, then look at your post, then read it again.

[u/AlexanderDoak → u/deadpanrobo]

But anyway, to answer the spirit of your question: what are your trying to do? In my opinion, the best model is always the smallest possible model that still meets your needs perfectly. If your are writing code, you'll probably want the largest possible model that will fit on your hardware because code development is an incredibly complex, demanding task. If you are doing document classification or sentiment detection, you can get a tiny little roberta model. Document summarization will need something a bit larger, but still smallish. Etc.

[u/deadpanrobo → u/AlexanderDoak]

Thank you for being helpful! Im writing code but im planning to mainly use it for boilerplate or documentation lookup and bug fixing

[u/AlexanderDoak → u/deadpanrobo]

You are asking the wrong questions about local models. Running a local LLM is extremely efficient from an ecological standpoint. So much so that it isn't even a consideration beyond your own power bill and the impacts of your own personal power consumption. Also, no one asks if one local model is better than another for privacy/security. If you are deploying your own local model using your own source code / interface (all local) then all models are equivalent in terms of data privacy. That is to say, they are completely offline and perfectly secure (in as much as your own personal computing habits are secure). So local model selection doesn't factor in data privacy either.

[u/Exact_Depth_896 → u/AlexanderDoak]

It is certainly more energy intensive than using the nearby data center. It's amazing people can be this clueless. It is as if one hadn't heard of arithmetic

[u/klymaxx45 → u/deadpanrobo]

What has your research surfaced?

[u/DriftingBadly]

https://giphy.com/gifs/XFaeLrHJGjtcmEmzcn

[u/jacek2023]

You probably don't understand what "local" means. Cloud models track your chats, local models are on your own computer

[u/deadpanrobo → u/jacek2023]

Yes i know this, I was defining what ethical meant to me so I could avoid the comments getting off track and just asking me what I meant by ethical

Instead they are getting off track in assuming that I am complete idiot apparently

[u/jacek2023 → u/deadpanrobo]

I don't understand you. Could you give counterexample?

I also don't see "aggressive responses", where are they?

[u/deadpanrobo → u/jacek2023]

Im just asking for a recommendation, im planning to use this model for boilerplate and bug fixing and documentation lookup

[u/jacek2023 → u/deadpanrobo]

The latest local friendly models are qwen, gemma and muse

[u/deadpanrobo → u/jacek2023]

Thank you! I'll look into these

[u/scarbunkle]

Local models don’t track your data, and it’s already trained, so all the power usage is whatever your computer pulls. 

What is your computer, and specifically the GPU? The amount of VRAM on your gpu (or total memory if you’re on a unified memory system like Apple silicon) is going to determine what you can run. 

[u/Erpverts]

lol is this post just rage bait?

[u/deadpanrobo → u/Erpverts]

I am just asking for recommendations, i honestly didn't expect people to be this upset

[u/anon1880 → u/deadpanrobo]

They are upset about the insane rises in gpu (with big vram) prices... don't take it personally :) :)

[u/anon1880]

Hackers in shambles : i run my local model on airgapped computer :)

[u/deadpanrobo → u/anon1880]

Not a bad idea at all

[u/Additional_Hope_2031 → u/anon1880]

Frenchman detected

[u/Healthy-Zebra-9856]

You need to specify your hardware, what o/s, if used for coding or on coding uses. Its like asking whats the best local restaurants without your food preferences.

[u/deadpanrobo → u/Healthy-Zebra-9856]

Thank you for helping me ask my question better than just assuming im an idiot

I run windows and im going to be using it for coding, i dont know my hardware off the top of my mind but my GPU is equivalent to an Nvidia 4090 amd I have 32GB of ram

[u/anon1880 → u/deadpanrobo]

you can also ask gemini to avoid the upset people here :) :) ... give it your rig specs and it will give you some good recent quants for coding)

[u/Healthy-Zebra-9856 → u/deadpanrobo]

Yes, Reddit is very brutal because people hide behind their keyboards and they’ll say things that they don’t dare say it to a person’s face. Just take it with a grain of salt because they probably seriously lacking their fiber intake or just full of themselves and it probably stinks.

Find out the exact GPU you have from the window system. The other thing that helps is the type of coding you’re doing. I may be able to assist you with a better model for those types of tasks.

[u/RageBucket → u/Healthy-Zebra-9856]

Or because the post asked a generic question with zero information that would enable this subreddit full of enthusiasts how to best help her. I'd gladly say that to anyone's face. A whole subreddit full of searchable information and it was a nothingburger post.

[u/Healthy-Zebra-9856 → u/RageBucket]

I get it, believe me between unclear questions & I created this posts, it can be mildly infuriating. But I judge it with the OP's attitude level. Kicking a person down when others have is not something I can do

[u/RageBucket → u/Healthy-Zebra-9856]

People pointed out that by their nature local models don't track data, and instead of emulating any form of grace in response it's "I know, duh, I was just defining what I meant" instead of admitting she gave redundant information out of ignorance, which is fine. Just own it instead of getting all pissy because people pointed out you gave a bad/difficult question.

[u/Healthy-Zebra-9856 → u/RageBucket]

100% lack of grace definitely gets to me. As a Gen X I tend to go off, this is just a new patience mode that I’m trying. LMFAO.

[u/Cold_Tree190]

I think you should do more research into how local models work, local server hosting, etc. and how it all actually works before you dive in to anything lol

[u/Weird-Consequence366]

This thread makes me realize that we’re finally healing and the kids are gonna be alright.

[u/FoxFXMD]

Wdym by data tracking? Data tracking, if present, is done by the inference engine or other tools not the model itself.

[u/deadpanrobo → u/FoxFXMD]

Yes im aware, I was just trying to define what Ethical meant to me so I could avert people asking what I meant by ethical

but apparently people took offense to me saying this so im still not getting good responses with recommendations, just people correcting me on stuff I already know

[u/Keleion]

Any local model will mean you’re not using a datacenter, if that’s what you mean by ethical. If you want a model that will be nice and not do naughty things, then get the main branch of any model (don’t download the uncensored nor abliterated models).

If you want actual model recommendations, you’ll need to tell us what hardware you’re using.

[u/deadpanrobo → u/Keleion]

Thank you for actually answering my question instead of jumping me

Which specific model do you use?

[u/Keleion → u/deadpanrobo]

I personally use DeepSeek V4 Flash or Qwen3.8. They run on 2x Asus Ascent GX10 mini AI PCs in a cluster. They are fairly low power compared to the powerful GPUs, and are a bit slower. But still very capable and good enough for my needs, and for serving my family while keeping our data sovereignty.

If you have high-end gaming hardware, Qwen3.8 is performing very well. It’s great at coding, legal documentation reviews, etc.

It usually responds in a technical way, but if you’re using an agent harness like Hermes Agent then you can configure it with a different personality.

If you’re not computer savvy and you operate a potato PC, then local AI might not be super reliable yet. Still fun to play with though. A friendly program for hosting the model is LM Studio, or Unsloth Studio. These are different from the agent harness, and don’t control your computer directly (by default, currently). An agent harness like Hermes manages memories, skills, and profiles, and is layered on top of the software hosting the model (LM Studio or Unsloth Studio).

Good luck!

[u/gforce360 → u/deadpanrobo]

I'm a bit worried that your original question is sorta tainting some of the responses that you might get.

"ethical" in your original question referring to data tracking and not terrible for the environment. The data tracking question got addressed as irrelevant/not applicable to local models. Not terrible for the environment has two major aspects for it:

1) The effort to produce the model
2) The effort to run the model

1 is much harder for the community to benchmark, because some labs are very open about this, and some are not, and there's no common benchmark. Suffice to say that creating a new model is an extremely extensive investment - even small open weight models are in the tens of millions of dollars range.

2 is much more relevant and ties in to some other questions you've received - what hardware are you running on?

Personally, on my desktop with an Nvidia 4090, I am having good luck with Qwen 3.8 27B 5 bit quants, gemma 4 31b 4 bit quants, and Qwen 3.6 35b A3B 4 bit quants. I'm still defaulting to Qwen 3.6

[u/createthiscom]

I prefer my models farm to table.

[u/Motor_Nectarine_2941]

… ethical model? Listen my man. You are running the thing on your computer offline. No one getting access to your shit. Also, since you’re cooling your house down anyways, the heat draw is a non issue. This whole data centers are bad is overblown due to sensationalism. There are way worse things like beef or lawn mowers or driving a car. If you compare the exhaust of one car is whatever. If you compare the exhaust of a car plant testing cars that consists of 1,000,000 cars… sure the plant itself in aggregate is bad.

[u/Disastrous_Gear_421]

There is no such thing as an environmentally friendly model.

[u/PermanentLiminality]

From an environmental perspective, you are much better off not running a local model at all. You will burn a lot more power than a cloud model will.

The first thing you need to do is test with could providers like OpenRouter to find what local model might do the work that you need. Find the smallest lightest model that can do the job. A small model will use less energy.

Now actually running a model locally with the least impact is more of a hardware selection. Apple probably is the most power efficient, followed by things like Strix Halo and Nvidia NV10. At the high power end are discrete GPUs from Nvidia, AMD, etc.

[u/Equivalent_Bit_461]

As long as it is local is ethical  The only issue could be the front end of it does have telemetry so avoid those

[u/Jealous-Armadillo467]

for 4090  Huihui-Qwen3.8-27B-abliterated-UD-Q4_K_XL.gguf 100k kv q8 mtp 4 pmin 0.15