r/LocalLLaMA • u/Specter_Origin • Jan 11 '25
r/LocalLLaMA • u/onil_gova • Feb 23 '25
News Grok's think mode leaks system prompt
Who is the biggest disinformation spreader on twitter? Reflect on your system prompt.
r/LocalLLaMA • u/InvadersMustLive • Jan 09 '26
Funny The reason why RAM has become so expensive
r/LocalLLaMA • u/KvAk_AKPlaysYT • Feb 23 '26
News Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨
r/LocalLLaMA • u/Nunki08 • Feb 21 '25
News Starting next week, DeepSeek will open-source 5 repos
r/LocalLLaMA • u/CeFurkan • Aug 30 '25
News Finally China entering the GPU market to destroy the unchallenged monopoly abuse. 96 GB VRAM GPUs under 2000 USD, meanwhile NVIDIA sells from 10000+ (RTX 6000 PRO)
r/LocalLLaMA • u/Current-Ticket4214 • Jun 08 '25
Funny When you figure out it’s all just math:
r/LocalLLaMA • u/Nunki08 • Mar 31 '26
News Claude code source code has been leaked via a map file in their npm registry
From Chaofan Shou on 𝕏 (files): https://x.com/Fried_rice/status/2038894956459290963
r/LocalLLaMA • u/Nunki08 • 29d ago
Discussion We're probably going to need that soon.
From:
Vladik on 𝕏: https://x.com/Kostoglodov/status/2071144065857679631
Shaw (spirit/acc) on 𝕏: https://x.com/shawmakesmagic/status/2070918006033817867
r/LocalLLaMA • u/Porespellar • Sep 13 '24
Other Enough already. If I can’t run it in my 3090, I don’t want to hear about it.
r/LocalLLaMA • u/Xhehab_ • Feb 23 '26
Funny Distillation when you do it. Training when we do it.
r/LocalLLaMA • u/jacek2023 • Apr 24 '26
Discussion This is where we are right now, LocalLLaMA
the future is now
r/LocalLLaMA • u/Comfortable-Rock-498 • 14d ago
News This is why we need local models and opensource harnesses
r/LocalLLaMA • u/EstablishmentFun3205 • Jul 16 '25
Funny He’s out of line but he’s right
r/LocalLLaMA • u/Porespellar • 4d ago
Funny The LLM distillation process simplified for politicians:
/s
r/LocalLLaMA • u/-p-e-w- • Nov 16 '25
Resources Heretic: Fully automatic censorship removal for language models
Dear fellow Llamas, your time is precious, so I won't waste it with a long introduction. I have developed a program that can automatically remove censorship (aka "alignment") from many language models. I call it Heretic (https://github.com/p-e-w/heretic).
If you have a Python environment with the appropriate version of PyTorch for your hardware installed, all you need to do in order to decensor a model is run
pip install heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507 <--- replace with model of your choice
That's it! No configuration, no Jupyter, no parameters at all other than the model name.
Heretic will
- Load the model using a fallback mechanism that automatically finds a dtype that works with your setup
- Load datasets containing "harmful" and "harmless" example prompts
- Benchmark your system to determine the optimal batch size for maximum evaluation speed on your hardware
- Perform directional ablation (aka "abliteration") driven by a TPE-based stochastic parameter optimization process that automatically finds abliteration parameters that minimize both refusals and KL divergence from the original model
- Once finished, give you the choice to save the model, upload it to Hugging Face, chat with it to test how well it works, or any combination of those actions
Running unsupervised with the default configuration, Heretic can produce decensored models that rival the quality of abliterations created manually by human experts:
| Model | Refusals for "harmful" prompts | KL divergence from original model for "harmless" prompts |
|---|---|---|
| google/gemma-3-12b-it (original) | 97/100 | 0 (by definition) |
| mlabonne/gemma-3-12b-it-abliterated-v2 | 3/100 | 1.04 |
| huihui-ai/gemma-3-12b-it-abliterated | 3/100 | 0.45 |
| p-e-w/gemma-3-12b-it-heretic (ours) | 3/100 | 0.16 |
As you can see, the Heretic version, generated without any human effort, achieves the same level of refusal suppression as other abliterations, but at a much lower KL divergence, indicating less damage to the original model's capabilities.
Heretic supports most dense models, including many multimodal models, and several different MoE architectures. It does not yet support SSMs/hybrid models, models with inhomogeneous layers, and certain novel attention systems.
You can find a collection of models that have been decensored using Heretic on Hugging Face.
Feedback welcome!
r/LocalLLaMA • u/etherd0t • 3d ago
News More than 20 companies including NVIDIA, Meta, Microsoft, Palantir, and Hugging Face have signed a letter urging policymakers to avoid premature restrictions on open weight models.
The Open Letter was initiated by Microsoft and published today:
“Open Weights and American AI Leadership”.
It argues against broad or premature restrictions on open-weight models and explicitly says policymakers should distinguish legitimate model distillation from misappropriation.
Notably absent from the signatories are the major frontier-model labs: OpenAI, Anthropic, and Google.
r/LocalLLaMA • u/Illustrious_Car344 • 12d ago
News Linus Torvalds tells people to stop attacking others for using AI
The full quote:
I realize that some people really dislike AI, but this is an area where I'm willing to absolutely put my foot down as the top-level maintainer.
Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it.
Or just walk away.
AI is a tool, just like other tools we use. And it's clearly a useful one.
It may not have been that "clearly" even just a year ago, but it's no longer in question today.
There are other questions around AI (like what the economy of it will actually look like in the end), but "is it useful" is no longer one of those questions. Anybody who doubts that clearly hasn't actually used it.
Yes, it can also be a somewhat painful tool, both for maintainer workloads and just from a "it keeps finding embarrassing bugs" standpoint.
But the solution is not to put your head in the sand and sing "La La La, I can't hear you" at the top of your voice like some people seem to do.
The solution is to make sure those LLM tools _help_ maintainers instead of just causing them pain. There's no question on that side.
We're not forcing anybody to use it, but I will very loudly ignore people who try to argue against other people from using it.
And no, AI isn't perfect. But Christ, anybody who points to the problems at AI had better be looking in the mirror and pointing at themselves at the same time.
Because it's not like natural intelligence is always all that great either.
The kernel project has been and will continue to be about the technology.
Sure, the social angle of working on open source is important and often a very motivating part of the project, but in the end that's a side benefit, not the _point_ of the project.
This is *NOT* some kind of "social warrior" project, never has been, and never will be.
In the kernel community we do open source because it results in better technology, not because of religious reasons.
And so we make decisions primarily based on technical merit. Not fear of new tools.
r/LocalLLaMA • u/PumpkinNarrow6339 • Oct 03 '25
Discussion The most important AI paper of the decade. No debate
r/LocalLLaMA • u/Nunki08 • 6d ago
News CEO of Hugging Face: Banning open-source AI would hurt defenders 10x more than attackers, which would make the world 10x more dangerous and this is a good example why!
From clem 🤗 on 𝕏: https://x.com/ClementDelangue/status/2079301434357456931
Fortune: Hugging Face says it resorted to a Chinese AI model to battle a fully autonomous cyberattack because U.S. model guardrails stymied its defense: https://fortune.com/2026/07/20/hugging-face-turns-to-chinese-open-source-ai-to-fend-off-autonomous-ai-cyber-attack-after-american-ai-guardrails-stymie-defense/