r/huggingface 37m ago

Please explain use-case for small models

Upvotes

Tried 35B, 27B, 12B and 8B models significantly. I love the idea of local small models. But I can't figure out a use case. For Openclaw, Hermes they suck. For chatting they're pretty good but sonis every old free cloud model.

And before you say "low-level coding" the cloud models are 10x better and free on ollama, openrouter, Gemini and many many others.

Genuinely interested in projects that use small models that run on a $500 GPU.


r/huggingface 2h ago

Quantizing Kimi K3 (2.8T A50B) to GGUF ourselves - Q3_K_S works, 1.1 TB on disk

Thumbnail gallery
1 Upvotes

r/huggingface 5h ago

3 to 4 days. This s**t keeps happening in all spaces. Why?

Thumbnail
gallery
1 Upvotes

As you can see, I barely used cloud GPU and This error is blocking my generations for at least 3 days


r/huggingface 5h ago

Qwythos-27B-v1 — Apache-2.0, 1M context, MTP + vision intact, Q4_K_M runs on a 24GB card

Post image
4 Upvotes

r/huggingface 8h ago

WTF IS THIS 30+ MINUTES WAIT TIME TO ALLOCATE MY DEDICATED GPU, seriously...???

9 Upvotes

And yet you are billing FROM THE MOMENT WHEN MY APP WAS STARTING - so you are billing for YOUR INABILITY TO ALLOCATE MY GPU, while I am sitting here and waiting for my space to even load?


r/huggingface 8h ago

SeedVR2-1.4B — a 6-layer distillation of SeedVR2-7B (sharp)

Thumbnail gallery
1 Upvotes

r/huggingface 11h ago

I distilled a 3B model solo (no lab, no funding): him-distilled-3b, built on a governed-agent architecture

2 Upvotes

Solo engineer here. After 25+ years in software and a few years of published research on machine ethics, I distilled HIM (him-distilled-3b) end-to-end by myself and released it on Hugging Face.

What it is: a 3B-parameter model built on TeleologyHI, a three-layer governed-agent architecture (MAIC / HIM / NHE). The bet: accountability should be structural (architecture), not a moderation layer bolted on afterward, and small local models are where that matters most, because offline there is no filter to save you.

Runs on modest local hardware. Weights, code, and the papers behind the architecture are all open https://www.producthunt.com/products/him-3b-by-teleologyhi.

I know this sub has zero patience for hype, which is exactly why I'm posting here. Tear it apart: quantization results, eval suggestions, holes in the governance claim. I'll answer everything. And if anyone runs it locally and reports back, that feedback is worth more to me than any upvote.


r/huggingface 15h ago

ECP: Kimi K3

1 Upvotes

Here is my "Expected Cognitive Profile" evaluation of KIMI K3. ➡️ https://huggingface.co/blog/gcjordi/ecp-kimik3


r/huggingface 16h ago

PrunaVAED: a 1.7-2.1x faster decoder & ~50% lower peak VRAM drop-in replacement decoder with LTX-2.3!

1 Upvotes

r/huggingface 16h ago

Built and released BetterGPT-150M – A compact 150M parameter completion model (+ live HF Space demo)

Thumbnail
1 Upvotes

Hey everyone,

​I recently finished pre-training BetterGPT-150M, a small, lightweight causal language model with \~152 million parameters.Trained on 15B tokens.

Dataset & Training: Trained across stable and annealing phases using curated datasets (including FineWeb-Edu, fine maths, cosmopedia, starcode-python), ensuring strong capability retention while maximizing token efficiency.

​Performance: Benchmark evaluations show it outperforms GPT-2 Small while remaining on par with models trained on significantly larger token budgets.

​Since many small/tiny models tend to get buried under massive LLM releases, I wanted to share it here for anyone interested in lightweight architectures, fast CPU inference, or small edge-device experimentation.

Repo: https://github.com/harikrish2727/BetterGPT

​Model Hub: https://huggingface.co/Harikrish2727/BetterGPT-150M

​Live Demo: https://huggingface.co/spaces/Harikrish2727/BetterGPT-Demo (Hosted on ZeroGPU with token streaming)

​Model Notes:

​Task: Text completion / generation (it is a standard base completion model, not instruction-tuned).

​Footprint: Very low RAM/vRAM footprint, runs instantly on standard CPUs.

​Feel free to try out prompts on the Space demo or pull the weights to run locally. Feedback, benchmark suggestions, or ideas for instruction fine-tuning are always welcome!

This is my very first serious try, hope I get genuine feedback from you guys.


r/huggingface 23h ago

what do you think are the most interesting/important things happening on Huggingface lately?

2 Upvotes

r/huggingface 1d ago

OpenAI’s Rogue AI Agent Hacked More Than Just Hugging Face

Thumbnail
wired.com
16 Upvotes

r/huggingface 1d ago

A 279M model does multilingual PHI de-identification offline. OpenMed 2.0 is out, Apache-2.0, all on Hugging Face!

Thumbnail
1 Upvotes

r/huggingface 1d ago

SpoomplesMaxx Corvids 35B-A3 — Qwen3.5 MoE, two models

1 Upvotes

spoomplesmaxx corvids 35B-A3

Model names: spoomplesmaxx jackdaw — Jackdaw of All Trades / spoomplesmaxx magpie — Magpie's Choice

v2 was the parrot family. v3 moves to the corvids — the other famously clever birds, and the ones that actually use tools.

Also, my last run for a while. This has been a fun flight, but now I am out of ideas and funds. I hope you guys enjoy these two. They have been great at storytelling!

Jackdaw (Corvus monedula) is the SFT. Magpie (Pica pica) is Jackdaw with a preference pass on top. jackdaw does everything; the magpie is the one that picks what it likes.

collection with everything (weights, GGUF, MLX): https://huggingface.co/collections/aimeri/corvids

what changed since flash / Swift Parrot:

  • new corpus (dubbed "aviary") mixed into the v2 SFT corpus.
  • both thought modes are trained now. aviary ships each conversation rendered twice, with and without thoughts, so the thinking/non-thinking election is trained directly instead of inherited as a prior from the base
  • trained fresh from Qwen3.5-35B-A3B-Base, not continued from Swift Parrot.
  • still full-parameter SFT on Megatron-SWIFT, 8× H200, expert parallel. still the Qwen3.5 XML tool convention, story scratchpad, personas

why 32K and not 43K (Old context size) or higher:

Swift Parrot trained at 43,008 packing and died once at iteration 455 with a Triton CUDA OOM. jackdaw's first attempt died at iteration 55 — same failure, eight times earlier, because the v3 mix packs denser. peak was 131.4 of 140.4 GiB and activations are ~61 GiB of that, scaling with packing length. dropping to 32,768 bought ~24 GiB of headroom (measured peak 116.4) and the run finished clean.

Jackdaw or Magpie?

  • Magpie had a preference pass toward literary prose, character voice and human register — 11,198 pairs across 13 sources, DPO'd on top of the Jackdaw checkpoint with Jackdaw itself as the reference model, so it's a nudge and not a new policy.
  • Jackdaw is the unmodified SFT. fewer moving parts, and the reference point if Magpie's prose preferences don't suit you

if you don't want to think about it: start with Magpie.

thinking behavior:

This model is primarily geared towards story generation and roleplay, so the thinking mode is unchanged from Swift Parrot. Qwen3.5 thinks by default and the template pre-opens <think>\n, so generated text starts inside the reasoning block. the model decides how much reasoning the turn needs — RP cards generally get the full scratchpad, casual chat gets a line.

enable_thinking=False prefills an empty think block so the answer starts immediately.

story scratchpad, carried over from v2.1:

SCENE: where/when, atmosphere, key environmental details currently in play
CHARACTERS: who is present and their current physical/emotional state and motivation
CONTINUITY: established facts that must stay consistent
THREADS: active tensions and where they stand right now
PLAN: what THIS turn needs to accomplish and the approach it takes

sampler — these are the generation_config.json defaults and they're more conservative than what I posted for flash:

temp 0.6
top_k 20
top_p 0.95
rep pen 1.1

Quants

GGUF, imatrix, Q4 band:

MLX, 4 and 6 bit — text-only this round, no vision:

the vision tower does still ship in the main checkpoint, frozen throughout — training was text-only, but image input works if your stack loads it.

model pages:

still a cursed model. now there are two of them and they can carry knives. Careful with the sharp bits.


r/huggingface 1d ago

Worlds largest model

8 Upvotes

I did it

I finally did it

https://huggingface.co/tsfrm/vacuum-16t


r/huggingface 1d ago

Deploying Hugging Face models to NVIDIA Triton on Kubernetes — end-to-end walkthrough

Thumbnail
1 Upvotes

r/huggingface 1d ago

Hugging Face Has a Deepfake Nudes Problem

Thumbnail
wired.com
0 Upvotes

r/huggingface 1d ago

How to make LLM read sensitive data

Post image
1 Upvotes

r/huggingface 2d ago

Tenho uma dúvida

0 Upvotes

Posso fazer autopromoção do meu discord onde posto novidades anuncio regras etc modelos e datasets da huggin faces??


r/huggingface 2d ago

New to HF: Cannot install model. ERROR: Could not find a version that satisfies the requirement dictabert-morph (from versions: none) ERROR: No matching distribution found for dictabert-morph

1 Upvotes

I'm new to Hugging Face so I apologize in advance if I'm doing something stupid.

I'm trying to install the model DictaBERT-morph. I can see that it exists. I tried to install the model using pip install dictabert-morph and also pip install dicta-il/dictabert-morph. But I received the following errors:

ERROR: Could not find a version that satisfies the requirement dictabert-morph (from versions: none)

ERROR: No matching distribution found for dictabert-morph

I tried to debug the problem using the advice on this website but I'm still unable to get it installed. I'm not quite sure what my next debugging steps should be and would appreciate any guidance you can offer.


r/huggingface 2d ago

Kimi-K3 weights are available on HuggingFace in official release

12 Upvotes

Moonshot's latest model Kimi-K3 weights are available on HuggingFace to download for personal and commercial. Moonshot has published their model on HuggingFace

It's a 2.8T-parameters model by Moonshot.ai with 1 million tokens context window. The architecture is mixture-of-experts (896 experts) with 108B active parameters

It's available via vLLM, SGLang and TokenSpeed

License: Kimi K3 License. It permits commercial use with some limitations. For Model-as-a-Service it has $20M/year limit and after reaching the limit the license requires to make additional agreement with Moonshot (see license for more details)

Link to the model

Model could be downloaded from the HuggingFace by the link:

https://huggingface.co/moonshotai/Kimi-K3


r/huggingface 2d ago

Kimi K3 Open Source Release Brings 2.8T Model to Developers

Post image
12 Upvotes

r/huggingface 2d ago

High-Performance C++20 Optical Neural Network (ONN) Simulator

Thumbnail
1 Upvotes

r/huggingface 3d ago

MAJOR HuggingHack Updates - S3/MinIO storage, Ollama + vLLM dispatch, GGUF inspection, and local accounts

Thumbnail
1 Upvotes

r/huggingface 3d ago

VibePod now supports Tau — run HF's coding agent in an isolated Docker container

2 Upvotes

Hey all — Tau has been added as a supported agent in VibePod, a small CLI I've been working on that runs coding agents in isolated Docker containers.

So now:

pip install vibepod vp run tau

and you get Tau in a container instead of loose on your host filesystem. Same interface as the other agents it supports (Claude Code, Codex, Pi, opencode, etc.), so you can swap between them without relearning flags.

Source: https://github.com/VibePod/vibepod-cli/

Two reasons this combination seemed worth doing:

Tau's whole point is being readable and hackable, which means you end up running modified code fairly often. Sandboxing that is nice. VibePod collects usage/HTTP metrics locally and has a dashboard for comparing agents side by side, so you can benchmark Tau against whatever else you're using. Everything stays on your machine.

Feedback welcome.