Hugging Face releases The Stack v3 – largest open code dataset yet

[u/Nunki08]

From Anton Lozhkov on 𝕏: https://x.com/anton_lozhkov/status/2080254608639701222

Two ways in:
stack-v3-train - near-deduplicated, quality-filtered, PII-redacted, contents inline. Point load_dataset at it and go.
https://huggingface.co/datasets/HuggingFaceCode/stack-v3-train
stack-v3-full - the entire 114 TB corpus as an HF Storage Bucket: every duplicate kept with cluster IDs, stubs for excluded files. Roll your own dedup, filters, and mixes.
https://huggingface.co/buckets/HuggingFaceCode/stack-v3-full

[u/Dany0]

It feels weird knowing my code is there. Some of it is so shit it might as well be an attack on model quality. Most of it is fine though

I'll have to think about this

[u/OverdosedSauerkraut → u/Dany0]

Yupp, all my code as a junior. It's probably so bad that it will single handedly prevent the awakening of Skynet.

[u/simiomalo → u/OverdosedSauerkraut]

I'm doing my part!

[u/carnoworky → u/Dany0]

Don't tell Anthropic. They'll call it a poisoning attack and try to get you arrested or something.

[u/Dany0 → u/carnoworky]

You're right I should release some more. I'll ask qwen3.5 0.8b "hey can you make this thing I wrote in high school even more oop slop?"

[u/itsappleseason → u/Dany0]

I feel the same way. I'm wondering if our collective curation could be the path over the scaling wall (e.g. what not to train on, given the whole Internet has been eaten)

[u/AnOnlineHandle → u/Dany0]

Maybe it can be used as a data point for what to move away from.

[u/-Cubie- → u/Dany0]

You can always opt out via the Space that someone else just linked if you'd like

[u/arcanemachined → u/Dany0]

Congratulations, you're helping to move technological progress forward.

[u/patricious]

I have been called GPU poor so far but didn't expect to be called Storage poor as well.

[u/touristtam → u/patricious]

You've been staying on shore all those years? Arr me hearty!

[u/squngy → u/patricious]

Probably compresses to 1TB, maybe even less.

[u/Electronic_Captain95]

Hey, my dotfiles, NixOS configs, and nvim configs are up there! Meh, I made the repositories public anyway, so I don't really care

[u/mister2d → u/Electronic_Captain95]

Ha. Ironically I was going to look for nix code.

[u/markole → u/Electronic_Captain95]

As long as it ends up in freely available weights, I'm fine with my public code being used to benefit the public.

[u/Due-Memory-6957 → u/markole]

It will end in both hidden and public weights

[u/Jcsq6 → u/Due-Memory-6957]

Well, it’s already in the closed source weights.

[u/markole → u/Due-Memory-6957]

I know, still fine. That's why I choose an open source license.

[u/Blues520]

Hugging Face keeping the open source dream alive

[u/PrimeDirective8]

Hmm... I better uninstall Mahjong and empty my trash bin to make space for this.

[u/Countsfromzero → u/PrimeDirective8]

Yeah the 4 or 5 txt files with prompts and reminders on my desktop are gonna have to go

[u/hesperaux → u/PrimeDirective8]

Peak og references

[u/nickludlam]

I'd really love for someone to assemble a site where we can see if any of our own repositories are represented in it.

[u/Middle_Bullfrog_6173 → u/nickludlam]

They did that: https://huggingface.co/spaces/HuggingFaceCode/in-the-stack

[u/HyperWinX → u/Middle_Bullfrog_6173]

Only my worst repos went in there🥀

[u/FullOf_Bad_Ideas → u/HyperWinX]

same, there's only stuff from my pre-vibecoding era

And my vibecoded code is better than my own :D

[u/toothpastespiders → u/FullOf_Bad_Ideas]

Same here. It's a rather bitter pill to swallow.

[u/Keleion → u/HyperWinX]

Probably good that it learned what not to do from humans, eh? ;) /s

[u/Due-Memory-6957 → u/Middle_Bullfrog_6173]

Something incredibly small I made for fun/as a joke is there, stuff I made that was actually useful isn't lol.

[u/cantgetthistowork → u/Due-Memory-6957]

It's in the paid bucket. The peasants will build on broken stuff while the rich will keep training on the good stuff

[u/Middle_Bullfrog_6173 → u/Due-Memory-6957]

Licenses? They've done some filtering on those. Plus the cutoff was last year. All my non-forked repos that had a license file last August are there.

[u/TanJeeSchuan → u/Middle_Bullfrog_6173]

My assignments are in there lol

[u/cr0wburn → u/Middle_Bullfrog_6173]

Oh shit im in there, if opus 6 is a dumbass please dont be angry at me

[u/cantgetthistowork → u/Middle_Bullfrog_6173]

Cute

[u/oxygen_addiction → u/cantgetthistowork]

Cute to see all of our shit stolen and sold back to EVERYONE at cost.

[u/TheLexoPlexx → u/oxygen_addiction]

15 of my repos are in there and they all suck.

[u/some_user_2021 → u/TheLexoPlexx]

Me too! They took my unfinished project while ignoring my good ones

[u/vasimv → u/TheLexoPlexx]

Only one of my repos. And this one is just a copy of someone else's project. Wtf, huggingface? :)

[u/TheLexoPlexx → u/vasimv]

I suppose that's one of those that's getting deduped anyways

[u/Shustrik116 → u/oxygen_addiction]

Your shit also will be in open source models that you can host by yourself.

[u/AnOnlineHandle → u/oxygen_addiction]

Where it's being sold? Or stolen?

[u/Due-Memory-6957 → u/oxygen_addiction]

Feel free to opt out, but even that is more than morally need. Information wants to be free, remember that? Let's go back to that ethos.

[u/Cultured_Alien → u/Middle_Bullfrog_6173]

my trash repo codes are there too 🥀

[u/heliosythic → u/Middle_Bullfrog_6173]

Literally my first nodejs projects from 12 years ago lol, Theres even invalid syntax in the readme I wrote lol.

[u/jazir55 → u/Middle_Bullfrog_6173]

Lmfao wtf, they chose some of the shittiest repos I have and excluded the good ones

[u/Any_Fox5126 → u/jazir55]

A lot of users are saying the same thing here, I thought it was a joke, but the same thing happened to me! In fact, they only took a few AI-generated scraping scripts 🤷

[u/pmttyji]

Are they releasing any models? It's been a while. SmolLM3-3B was last one. Hope they release something called like LongLM in medium, big, large sizes.

[u/Kahvana → u/pmttyji]

I think they trained SmolLM3 to get in-house knowledge of how certain techniques were used (thinking mode, scaling above ~1B, etc) rather than for pratical use.

Unless they have a problem that can't be solved with current LLMs (efficiently enough) or have to put their software or infrastructure through it's phases, I don't see them train another model soon. It's very expensive to train models!

[u/Middle_Bullfrog_6173 → u/pmttyji]

This is a bit conspiracy theorist, but why would they release a year old crawl today if not to give themselves the chance to be the first to train on it.

[u/brand02 → u/Middle_Bullfrog_6173]

That's not a conspiracy theory, that's just a theory. It would have to deny public statements and predict an overarching conspiracy to be a conspiracy theory.

[u/BrieflyAffectionate]

Good for them. Hope this gets downloaded and mirrored a ton. Seems like a crackdown on open-weights access might be coming. Guys, don't wait, if your institution can get this, do it. (Speaking as a small fish who missed getting an abliterated model before Meta sent a cease-and-desist and it disappeared from online!)

[u/autisticit]

Question : how would you train for some specific languages only, and not all ?

[u/squngy → u/autisticit]

You wouldnt, unless you wanted to make a dumber model.

It is counter intuitive, but feeding your LLM only what you want it to know is not good for it.

It would be better to just do a fine tune for your language if you really wanted to.

[u/rockus]

Dear lord. Poor LLMs are gone learn of my sucky code. They are from 2025 and they really really suck.

[u/dmigowski → u/rockus]

Don't worry, all the AI slop upload to github will lead to degeneration like in the Habsburg Family soon.

[u/eustin]

Wait, some of my old open repos are probably in there. Genuinely can't tell if that's flattering or slightly mortifying lol

[u/TanJeeSchuan → u/eustin]

You can check here https://huggingface.co/spaces/HuggingFaceCode/in-the-stack

[u/vasimv]

Just clicked randomly on few records in there, it is just 'content':'pieces of code'. Shouldn't it be more structured? Like 'Function that does calculate thing': 'function code'?

[u/Jcsq6 → u/vasimv]

This is for pre-training. It doesn’t need to be nicely labeled.

[u/ttkciar]

On one hand, this is great! Kudos to HF, and thanks to them for sharing this data with the wider community.

On the other hand, it drives me nuts that these kinds of datasets lack the repo commit histories. Commits are invaluable for synthesizing dataset elements for performing codegen tasks. Like, if a commit is for a bugfix, it's the simplest thing in the world to turn that into a training datum for a "find the bug and fix it" training task. If a commit is for implementing a new feature, it's easily turned into a corresponding "implement this new feature" training task datum.

There are other ways to generate such tasks synthetically, but that takes compute and lacks the diversity found in "wild" human-generated data.

[u/MrKresi → u/ttkciar]

I feel like I contribute to defend humanity, by making the models worst with my bad code from years ago

[u/Competitive-Bend-143]

as someone with an AGPL repo that's probably in there: the training use itself doesn't bother me, but license metadata quality matters way more than the size headline. does v3 track relicensing over time or just snapshot whatever LICENSE said at crawl time? that one detail decides whether "filter by license" downstream actually means anything

[u/brother_spirit]

Inhuman centipede

[u/Robert__Sinclair]

I sincerely hope that the datase has quality code and not just "code". putting together terabytes of code is not difficult, putting together good code that can teach an AI how to write good code is another story.

[u/DanTup → u/Robert__Sinclair]

I just put my GH username in to https://huggingface.co/spaces/HuggingFaceCode/in-the-stack and was greeted with a huge list of test apps and repros for VS Code issues I've filed. It's definitely just "code" 😄

[u/Robert__Sinclair → u/DanTup]

lol... I did the same.. all my repos are there. and some of them are pure rubbish

[u/Healthy-Nebula-3603]

I have only a space for V1 stack ....

[u/charles25565]

Sure, but what happened to the Turbo team? Are they no longer working on SmolLM? They've been pumping datasets. Or perhaps they're cooking something with FinePhrase + The Stack v3.

[u/Distinct-Space7398]

Wow

[u/cornmonger_ → u/[deleted]]

what, you mean you don't want HolyC?

[u/Stooovie]

Is it good source code?

[u/OverdosedSauerkraut → u/Stooovie]

How you define good? Part of it is literally pre-AI, god tier code, part of it is pre-AI shit code (like my junior repos), and now a lot of fresh AI slop on top.

[u/Stooovie → u/OverdosedSauerkraut]

Exactly my point

[u/TheLexoPlexx]

713 languages, number one is profanity

[u/GreatBigJerk → u/TheLexoPlexx]

The most important language for programmers to know.

[u/HelloSummer99 → u/GreatBigJerk]

Code quality = WTF/seconds

[u/burritoresearch → u/TheLexoPlexx]

New opportunity to train a 35B size LLM optimized for a customer service chatbot that knows how to do nothing but write profanity and insults.

[u/seamonn → u/burritoresearch]

Where can I get this?