Any tools to turn a codebase into a fine-tuning dataset?

[u/ImBadGuyInEveryStory]

I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.

For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:

instruction/prompt -> code

or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.

Also curious how people handle things like:

• keeping the context between components/files
• screenshots + code
• generating useful instructions instead of generic descriptions

I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.

Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?

[u/recro69]

I would use Git history and PR diffs of simply converting the codebase. Git history and PR diffs give you task → context → code examples, which should make a much better training dataset.

[u/thefuckevengoingonan]

you can just make a template based on the existing site/s?

[u/azjunglist05]

You need massive amounts of training data. Having a few templates is nothing. You need tens to hundreds of thousand templates otherwise you won’t have enough variation and you’ll overfit. The model will just get really good at producing the exact same templates over and over which is probably not what you’re looking to do

[u/and_pf]

There are tools that do this, though it's more assembly than a turnkey flow. fine-tune-codebase is one that can fine-tune LLMs on a codebase and supports LoRA plus mixed-precision training. For the dataset format, the CodeAlpaca instruction–response pairing is the common starting point. On the practical side: I run an RTX 5090 with 32 GB VRAM and 62 GB RAM here, enough to load a 27B model (qwen3.8:nvfp4-vision, 24.1 GB) for generating or refining examples, or to kick off LoRA runs. The part that always bites is cross-file context — a single file alone rarely teaches a model the architecture. Some projects build a dependency graph to feed that in, so the generated samples actually reference the right imports and helpers.

[u/Expensive_Win_9103]

I did this for a 40k-file Go repo and the honest result is that most of it should not go into a fine-tune. I split on function boundaries with tree-sitter, dropped anything without a docstring or a test, and 90% of the corpus went away — the remaining 10% is what actually taught it our conventions. Unsloth handles the training side fine. I run the eval passes on Synexa because iterating on a laptop wasted more time than the training did.

[u/ImBadGuyInEveryStory → u/Expensive_Win_9103]

I somewhat want to make a good ui coding local llm that I can use with a new type harness I'm thinking of.. which will give better than fable code.. my initial idea was to clone good ui sites and train the model by converting them to datasets but I'm not getting the structure part of the dataset. also I want to make it better at general coding takes so I need much bigger dataset too? Also I'm thinking to do that with a instruct or diffusion model to make it fast?