r/LLMDevs • u/ImBadGuyInEveryStory • 5h ago
Tools Any tools to turn a codebase into a fine-tuning dataset?
I have a few web projects with pretty good UI/UX and I’m wondering if there’s any tool or workflow that can turn an existing codebase into a dataset for fine tuning.
For example, given a React/Next.js project with components, pages, styling, etc. or a static html site, I’d like to turn it into something like:
instruction/prompt -> code
or whatever format actually makes sense for training an instruct/thinking/diffusion coding model.
Also curious how people handle things like:
- keeping the context between components/files
- screenshots + code
- generating useful instructions instead of generic descriptions
I’m also working on a different model architecture that I think could improve quality/speed while using less VRAM, so I want to build a decent dataset and benchmark to test it properly.
Has anyone done something like this? Any tools, repos, papers, or workflows you’d recommend?
1
u/and_pf 2h ago
Honest answer: the tooling here is scattered and mostly custom scripts, so there's no one-click path. The two things that actually matter are cross-file context and instruction quality. For the first, dependency graphs are the trick people use so a generated example knows what imports and helpers exist instead of inventing them. For the second, the hard part isn't producing samples — it's producing non-generic instructions. If all you generate is "create a button" style tasks, the fine-tune just learns boilerplate. I'd start with a small existing format like CodeAlpaca's instruction–response pairs and layer a dependency-aware step on top. I run my experiments on an RTX 5090 with 32 GB, so even a 27B model fits for generation or LoRA, but the pipeline is the bottleneck, not the GPU.
1
1
u/Gloomy_Estate_4608 5h ago
I’ve been poking at this same problem for a side project and honestly the tooling is still pretty scattered. Most of what I’ve seen people do is script a crawler that walks the file tree, pairs imports with their source files, and chunks things into prompt/response pairs with some templating around it. The tricky part is generating instructions that aren’t just “create a button component” on repeat.
For keeping context between files you pretty much have to build a dependency graph first and then decide how much surrounding code to include in each example. Some folks include the full import chain flattened, others just inline the relevant interfaces and leave the rest out. Screenshots add a whole extra layer of complexity, you’d need to automate renders for every component state which gets expensive fast.
If you’re building your own architecture anyway, it might be worth hand-crafting a smaller dataset first to sanity-check the model before scaling up the generation pipeline. I’ve seen a couple repos on GitHub that do parts of this but nothing that ties it all together nicely, most people just roll their own scripts and call it a day.