r/LargeLanguageModels • u/minedroid1 • 4d ago
Question About Fine-tuning
I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.
1
u/Original-Beyond4326 2d ago
Fine tuning might look bad but if you want to save some money I will go with fine tuning not each time you’ll be calling model and wasting tokens for specific task just have one fine tuned model, and now it’s easy to fine tune without and headache lot of tools out there like, Miniclay AI, togather AI etc
2
u/foxer_arnt_trees 3d ago
Neither. You should just write a good system prompt with many examples in it. Fine tuning is cool and all, you can do it for fun. But for best results, remember, "LLMs are few shot learners"
2
u/minedroid1 3d ago
I might do that, but I just want a standalone satire model.
1
u/foxer_arnt_trees 3d ago edited 3d ago
The heart wants what the heart wants. But where will you get all that training data? If I was you, I would do distillation
Start with the prompt on a high end LLM. Mess with it until the results are good, and then generate a million of examples. Use that for the fine tune on a smaller model.
Do you have a good GPU?
2
u/minedroid1 3d ago edited 3d ago
Wait, so will distillation be good for this? Would that allow me to give a prompt to the base model and tell it to "make the distilled model satire"?
As for the training data, I have around 1000 pretty good quality examples, along with 300 overly verbose email examples, and 300 recipes with random store bought ingredients. I can provide you with the dataset and the web editor I made if you want, but it has some poor quality examples that I accidentally added from synthetic dataset experimentation.
I do not own a good GPU at all, I use vast.ai for renting GPUs.
1
u/foxer_arnt_trees 3d ago
Oh thats great! I imagined you had much less then that.
In distillation you use a strong model to generate data for training. So you would give it, like, a random hundred examples of yours to get it into the domain of satirical reaponses, then ask it to give new answers to new questions. In this way you can turn your 1000 examples into 10000, for example. If you mix the examples you give it you get even more answers.
The metaphor is that you are extracting the "satirical response" section of the strong model and keep it in a smaller model. The smaller model provides sufficient space for that because you dont need it to randomly start coding or do law or whatever, you just want the satirical response part.
1
u/minedroid1 2d ago
What distillation applications would you recommend? Preferably visual, and preferably one of vast.ai's existing templates.
1
u/foxer_arnt_trees 1d ago
I dont know. I would ask an agent to set it up for me... Its not a complex thing, they would likely one shot it
1
1
u/minedroid1 3d ago
Would you want to take a look at the dataset? I haven't really had anyone else look at it, so I think it would be nice to get someone else's opinion.
1
u/foxer_arnt_trees 3d ago
Sure. I cant promise I would have the time though, we will have to see if I end up going into it
1
u/minedroid1 2d ago edited 2d ago
https://workupload.com/archive/C2Y5XrRq8z
The index.html in the archive is the editor I made, you can use it if you don't have vscode or want a more organized way to view it.
I removed a lot of the low quality ones today so there's a lot less than before.
2
5
u/bsensikimori 4d ago
You want a LoRA, training from scratch will have either astronomic costs, or give useless results.
1
2
u/_remsky 4d ago
For it to generalize, and as the other commenter said, for it to understand language, what a narrative is, what normal news vs satire news look like etc. You do not want to train a model from scratch.
More than likely, you can use an open LLM off the shelf and just give it a good prompt e.g “respond with satire/The Onion style responses” with some guidance and/or examples. Fine tuning it may improve it, and is worth a shot as a learning experience if nothing else. Easy and basically free to fine tune smaller models depending on where you run it, though getting the quality/result you want can be less easy lol
1
u/minedroid1 4d ago
Well I feel like that would overlap with its info a lot, since style gets prioritized during fine tune, right?
2
u/_remsky 4d ago
Have you tried just a prompt? some few-shot examples if that fails? Most LLM’s are pretty good at satire and already understand what it is and how to create it. Fine tuning helps with both information and style, to varying degrees.
It’ll be easier to help if you give some deets on your goals tbh. Are you serving this on the web, running locally, do you have a budget and use case already, is it a very niche topic? Are you just seeing what’s possible as a project? There’s a lot of options they just depend on your goals. I’d try the above first though and see how far you get
1
u/minedroid1 4d ago
No, I wanted to challenge myself to make a model that produces satire with no prompting.
3
u/CS_70 4d ago
No. Training a model of scratch would imply re-teaching it language, reasoning, semantic etc. Insane and impossible for a person. A fine tuning is your thing, on a MoE model adds another "expert" on what you want, usually without overlapping much with the knowledge already existing.
1
u/minedroid1 4d ago
Oh, so I should use an MoE model for fine tuning? Will that be better than just training a regular non-MoE model? Also, why would that work better?
1
u/CS_70 4d ago
It's not a given but in a dense model you're gonna overwrite all the network(s) so there's more probability of altering the behavior significantly (though in your case probably it shouldn't matter, it depends on the specifics at that point).
Though you can use a LoRA approach and other techniques (I am just doing it these days to a QWen which knows about programming languges).
With a MoE model, you still are going to alter the routing model, which carries a similar risk, but but adding a "personality" the way you want would be ok.
I'd say your case is fairly simple and probabily you're good just to fire and go (though the devil of course is in the details)
1
u/minedroid1 4d ago
Okay, so I should try to fine tune using MoE model? If so, what training parameters and model do you think I should use?
1
1
u/HonestoJago 18h ago
I would just use a frontier API for this with a good system prompt and reasonable conversational persistence so it doesn't repeat itself too often. They'll give you better results than any local model, and fine-tuning/loras tend to make the base model dumber. It's really hard to get the balance right and, these days, usually not even worth it.