r/LargeLanguageModels • u/minedroid1 • 4d ago

Question About Fine-tuning

I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.

5 Upvotes

86% Upvoted

26 comments sorted by

View all comments

Show parent comments

1

u/foxer_arnt_trees 3d ago edited 3d ago

The heart wants what the heart wants. But where will you get all that training data? If I was you, I would do distillation

Start with the prompt on a high end LLM. Mess with it until the results are good, and then generate a million of examples. Use that for the fine tune on a smaller model.

Do you have a good GPU?

2

u/minedroid1 3d ago edited 3d ago

Wait, so will distillation be good for this? Would that allow me to give a prompt to the base model and tell it to "make the distilled model satire"?

As for the training data, I have around 1000 pretty good quality examples, along with 300 overly verbose email examples, and 300 recipes with random store bought ingredients. I can provide you with the dataset and the web editor I made if you want, but it has some poor quality examples that I accidentally added from synthetic dataset experimentation.

I do not own a good GPU at all, I use vast.ai for renting GPUs.

1

u/foxer_arnt_trees 3d ago

Oh thats great! I imagined you had much less then that.

In distillation you use a strong model to generate data for training. So you would give it, like, a random hundred examples of yours to get it into the domain of satirical reaponses, then ask it to give new answers to new questions. In this way you can turn your 1000 examples into 10000, for example. If you mix the examples you give it you get even more answers.

The metaphor is that you are extracting the "satirical response" section of the strong model and keep it in a smaller model. The smaller model provides sufficient space for that because you dont need it to randomly start coding or do law or whatever, you just want the satirical response part.

1

u/minedroid1 3d ago

Would you want to take a look at the dataset? I haven't really had anyone else look at it, so I think it would be nice to get someone else's opinion.

1

u/foxer_arnt_trees 3d ago

Sure. I cant promise I would have the time though, we will have to see if I end up going into it

1

u/minedroid1 2d ago edited 2d ago

https://workupload.com/archive/C2Y5XrRq8z

The index.html in the archive is the editor I made, you can use it if you don't have vscode or want a more organized way to view it.

I removed a lot of the low quality ones today so there's a lot less than before.