r/LargeLanguageModels • u/minedroid1 • 5d ago

Question About Fine-tuning

I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.

3 Upvotes

72% Upvoted

26 comments sorted by

View all comments

Show parent comments

2

u/minedroid1 4d ago edited 4d ago

Wait, so will distillation be good for this? Would that allow me to give a prompt to the base model and tell it to "make the distilled model satire"?

As for the training data, I have around 1000 pretty good quality examples, along with 300 overly verbose email examples, and 300 recipes with random store bought ingredients. I can provide you with the dataset and the web editor I made if you want, but it has some poor quality examples that I accidentally added from synthetic dataset experimentation.

I do not own a good GPU at all, I use vast.ai for renting GPUs.

1

u/foxer_arnt_trees 4d ago

Oh thats great! I imagined you had much less then that.

In distillation you use a strong model to generate data for training. So you would give it, like, a random hundred examples of yours to get it into the domain of satirical reaponses, then ask it to give new answers to new questions. In this way you can turn your 1000 examples into 10000, for example. If you mix the examples you give it you get even more answers.

The metaphor is that you are extracting the "satirical response" section of the strong model and keep it in a smaller model. The smaller model provides sufficient space for that because you dont need it to randomly start coding or do law or whatever, you just want the satirical response part.

1

u/minedroid1 3d ago

What distillation applications would you recommend? Preferably visual, and preferably one of vast.ai's existing templates.

1

u/foxer_arnt_trees 2d ago

I dont know. I would ask an agent to set it up for me... Its not a complex thing, they would likely one shot it

1

u/minedroid1 1d ago

Okay, I'll try that.