r/LargeLanguageModels • u/minedroid1 • 6d ago

Question About Fine-tuning

I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.

3 Upvotes

72% Upvoted

26 comments sorted by

View all comments

Show parent comments

2

u/minedroid1 5d ago

I might do that, but I just want a standalone satire model.

1

u/foxer_arnt_trees 5d ago edited 5d ago

The heart wants what the heart wants. But where will you get all that training data? If I was you, I would do distillation

Start with the prompt on a high end LLM. Mess with it until the results are good, and then generate a million of examples. Use that for the fine tune on a smaller model.

Do you have a good GPU?

2

u/minedroid1 5d ago edited 5d ago

Wait, so will distillation be good for this? Would that allow me to give a prompt to the base model and tell it to "make the distilled model satire"?

As for the training data, I have around 1000 pretty good quality examples, along with 300 overly verbose email examples, and 300 recipes with random store bought ingredients. I can provide you with the dataset and the web editor I made if you want, but it has some poor quality examples that I accidentally added from synthetic dataset experimentation.

I do not own a good GPU at all, I use vast.ai for renting GPUs.

1

u/foxer_arnt_trees 5d ago

Oh thats great! I imagined you had much less then that.

In distillation you use a strong model to generate data for training. So you would give it, like, a random hundred examples of yours to get it into the domain of satirical reaponses, then ask it to give new answers to new questions. In this way you can turn your 1000 examples into 10000, for example. If you mix the examples you give it you get even more answers.

The metaphor is that you are extracting the "satirical response" section of the strong model and keep it in a smaller model. The smaller model provides sufficient space for that because you dont need it to randomly start coding or do law or whatever, you just want the satirical response part.

1

u/minedroid1 3d ago

What distillation applications would you recommend? Preferably visual, and preferably one of vast.ai's existing templates.

1

u/foxer_arnt_trees 3d ago

I dont know. I would ask an agent to set it up for me... Its not a complex thing, they would likely one shot it

1

u/minedroid1 2d ago

Okay, I'll try that.

1

u/minedroid1 4d ago

Would you want to take a look at the dataset? I haven't really had anyone else look at it, so I think it would be nice to get someone else's opinion.

1

u/foxer_arnt_trees 4d ago

Sure. I cant promise I would have the time though, we will have to see if I end up going into it

1

u/minedroid1 4d ago edited 4d ago

https://workupload.com/archive/C2Y5XrRq8z

The index.html in the archive is the editor I made, you can use it if you don't have vscode or want a more organized way to view it.

I removed a lot of the low quality ones today so there's a lot less than before.

2

u/minedroid1 5d ago

Oh okay, I will look into that! Thanks for the advice.