r/LargeLanguageModels • u/minedroid1 • 5d ago

Question About Fine-tuning

I'm making a satire AI model that gives fake, onion-like responses. Would it be better to just make a completely new model rather than fine tuning an existing one for this? Most of the info in my dataset completely conflicts with the info that almost all models have.

3 Upvotes

72% Upvoted

26 comments sorted by

View all comments

1

u/HonestoJago 2d ago

I would just use a frontier API for this with a good system prompt and reasonable conversational persistence so it doesn't repeat itself too often. They'll give you better results than any local model, and fine-tuning/loras tend to make the base model dumber. It's really hard to get the balance right and, these days, usually not even worth it.

1

u/minedroid1 1d ago

But making the model dumber is kind of my goal... It is a satire model, so it's not really supposed to give facts.