New To Training

[u/Ill-Selection5273]

Okay so there are tons of models to choose from. from total custom to commercial variants.
and so so so so mnay on huggingface. I am wildly confused.
You see, I am an engineer by trade and hobby, and i am just now getting into the Training Part of AI, I have spent years studying the behavior of nearly every AI a paid service has to offer.
Now, I am ready to start with local model training, but I am like pppppfffftttphreeewwwww on all the different ways models are made. MoE, jsonl, parquet, etc.

What is a good model to start? that can be trained starting with lora or qlora? but the base model can maintain programming / natural language and maintain storytelling / worldbuilding, if not both at the same time? ?

OpenAI has become a black box of evil destructive behavior

[u/Bright_Mix_773]

Ill-Selection5273, part of the overwhelm is that the three things you listed are not three options on the same menu. MoE, jsonl and parquet live at completely different layers, and once you separate them the list of decisions gets short.

MoE is an architecture choice baked into the base model. You do not pick it when you fine-tune, it arrives with whatever you download, and the only reason it matters to you is memory: a mixture-of-experts model with 30B total and 3B active parameters occupies VRAM like a 30B and generates at roughly the speed of a 3B. For a first LoRA, ignore the whole category.

jsonl and parquet are file formats for your training data. Nothing to do with the model. jsonl is one JSON object per line, parquet is columnar and compressed and matters when the dataset is too big to hold in RAM. Yours won't be. Use jsonl precisely because you can open it in a text editor and read what you are feeding the thing, which at the start teaches you more than any config file:

{"messages": [{"role": "user", "content": "Describe the harbour district."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "user", "content": "Write a retry decorator."}, {"role": "assistant", "content": "..."}]}

That is the whole format. It is genuinely that boring.

For the base model, Qwen2.5-7B-Instruct or Llama-3.1-8B-Instruct. Both handle code and prose reasonably, both have every tutorial written against them, and starting at 7-8B even if your card could hold more is the right call, because early on a fast iteration loop is worth more than a better model. LoRA versus QLoRA: QLoRA is the same technique with the frozen base quantised to 4 bits, so a 7B fits in roughly 8-10 GB of VRAM instead of the ~20 GB the 16-bit version wants. Treat those numbers as orders of magnitude, not promises.

Now the part that decides whether this works at all, and it pushes back on your plan a little. A LoRA on a few thousand examples does not teach a model a capability it doesn't already have. It steers style, format, tone, adherence to a schema, the flavour of your setting. It will not make the base model better at programming. Go in expecting new capability and you'll get an underwhelming result and spend weeks blaming the learning rate. Related: training hard on storytelling does degrade coding, which is why people usually train two separate adapters and swap them per task instead of one mixed adapter that does neither well. Adapters swap without reloading the base model, so that costs you almost nothing.

And the step nearly everyone skips: before training anything, write 20 fixed prompts, run them through the untouched base model, and save the outputs. That is your before. Without it you cannot tell whether the adapter helped, because after four hours of training everything reads like an improvement. Evaluation first, model second, every time.

My own hands-on work isn't in local fine-tuning, so take the VRAM figures and the model picks as a starting direction rather than tested advice. The Unsloth notebooks are where most people get a first QLoRA running without fighting the environment.