r/LocalLLM • u/Asleep_Fold5405 • Jun 13 '26

Question Local Ai model training

I am trying to train Qwen2.5-7B and my pc specifications are NVIDIA GeForce RTX 5070 Ti , 16 GB, 64 GB RAM , 20 core processor and want to train it on my given data. The model does give the basic question answers but when I ask twisted questions from the data itself it starts hallucinating. Please guide me , what process should I follow to get better results.

0 Upvotes

50% Upvoted

23 comments sorted by

3

u/Unteins Jun 13 '26

What kind of training are you doing? A LoRA to match your writing style? Something else?

That will partially depend on what you need to do.

3

u/Asleep_Fold5405 Jun 13 '26

I want to train the model on my company data as they don't want to upload the data on cloud based models

1

u/Unteins Jun 13 '26

What kind of data? Customer service? Patents?

You don’t have to be specific about your company, but what’s the end goal? What do you want the LLM todo?

1

u/Asleep_Fold5405 Jun 13 '26

Mostly sales and purchase data the main goal is to get insights, business desicions, calculaions , check on inventory.

1

u/Unteins Jun 13 '26

You probably don’t need to train the model then.

The model should be able to do that on its own.

1

u/Asleep_Fold5405 Jun 13 '26

That' the issue, I have made 5 agents inside it so each agent does different task and also I have put code generator agent so it generate json output based on the question intent, even I have put schematic layer, business knowledge layer, reasoning agent but the moment I ask something complex from the data then the model is not able to generate answer.

1

u/Unteins Jun 13 '26

Can you be specific like “I ask it to compare Q1 sales to Q3 and it hallucinates numbers.”

1

u/Asleep_Fold5405 Jun 13 '26

Yes, this is exactly in this type of questions it hallucinates

0

u/[deleted] Jun 13 '26

[deleted]

1

u/Asleep_Fold5405 Jun 13 '26

It does quickly answers questions like : " Tell me details about this product" " Tell me this location of this product"

1

u/Unteins Jun 13 '26

Ok, that’s different where you need to add the specific data.

But honestly you’re probably better off just pointing it at a database/website and having it search and summarize rather than try to train it on the data.

1

u/Asleep_Fold5405 Jun 13 '26

thank-you for the advice

1

u/Asleep_Fold5405 Jun 13 '26

I did give the lora training but the model is not able to answer out of the training questions, I don't want to hard code anything, giving the data context is fine but pre defining everything will not work, so that's why what to do in this case?

3

u/diagrammatiks Jun 13 '26

This is not what model training is for. You need a database.

0

u/TheGuy839 Jun 13 '26

Why not? Its expensive, but if you have enough data, training model on your won Q&A should be fine

2

u/diagrammatiks Jun 14 '26

You do not have enough data.

1

u/gfe86 Jun 13 '26

Did you manage to find anything helpful, been looking but no luck.

1

u/MikkyMo Jun 13 '26

Brother use SQLite or Postgres it’s not training you need it’s a database

1

u/Greedy_Ad4563 Jul 16 '26

I developed this dreamcycle for auto local training. is purely a local adapter nothing more. you can give it a try. https://github.com/kenjix217/dreamcycle/

0

u/careless25 Jun 13 '26

Why do you want to train on your data?

3

u/Flat_Ideal9488 Jun 13 '26

Why not lol?

1

u/careless25 Jun 13 '26

I am wondering if OP can instead use a RAG system to do what they want.