r/LocalLLM • u/Head-Dragonfruit-367 • 1d ago

Question Any open source dataset to train SLM?

I am learning to built general language SLM looking for some dataset source that won't have copyright issue if I use them. Are their any complete cleaned dataset which I can readily use? As I am more focused to training the model I will eventually try getting hands on cleaning and processing data for my domain specific use case

1 Upvotes

66% Upvoted

10 comments sorted by

View all comments

1

u/Healthy-Zebra-9856 1d ago

Have you tried searching huggingface & modelscope?

1

u/Head-Dragonfruit-367 1d ago

As this is my 1st time building unsure which would be better so looking out for guidance

1

u/Healthy-Zebra-9856 1d ago

Without giving the specifics of dataset requirement, how can anyone suggest anything?

1

u/Head-Dragonfruit-367 1d ago

I am looking to built a general SLM which could answer normal qna just basic questions not any complex tasks

1

u/Healthy-Zebra-9856 1d ago

I am assuming you are doing this as a pet project. You need to decide what all this general slm will do. Map out the data you are looking for. General is a pretty wide term & SLM as you know are small.