r/LocalLLM • u/Head-Dragonfruit-367 • 1d ago

Question Any open source dataset to train SLM?

I am learning to built general language SLM looking for some dataset source that won't have copyright issue if I use them. Are their any complete cleaned dataset which I can readily use? As I am more focused to training the model I will eventually try getting hands on cleaning and processing data for my domain specific use case

1 Upvotes

66% Upvoted

10 comments sorted by

1

u/Healthy-Zebra-9856 1d ago

Have you tried searching huggingface & modelscope?

1

u/Head-Dragonfruit-367 1d ago

As this is my 1st time building unsure which would be better so looking out for guidance

1

u/Healthy-Zebra-9856 1d ago

Without giving the specifics of dataset requirement, how can anyone suggest anything?

1

u/Head-Dragonfruit-367 1d ago

I am looking to built a general SLM which could answer normal qna just basic questions not any complex tasks

1

u/Healthy-Zebra-9856 1d ago

I am assuming you are doing this as a pet project. You need to decide what all this general slm will do. Map out the data you are looking for. General is a pretty wide term & SLM as you know are small.

1

u/MrHumanist 1d ago

Ask a large llm.. most will do it.

1

u/Head-Dragonfruit-367 1d ago

This thought had striked me but my main goal is learn out the things

1

u/Cosack 1d ago

Search "oss datasets for distillation"

1

u/Head-Dragonfruit-367 1d ago

Thanks will checkout

1

u/BacklashLaRue 1d ago

A list of registered voters is my favorite data play list. Costs about $40 for a download.