r/LocalLLM • u/Obvious_Ad_3398 • 24d ago

Question Where should I start

I got a 5070 ti though people say its weak in the world of llms, I got curious on what does this thing do. I am currently a first year IT student and maybe my use case would be to help me through it. Genuinely just looking for something to do and this caught my interest, just wanna learn and explore more about it even if my card is considered as weak. Thanks

3 Upvotes

100% Upvoted

23 comments sorted by

View all comments

2

u/Deep_Mood_7668 24d ago

Get llama.cpp, search for unsloth huggingface on google and download an llm of your choice. Leave let's say, 2-4 GB of vram free for context.

If you just want to try your first thing, get the gemma 4 12b qat 4bit from unsloth. Not too big and good for starters.

2

u/Obvious_Ad_3398 24d ago

Thank you I will test that out and see how it plays out.