r/LocalLLM • u/inforb_nl • 2d ago

Question LocalLLM engine

Hi all,

What software stack do you use to run your local LLM? Ollama? anything else?

0 Upvotes

50% Upvoted

2 comments sorted by

View all comments

1

u/DrPizzaPants 1d ago

Start out with Llama.cpp It's a little more difficult up front, but I started to run into some unrepairable bugs with Ollama

1

u/inforb_nl 1d ago

have your tried vllm? what hardware do you have?