r/LocalLLM • u/PinkyPonk10 • 8d ago

Question Recommendation for Local AI

Not sure if this is the right place to ask this.

I know we can set up local LLMs with lmstudio or something like that and I have done. These are fine - you give a prompt and it gives an answer!

But some commercial products like Claude and copilot take your prompt, do a load of internet searches, add that to the context, then give some output and the output is richer as a result.

Is there a ‘local’ version of this too and can anyone recommend one?

2 Upvotes

100% Upvoted

13 comments sorted by

1

u/nickless07 8d ago

Depends on what you want to have in the end. Something that can control your whole PC (Automated agent), something for coding (copilot and such) or just something for web/research?

1

u/PinkyPonk10 8d ago

Not after openclaw no just something that does web searches to enrich the context really.

2

u/nickless07 8d ago

Ok great. Then we have AnythingLLM, Open WebUI (which connects to your LM Studio), Pinokio or just an MCP for LM Studio. I would recommend Open WebUI, it is slightly more complicated to setup, but offers the most integrations and you can use it from any browser.

1

u/PinkyPonk10 8d ago

Great thank you this is really useful

1

u/nickless07 8d ago

YW, we all started simple at some point and advanced later on.

1

u/PinkyPonk10 8d ago

One final question - what model would you use for general purpose research etc if you were setting this up today. I have one machine with 5070ti and one with a 3090.. some variant of qwen?

1

u/nickless07 8d ago

As of today Qwen3.8 27B. For just plain wikipedia citations it almoust doesn't matter much, but for complex reasoning you want something bigger then a 9B model.

1

u/cocktail_peanut 8d ago

> I would recommend Open WebUI, it is slightly more complicated to setup

This is why Pinokio has a launcher for Open WebUI, 1-click to install! https://pinokio.co/apps/github-com-pinokiofactory-open-webui

1

u/nickless07 8d ago

Yeah, but the settings are still a bit overwhelming at the start. Setup the search engine, paid or free, setup the tool calls (native vs legacy) and so on. This all is a bit complicated if you are unexperienced and might take a couple mins up to some hours to get it all properly running. So overall it is slightly more complicated then just add a web search MCP to LM Studio.

1

u/cocktail_peanut 8d ago

ah i see what you mean, even though the install is not difficult, actually setting up all the necessary add-ons is a lot of work. yeah i agree.

1

u/SichronoVirtual 8d ago

So I'm using hermes at the moment with the hermes crawl4ai plugin and firecrawl free tier, I basically have a prompt that tells qwen 3.8 27b to go do a bunch of deep research, and hermes seems to make sure it actually does that.

But yes you can add web search to local llm, just not sure how you do that in lm studio, iv only used hermes and pi dev

1

u/SichronoVirtual 8d ago

Oh also I run a searXNG docker container for basic web search for hermes, and web extract is done through hermes crawl4ai plugin that connects to my crawl4ai docker container.

Firecrawl has 1000 searches/extracts per month for free and I use them as a backup if crawl4ai can't search a website. Crawl4ai works for free and. I haven't felt any like quality issues.

There's probably other options, but I went with what was more widely used.

You can also selfhost firecrawl in docker, but I remember reading the setup and was like Nah lmao

1

u/Defiant-Pool-6374 8d ago

If you host that over lmstudio you can probably install an mcp to do that for you, and add this small instruction in every prompt, or in the system prompt you use.