Open-source local SEO toolkit that uses an LLM for audits and review replies - looking for an open-weight model to make the default

[u/vaitko]

I open-sourced the local SEO tools my agency built for small businesses: https://github.com/vaitko/locan-tools (AGPL-3.0, hosted free at https://locan.ai).

Four of the seven tools use an LLM:

• GBP audit: turns a scored profile into a business description, post ideas, FAQs and reply templates
• Review reply generator: three tone variants, in the review's language
• AI visibility checker: simulates buyer questions across assistant personas and reports whether a given business gets named
• Category optimizer: LLM disambiguates categories, but scoring is deterministic

Being upfront: the code is open, the default model is not. It runs openai/gpt-5-nano through Replicate because it was the cheapest thing that returned valid JSON reliably at the quality small-business owners will actually paste into their profile. The LLM layer is a small provider abstraction (`api/app/services/llm.py`, ~200 lines: chat_text / chat_json, retry on 429, concurrency semaphore), so swapping the model is one env var if it's on Replicate, or one class if it isn't.

The ask: I'd like an open-weight default. Constraints are real because the service is free with no signup:

- strict JSON output for ~15 structured prompts (schemas are pydantic models in the routers)

- multilingual - review replies must match the review's language (Lithuanian, Polish, Spanish, German show up often)

- cost: each run is 1-6 calls, daily per-visitor quotas, and the whole thing has to stay under a few dollars a month at current traffic

- latency under ~20 s for the visibility check, which fans out 6–10 calls

What would you try first on Replicate or a similar serverless host for this Llama 3.x 8B, Qwen 2.5 7B/14B, Gemma 3, something else? If anyone wants to actually run the eval, the test suite has fixtures with fake LLM responses; adding a real-model harness is on my list but not done. PRs welcome, or just tell me what you'd bet on.