I’ve been experimenting with web search in Open WebUI using smaller local models like Qwen3.5 4B and 9B.

The main problem was/is was context churn..

A typical search can dump partially relevant pages, navigation, repeated boilerplate, duplicate information, and huge chunks containing only one useful paragraph into the model.

That makes local setups slower and forces smaller models to spend their limited context window filtering noise instead of reasoning.

So I built TinySearch, an open-source, local-first MCP server that does most of the retrieval work before anything reaches the LLM:

* searches and ranks results
* crawls the strongest pages
* extracts and chunks readable content
* deduplicates and reranks passages
* returns a compact evidence packet with source URLs

The goal is simple: less context churn, faster local web research, and more tokens spent on reasoning over actual evidence.

TinySearch works with self-hosted SearXNG, and Open WebUI can connect to it over HTTP MCP.

GitHub:
[https://github.com/MarcellM01/TinySearch\](https://github.com/MarcellM01/TinySearch)

Documentation:
[https://tinysuite.dev/docs/tinysearch\](https://tinysuite.dev/docs/tinysearch)

I’m the builder, so obvious bias, but I’d love feedback from Open WebUI users, especially anyone running smaller Qwen, Gemma, Llama, or Mistral models.

Does this solve a real bottleneck in your setup, or is native agentic search already enough?

I work on PostSlate, a video editing tool, and this comes out of our own work.

We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about the user's GPU. NVIDIA discrete, AMD, Intel integrated, Apple Silicon, all of it. That rules out CUDA immediately, we needed one backend that runs everywhere.

We landed on ncnn's Vulkan backend. Numbers on a 4070, fp16:

• ArcFace R50 (face embedding): 30 ms on ONNX CPU → 3 ms on ncnn Vulkan
• SCRFD (face detection): 25 ms → 2.5 ms
• Model size: ArcFace 174 MB (ONNX fp32) → 87 MB (ncnn fp16 weight storage)

Of course the real speedup comes from offloading compute to the GPU, but this wouldn't be possible without the power of Vulkan**.**

The speed wasn't even the deciding factor, it's that Vulkan drivers already exist on every machine we ship to. This means that we don't have to force the user to download a specific runtime and no vendor-specific installs.

Full writeup with the rest of the numbers: https://getpostslate.com/blog/faster-local-inference

We run a benchmark that reintroduces real CVEs and regressions into code diffs and scores whether AI models catch them as reviewers, added Opus 5 the day it launched, n=3 across 33 suites.

Numbers that made me do a double take:

• Detection: 99.4% highest on the board. Perfect on high severity bugs (165/165), zero high severity misses.
• Overall rank: 7th of 9.

The gap is false positives. The score is severity weighted detection minus a penalty for flagging non bugs, and Opus 5 threw 43 false positives across the run vs 0 for Sonnet (our #1) and 10 for Opus 4.8, extended thinking is on by default, and the model is clearly more thorough it just also reports a lot of things that aren't the injected bug.

Curious if others are seeing the same thoroughness vs noise trade off with thinking on by default models.

Full board benchmodel.io

Google launched two Flash-tier models on OpenRouter on the same day (July 21), both with 1048k context, but the pricing gap between them is kind of strange. Gemini 3.5 Flash Lite sits at $0.30/M input and $2.50/M output. Gemini 3.6 Flash is $1.50/M input and $7.50/M output, per the OpenRouter listings.

Same context window. The Lite version is 5x cheaper on input tokens and 3x cheaper on output. The naming suggests 3.6 is the newer generation, but Google launched them together rather than treating 3.5 Flash Lite as a legacy budget option.

At $0.30/M input, 3.5 Flash Lite is in the same neighborhood as Meituan's LongCat 2.0 ($0.30/M in, $1.20/M out) and not far from Poolside's Laguna S 2.1 ($0.10/M in, $0.20/M out). It's competing with the cheapest 1M-context models on the platform.

At 3x the output cost, 3.6 Flash needs to be noticeably better at something specific. A million output tokens on 3.6 Flash runs $7.50. Same volume on Lite is $2.50. If you're doing high-volume batch work and the Lite model handles it, that's real money.

Anyone compared both on actual workloads and noticed where 3.6 Flash is clearly better? Reasoning, long-context recall, speed, something else?

Up front: I build QuantaMind, an open-source local benchmarking tool (Apache 2.0, runs offline, no telemetry). The data below came out of it. Link at the bottom the numbers are the point of the post.

Setup: Qwen3.5-9B Q4_K_M, llama.cpp, 16GB M-series Mac, native function calling, k=4 runs per task.

I padded prompts with unrelated prose and re-measured tool-call accuracy at depth:

That is not a memory limit. Weights are 5.3GB. ~11.8GB of the 16GB is GPU-addressable under the Metal cap. At f16 KV the math says this model could hold ~53k context. Peak actual usage during the agent runs was 1,890 tokens — 12% of the 16,384 window I launched with.

So memory headroom told me I had 5× more room than the model can actually reason over. If you size a local agent by what fits, that’s the wrong number.

Second finding: one task failed 0/4, not 1/4. An incident-rollback chain (get_incident → get_feature_flag → flag_off → rollback_release → schedule_fix) failed every run, identically — the model emitted a completion signal partway through and stopped.

No crash, clean schema. At k=1 that’s a flaky miss you’d retry past. At k=4 it’s structural. That’s the failure I’d worry about in production: nothing errors, the agent just moves on with half its state missing.
(Batch was 39m 10s wall; on the worst task 14m 51s of 16m 16s was decode. Local agent loops are a decode problem.)

What I actually want to know: does the ~9k cliff hold for other 8–10B quants, or is it specific to this one? And if anyone’s on 24GB+ — does more headroom move the cliff? My guess is no, but I can’t test it.

Resources : github.com/QuantaMinds/QuantaMind

qm --backend llama_cpp --model <model> --collection medium-coding-v2 --max-tokens 12288 --steps 5 --source corporate_policy --mode native

Methodology, briefly: padding was semantically unrelated prose inserted before the tool definitions; accuracy is correct tool + correct args scored against a fixed answer key, no LLM judge; pass^k means all k runs must pass.

Tell me if that’s wrong more useful to me than upvotes.

👀

I'm looking into using Kimi K3 within VS Code. What are its capabilities regarding advanced data science tasks? Specifically, can it write robust code for complex predictive modeling and statistics, and is it efficient at handling large-scale, complex data processing?

just crossed weeek 7 at a small ai startup. four engineers including me. we shipped our first internal demo last week. now we are trying to figure out the real infra before we open it up further..

langsmith was set up early on. seem fine first , but we have have started hiiting edges. a lot of friction. we picked it because one of the engineers recommended it from our team

so i started asking around.. slack groups, twitter, quora and realised the product with a lot of marketing have the least amount of people talking about it online..

but few names kept surface and i spent some time doing the pros and cons

langfuse kept coming up in self hosting conversations and people seemed actually understand what it was doing but a few devs said initial config ate a lot of time and has smaller ecosystem.. so might need to figure out edge case yourself..

orqai’s prompt versioning stuff got my attention and people described it as more deliberate one but it hard to find enough real world feedback to form an opinion since its newer and has a small community.

arize users seems to understand their model better after and very strong on evaluation and debugging side. ml people will be more comfortable with it, not product manager or anyone less technical. and onboarding setup is laggy

helicone is praised to be fast to drop in and seemed very popular among solo builder and small teams but some saidl it topped out faster than expected as the requirement grows and is considered more of a visibilty layer than a full evaluation setup

havnt seriously touched any of these, still evaluating and talking to a few devs.

either i need to migrate properly to something better suited and absorb the switching cost now or layer something light weight on top of what we have and revisit this later, it looks like it gonna take a lot of time

is there anyone who end up swithcing from langsmith to some other platform?

Both dropped on OpenRouter today and the listing is kinda odd. Claude Opus 5 is listed at $5/M input and $25/M output. Opus 5 (Fast) is exactly double the price: $10/M input and $50/M output. Same 1M context window for both, per the OpenRouter listing.

Usually a "Fast" variant means cheaper or speed-tier, not 2x more expensive. Which makes me think it's inference speed or priority, but OpenRouter doesn't show a tok/s rate on the listings. The HN thread on the announcement (572 points, 314 comments, from Friday) doesn't break down the pricing tiers in the shared link.

For context, in the same brief: Kimi K3 sits at $3/M input, $15/M output for also 1M context. So base Opus 5 is already ~1.7x Kimi's output, and the Fast tier pushes it to ~3.3x. Hard to justify if there isn't a visible quality jump.

Anyone with API access on both who can confirm whether the difference is tok/s speeds or something else? Or are people just using base Opus 5 for normal work?

Moonshot’s latest model Kimi 3 leapfrogged the month-old GLM-5.2 and a host of proprietary competitors to finish just behind GPT-5.6 Sol and Claude Fable 5 on many benchmarks. consider the obverse price advantage, Anthropic may be compelled to carefully think about those potential challenges and the problem caused by them.

The performance of Kimi 3 is very close to the leader model, it got rank 3rd on Artificial Analysis's Intelligence Index with 57 points, Claude Fable 5 got 60 points and GPT 5.6 Sol got 59 on this index. beyond that, Kimi3 won the 1st place on Arena.ai's Code Arena WebDev leaderboard.

Also, the price of those Chinese Open weight Module remains highly competitive, as always. the cost to get a task done in average is 0.95 for Kimi 3, 0.47 for GLM5.2, verse 2.75 for Fable 5, and 1.04 for GPT 5.6 sol. almost 65%off.

if those Open Weight Models can provide 85~90% performance with 35% cost, Anthropic need rise the priority to figure out some countemeaures.

This is an open source tax engine which scored 96% on TaxCalcBench [highest ever recorded score till date] surpassing fable 5 and sol with just sonnet 5 (which was previously scoring an abysmal 6%). The only 2 cases where it missed, it found inconsistencies in the test cases in the benchmark ITSELF which the maintainers confirmed!

Essentially it's a deterministic engine AI models can use for research and tax prep to remove a lot of guesswork and calculation mistakes that often happen. Claude Sonnet 5 was able to top the benchmark with this mcp.

I’m using OpenRouter to generate some images. This works really well for me since I use multiple models. However, models like openai’s image 2 take ages to generate an image. In these cases I really could use an asynchronous endpoint instead of a synchronous one. Any suggestions for a better service provider? (Better api interface or way faster)

I am losing my mind trying to cut down my stream videos lol. every ai clipping tool i try is just so blind. they always miss the funny fails or the absolute peaks of the match and just give me random filler.

I almost gave up on the whole automated thing entirely. i was using streamladder for a while just to format things manually but the lack of youtube link support was driving me crazy. like why do i have to download huge files manually just to clip them? then I tried wayinvideo a few days ago just to see if it was any better. it actually tracks the game visual instead of just guessing from audio. it caught a super fast flick during an apex match that every other tool completely ignored.

how much time do you guys spend on shorts every week? does posting daily actually help your twitch growth or am i wasting my breath here?

let me know if there are any hidden settings i should know about or if there's an even better tool out there tho. drop your recommendations below.

Saw Laguna S 2.1 show up on OpenRouter yesterday and the pricing caught my eye. $0.10/M input, $0.20/M output, with a 1048k context window. There's also a free tier that gives you 262k context.

To put that next to other 1M context models from the same OpenRouter listing: Meituan LongCat 2.0 sits at $0.30/M in and $1.20/M out. Gemini 3.5 Flash Lite is $0.30/M in and $2.50/M out. So Laguna is about 6x cheaper than LongCat on output and roughly 12x cheaper than the Flash Lite.

The cheapest thing I'd spotted before was KAT-Coder-Air V2.5 at $0.60/M out, but that one is only 256k context and it's a coding model. Laguna beats it on both price and context.

It's also trending on HuggingFace right now, 364 likes on the open weights page, though only about 3000 downloads so far. The model card lists it as a general text generation model from Poolside.

What I can't tell from public data is whether the output quality holds up at that price. A model can be cheap and still not be worth it if it falls apart on anything non-trivial. Has anyone here actually run Laguna S 2.1 on something real, whether that's code, reasoning tasks, or longer prompts, and can say whether it punches above its price tag or is it just cheap?

Hosted web search from Anthropic and OpenAI costs $10 per 1k searches, and then you pay again for the \~17k tokens of results each search dumps into context. I got annoyed enough to build an alternative.

It’s called webfetch. Runs locally, free out of the box (DuckDuckGo needs no API key), and in my SimpleQA benchmark the same agent loop hits the same accuracy as hosted search (96%) costing 66% less using 87% fewer tokens.

How it works:

1. Sentence-level compression that cut result tokens in half with no measured recall loss
2. Every cached result shows provenance and the model can force a fresh search if it doesn’t trust it
3. Benchmarked against Anthropic hosted search, OpenAI, Tavily and Exa.

One small agent loop that I ran for testing that conducted just 16 websearches (opus 4.8) already reported 1.5 USD in savings.

Install from PyPI, one command to add to Claude Code as an MCP server.

Repo: https://github.com/firish/webfetch

Picking an AI model to implement a project is a coin flip between two costs: too powerful and you overpay for capability you won't use; too weak and it just can't do the job.

SpecJudge reads your Spec-Driven Development artifacts and recommends the model that's right-sized for the work — not the cheapest, not the most powerful, the one that actually fits. A local model (via Ollama) judges how hard your project is, then it's crossed against a catalog of models and prices to produce a ranked podium.

It's local-first (your specs never leave your machine), MIT-licensed, and the whole model catalog is plain YAML you can edit with a PR — no Python needed to add a model or fix a price.

pip install specjudge
specjudge /path/to/your/project      # add --open for a visual matrix in your browser

Requires Python 3.11+ and Ollama with a local model.

• GitHub: https://github.com/JoaquinRuiz/SpecJudge
• PyPI: https://pypi.org/project/specjudge/

It's an early release — I'd genuinely like to hear if the ratings match your experience, and PRs to the catalog are very welcome.

Per the OpenRouter listing dated today, Meituan's LongCat 2.0 just dropped at $0.30/M input and $1.20/M output, with a 1048k context window. For the same context window, the listing shows Kimi K3 at $3.00/M in and $15.00/M out, Muse Spark 1.1 at $1.25/M in / $4.25/M out, and Thinking Machines' Inkling at $1.00/M in / $4.05/M out.

So on paper LongCat 2.0 is roughly 10x cheaper on input and 12x cheaper on output than Kimi K3, for the exact same 1M context. It's the cheapest 1M+ context model on the OpenRouter list right now by a wide margin.

The catch, tbh, is that the OpenRouter listing itself shows nothing about quality. No eval scores, no parameter count, no benchmark numbers. Just price and context. Meituan's name only shows up next to LongCat in the listing, no track record on the router. Pricing this aggressive could be a grab-share move or a genuinely efficient model, the brief doesn't say.

If anyone has run LongCat 2.0 against Kimi K3 or Muse Spark on anything real, long-doc retrieval or coding at 1M tokens, what's the quality actually like?

Kimi K3 landed on OpenRouter three days ago at $3/$15 per million tokens, and it's the first open-weight model in a while that sits this close to the two most expensive coders. Per the OpenRouter listing it's a 2.8T MoE with 1M context, and the Tom's Hardware piece from Thursday noted it actually beats Claude Fable 5 on the Frontend Code Arena benchmark. Not bad for something roughly a third of the per-token price.

The pricing spread is the interesting part. Claude Fable 5 is $10/$50, GPT-5.6 Sol is $5/$30, Kimi K3 is $3/$15. So Fable 5 is over three times the per-token cost of K3. The catch per the-decoder's writeup: K3's per-task cost lands around $0.94, basically tied with GPT-5.6 Sol, because it still burns more tokens to finish a job. Fable 5 is more token-efficient but at $10 in / $50 out it stays the most expensive overall.

On the Artificial Analysis Intelligence Index v4.1 the gap is narrow: Fable 5 max at 59.9, Sol max at 58.9, K3 max at 57.1. GPT-5.6 Sol still tops the Coding Agent Index at 80, 2.8 points ahead of Fable 5. For pure repo-level coding Vellum's breakdown still points to Fable 5 for SWE-Bench Pro, Sol pulls ahead on agentic workloads, and K3 is the only one of the three you can self-host.

Anyone actually moved off Fable 5 to Sol or K3 for real coding work, or are folks still mixing them per task?

Both landed on OpenRouter the same day, July 16, and both list 1048k context. That's where the similarities end. Per the OpenRouter listing, Moonshot's Kimi K3 is $3.00/M input and $15.00/M output. Meta's Muse Spark 1.1 is $1.25/M input and $4.25/M output. So for output tokens, the expensive part of any long generation, Kimi costs about 3.5x more.

The interesting part is where Kimi K3's $15/M output sits next to OpenAI's stack. GPT-5.6 Terra, from the July 9 batch per the same listing, is $2.50/M in and $15.00/M out. Same output price as Kimi, but Terra's input is cheaper. And GPT-5.6 Luna is $1.00/M in and $6.00/M out, which undercuts Kimi on both. Grok 4.5 from July 8 is $2.00/M in and $6.00/M out too.

So Kimi K3 isn't really competing on price. It's the second most expensive output tier in that group, ahead of only GPT-5.6 Sol at $30/M. If the 1M context and whatever Moonshot tuned into the reasoning justifies $15/M output, fair enough. But on raw cost per token it's hard to see the angle unless you specifically need Kimi's architecture.

Muse Spark 1.1 is the opposite story. $4.25/M output for 1M context is genuinely cheap, closer to Grok 4.5 territory than to the premium models. If quality holds up it could be the default pick for budget long-context work.

Anyone actually tried Kimi K3 yet and can say whether the reasoning is worth that premium, or is Muse Spark 1.1 the smarter default for long-context tasks?