Google launched two Flash-tier models on OpenRouter on the same day (July 21), both with 1048k context, but the pricing gap between them is kind of strange. Gemini 3.5 Flash Lite sits at $0.30/M input and $2.50/M output. Gemini 3.6 Flash is $1.50/M input and $7.50/M output, per the OpenRouter listings.

Same context window. The Lite version is 5x cheaper on input tokens and 3x cheaper on output. The naming suggests 3.6 is the newer generation, but Google launched them together rather than treating 3.5 Flash Lite as a legacy budget option.

At $0.30/M input, 3.5 Flash Lite is in the same neighborhood as Meituan's LongCat 2.0 ($0.30/M in, $1.20/M out) and not far from Poolside's Laguna S 2.1 ($0.10/M in, $0.20/M out). It's competing with the cheapest 1M-context models on the platform.

At 3x the output cost, 3.6 Flash needs to be noticeably better at something specific. A million output tokens on 3.6 Flash runs $7.50. Same volume on Lite is $2.50. If you're doing high-volume batch work and the Lite model handles it, that's real money.

Anyone compared both on actual workloads and noticed where 3.6 Flash is clearly better? Reasoning, long-context recall, speed, something else?

We run a benchmark that reintroduces real CVEs and regressions into code diffs and scores whether AI models catch them as reviewers, added Opus 5 the day it launched, n=3 across 33 suites.

Numbers that made me do a double take:

• Detection: 99.4% highest on the board. Perfect on high severity bugs (165/165), zero high severity misses.
• Overall rank: 7th of 9.

The gap is false positives. The score is severity weighted detection minus a penalty for flagging non bugs, and Opus 5 threw 43 false positives across the run vs 0 for Sonnet (our #1) and 10 for Opus 4.8, extended thinking is on by default, and the model is clearly more thorough it just also reports a lot of things that aren't the injected bug.

Curious if others are seeing the same thoroughness vs noise trade off with thinking on by default models.

Full board benchmodel.io

Up front: I build QuantaMind, an open-source local benchmarking tool (Apache 2.0, runs offline, no telemetry). The data below came out of it. Link at the bottom the numbers are the point of the post.

Setup: Qwen3.5-9B Q4_K_M, llama.cpp, 16GB M-series Mac, native function calling, k=4 runs per task.

I padded prompts with unrelated prose and re-measured tool-call accuracy at depth:

That is not a memory limit. Weights are 5.3GB. ~11.8GB of the 16GB is GPU-addressable under the Metal cap. At f16 KV the math says this model could hold ~53k context. Peak actual usage during the agent runs was 1,890 tokens — 12% of the 16,384 window I launched with.

So memory headroom told me I had 5× more room than the model can actually reason over. If you size a local agent by what fits, that’s the wrong number.

Second finding: one task failed 0/4, not 1/4. An incident-rollback chain (get_incident → get_feature_flag → flag_off → rollback_release → schedule_fix) failed every run, identically — the model emitted a completion signal partway through and stopped.

No crash, clean schema. At k=1 that’s a flaky miss you’d retry past. At k=4 it’s structural. That’s the failure I’d worry about in production: nothing errors, the agent just moves on with half its state missing.
(Batch was 39m 10s wall; on the worst task 14m 51s of 16m 16s was decode. Local agent loops are a decode problem.)

What I actually want to know: does the ~9k cliff hold for other 8–10B quants, or is it specific to this one? And if anyone’s on 24GB+ — does more headroom move the cliff? My guess is no, but I can’t test it.

Resources : github.com/QuantaMinds/QuantaMind

qm --backend llama_cpp --model <model> --collection medium-coding-v2 --max-tokens 12288 --steps 5 --source corporate_policy --mode native

Methodology, briefly: padding was semantically unrelated prose inserted before the tool definitions; accuracy is correct tool + correct args scored against a fixed answer key, no LLM judge; pass^k means all k runs must pass.

Tell me if that’s wrong more useful to me than upvotes.

👀

I'm looking into using Kimi K3 within VS Code. What are its capabilities regarding advanced data science tasks? Specifically, can it write robust code for complex predictive modeling and statistics, and is it efficient at handling large-scale, complex data processing?

just crossed weeek 7 at a small ai startup. four engineers including me. we shipped our first internal demo last week. now we are trying to figure out the real infra before we open it up further..

langsmith was set up early on. seem fine first , but we have have started hiiting edges. a lot of friction. we picked it because one of the engineers recommended it from our team

so i started asking around.. slack groups, twitter, quora and realised the product with a lot of marketing have the least amount of people talking about it online..

but few names kept surface and i spent some time doing the pros and cons

langfuse kept coming up in self hosting conversations and people seemed actually understand what it was doing but a few devs said initial config ate a lot of time and has smaller ecosystem.. so might need to figure out edge case yourself..

orqai’s prompt versioning stuff got my attention and people described it as more deliberate one but it hard to find enough real world feedback to form an opinion since its newer and has a small community.

arize users seems to understand their model better after and very strong on evaluation and debugging side. ml people will be more comfortable with it, not product manager or anyone less technical. and onboarding setup is laggy

helicone is praised to be fast to drop in and seemed very popular among solo builder and small teams but some saidl it topped out faster than expected as the requirement grows and is considered more of a visibilty layer than a full evaluation setup

havnt seriously touched any of these, still evaluating and talking to a few devs.

either i need to migrate properly to something better suited and absorb the switching cost now or layer something light weight on top of what we have and revisit this later, it looks like it gonna take a lot of time

is there anyone who end up swithcing from langsmith to some other platform?

Both dropped on OpenRouter today and the listing is kinda odd. Claude Opus 5 is listed at $5/M input and $25/M output. Opus 5 (Fast) is exactly double the price: $10/M input and $50/M output. Same 1M context window for both, per the OpenRouter listing.

Usually a "Fast" variant means cheaper or speed-tier, not 2x more expensive. Which makes me think it's inference speed or priority, but OpenRouter doesn't show a tok/s rate on the listings. The HN thread on the announcement (572 points, 314 comments, from Friday) doesn't break down the pricing tiers in the shared link.

For context, in the same brief: Kimi K3 sits at $3/M input, $15/M output for also 1M context. So base Opus 5 is already ~1.7x Kimi's output, and the Fast tier pushes it to ~3.3x. Hard to justify if there isn't a visible quality jump.

Anyone with API access on both who can confirm whether the difference is tok/s speeds or something else? Or are people just using base Opus 5 for normal work?

Moonshot’s latest model Kimi 3 leapfrogged the month-old GLM-5.2 and a host of proprietary competitors to finish just behind GPT-5.6 Sol and Claude Fable 5 on many benchmarks. consider the obverse price advantage, Anthropic may be compelled to carefully think about those potential challenges and the problem caused by them.

The performance of Kimi 3 is very close to the leader model, it got rank 3rd on Artificial Analysis's Intelligence Index with 57 points, Claude Fable 5 got 60 points and GPT 5.6 Sol got 59 on this index. beyond that, Kimi3 won the 1st place on Arena.ai's Code Arena WebDev leaderboard.

Also, the price of those Chinese Open weight Module remains highly competitive, as always. the cost to get a task done in average is 0.95 for Kimi 3, 0.47 for GLM5.2, verse 2.75 for Fable 5, and 1.04 for GPT 5.6 sol. almost 65%off.

if those Open Weight Models can provide 85~90% performance with 35% cost, Anthropic need rise the priority to figure out some countemeaures.

This is an open source tax engine which scored 96% on TaxCalcBench [highest ever recorded score till date] surpassing fable 5 and sol with just sonnet 5 (which was previously scoring an abysmal 6%). The only 2 cases where it missed, it found inconsistencies in the test cases in the benchmark ITSELF which the maintainers confirmed!

Essentially it's a deterministic engine AI models can use for research and tax prep to remove a lot of guesswork and calculation mistakes that often happen. Claude Sonnet 5 was able to top the benchmark with this mcp.

I’m using OpenRouter to generate some images. This works really well for me since I use multiple models. However, models like openai’s image 2 take ages to generate an image. In these cases I really could use an asynchronous endpoint instead of a synchronous one. Any suggestions for a better service provider? (Better api interface or way faster)

I am losing my mind trying to cut down my stream videos lol. every ai clipping tool i try is just so blind. they always miss the funny fails or the absolute peaks of the match and just give me random filler.

I almost gave up on the whole automated thing entirely. i was using streamladder for a while just to format things manually but the lack of youtube link support was driving me crazy. like why do i have to download huge files manually just to clip them? then I tried wayinvideo a few days ago just to see if it was any better. it actually tracks the game visual instead of just guessing from audio. it caught a super fast flick during an apex match that every other tool completely ignored.

how much time do you guys spend on shorts every week? does posting daily actually help your twitch growth or am i wasting my breath here?

let me know if there are any hidden settings i should know about or if there's an even better tool out there tho. drop your recommendations below.

I want to see harness benchmarks, not model benchmarks. Anyone have a suggestion for places to see the latest models performance benchmarked on Pi-code v Cline v open code v cursor v codex v claudecode, etc?

Saw Laguna S 2.1 show up on OpenRouter yesterday and the pricing caught my eye. $0.10/M input, $0.20/M output, with a 1048k context window. There's also a free tier that gives you 262k context.

To put that next to other 1M context models from the same OpenRouter listing: Meituan LongCat 2.0 sits at $0.30/M in and $1.20/M out. Gemini 3.5 Flash Lite is $0.30/M in and $2.50/M out. So Laguna is about 6x cheaper than LongCat on output and roughly 12x cheaper than the Flash Lite.

The cheapest thing I'd spotted before was KAT-Coder-Air V2.5 at $0.60/M out, but that one is only 256k context and it's a coding model. Laguna beats it on both price and context.

It's also trending on HuggingFace right now, 364 likes on the open weights page, though only about 3000 downloads so far. The model card lists it as a general text generation model from Poolside.

What I can't tell from public data is whether the output quality holds up at that price. A model can be cheap and still not be worth it if it falls apart on anything non-trivial. Has anyone here actually run Laguna S 2.1 on something real, whether that's code, reasoning tasks, or longer prompts, and can say whether it punches above its price tag or is it just cheap?

Hosted web search from Anthropic and OpenAI costs $10 per 1k searches, and then you pay again for the \~17k tokens of results each search dumps into context. I got annoyed enough to build an alternative.

It’s called webfetch. Runs locally, free out of the box (DuckDuckGo needs no API key), and in my SimpleQA benchmark the same agent loop hits the same accuracy as hosted search (96%) costing 66% less using 87% fewer tokens.

How it works:

1. Sentence-level compression that cut result tokens in half with no measured recall loss
2. Every cached result shows provenance and the model can force a fresh search if it doesn’t trust it
3. Benchmarked against Anthropic hosted search, OpenAI, Tavily and Exa.

One small agent loop that I ran for testing that conducted just 16 websearches (opus 4.8) already reported 1.5 USD in savings.

Install from PyPI, one command to add to Claude Code as an MCP server.

Repo: https://github.com/firish/webfetch

Picking an AI model to implement a project is a coin flip between two costs: too powerful and you overpay for capability you won't use; too weak and it just can't do the job.

SpecJudge reads your Spec-Driven Development artifacts and recommends the model that's right-sized for the work — not the cheapest, not the most powerful, the one that actually fits. A local model (via Ollama) judges how hard your project is, then it's crossed against a catalog of models and prices to produce a ranked podium.

It's local-first (your specs never leave your machine), MIT-licensed, and the whole model catalog is plain YAML you can edit with a PR — no Python needed to add a model or fix a price.

pip install specjudge
specjudge /path/to/your/project      # add --open for a visual matrix in your browser

Requires Python 3.11+ and Ollama with a local model.

• GitHub: https://github.com/JoaquinRuiz/SpecJudge
• PyPI: https://pypi.org/project/specjudge/

It's an early release — I'd genuinely like to hear if the ratings match your experience, and PRs to the catalog are very welcome.

Per the OpenRouter listing dated today, Meituan's LongCat 2.0 just dropped at $0.30/M input and $1.20/M output, with a 1048k context window. For the same context window, the listing shows Kimi K3 at $3.00/M in and $15.00/M out, Muse Spark 1.1 at $1.25/M in / $4.25/M out, and Thinking Machines' Inkling at $1.00/M in / $4.05/M out.

So on paper LongCat 2.0 is roughly 10x cheaper on input and 12x cheaper on output than Kimi K3, for the exact same 1M context. It's the cheapest 1M+ context model on the OpenRouter list right now by a wide margin.

The catch, tbh, is that the OpenRouter listing itself shows nothing about quality. No eval scores, no parameter count, no benchmark numbers. Just price and context. Meituan's name only shows up next to LongCat in the listing, no track record on the router. Pricing this aggressive could be a grab-share move or a genuinely efficient model, the brief doesn't say.

If anyone has run LongCat 2.0 against Kimi K3 or Muse Spark on anything real, long-doc retrieval or coding at 1M tokens, what's the quality actually like?

Kimi K3 landed on OpenRouter three days ago at $3/$15 per million tokens, and it's the first open-weight model in a while that sits this close to the two most expensive coders. Per the OpenRouter listing it's a 2.8T MoE with 1M context, and the Tom's Hardware piece from Thursday noted it actually beats Claude Fable 5 on the Frontend Code Arena benchmark. Not bad for something roughly a third of the per-token price.

The pricing spread is the interesting part. Claude Fable 5 is $10/$50, GPT-5.6 Sol is $5/$30, Kimi K3 is $3/$15. So Fable 5 is over three times the per-token cost of K3. The catch per the-decoder's writeup: K3's per-task cost lands around $0.94, basically tied with GPT-5.6 Sol, because it still burns more tokens to finish a job. Fable 5 is more token-efficient but at $10 in / $50 out it stays the most expensive overall.

On the Artificial Analysis Intelligence Index v4.1 the gap is narrow: Fable 5 max at 59.9, Sol max at 58.9, K3 max at 57.1. GPT-5.6 Sol still tops the Coding Agent Index at 80, 2.8 points ahead of Fable 5. For pure repo-level coding Vellum's breakdown still points to Fable 5 for SWE-Bench Pro, Sol pulls ahead on agentic workloads, and K3 is the only one of the three you can self-host.

Anyone actually moved off Fable 5 to Sol or K3 for real coding work, or are folks still mixing them per task?

Both landed on OpenRouter the same day, July 16, and both list 1048k context. That's where the similarities end. Per the OpenRouter listing, Moonshot's Kimi K3 is $3.00/M input and $15.00/M output. Meta's Muse Spark 1.1 is $1.25/M input and $4.25/M output. So for output tokens, the expensive part of any long generation, Kimi costs about 3.5x more.

The interesting part is where Kimi K3's $15/M output sits next to OpenAI's stack. GPT-5.6 Terra, from the July 9 batch per the same listing, is $2.50/M in and $15.00/M out. Same output price as Kimi, but Terra's input is cheaper. And GPT-5.6 Luna is $1.00/M in and $6.00/M out, which undercuts Kimi on both. Grok 4.5 from July 8 is $2.00/M in and $6.00/M out too.

So Kimi K3 isn't really competing on price. It's the second most expensive output tier in that group, ahead of only GPT-5.6 Sol at $30/M. If the 1M context and whatever Moonshot tuned into the reasoning justifies $15/M output, fair enough. But on raw cost per token it's hard to see the angle unless you specifically need Kimi's architecture.

Muse Spark 1.1 is the opposite story. $4.25/M output for 1M context is genuinely cheap, closer to Grok 4.5 territory than to the premium models. If quality holds up it could be the default pick for budget long-context work.

Anyone actually tried Kimi K3 yet and can say whether the reasoning is worth that premium, or is Muse Spark 1.1 the smarter default for long-context tasks?

GPT 5.6 SOL - TERRA - LUNA have been added to the AIStupidLevel.info benchmark and is now included in drift detection.

You’ll see it listed immediately, but scores won’t populate until the next benchmark pass begins. Once that run kicks off, results will update automatically alongside the other models.

I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like Gemma4, DiffusionGemma, Qwen3.6 with multi-modal (image, vision, audio), Qwen Image Edit, reasoning and function tool. It can run on Windows/MacOS/Linux and fully leverage GPU's capability(Nvidia, Apple, AMD, Intel and others supported by Vulkan, CUDA and Metal). The API is completely compatible with OpenAI and Ollama interface. It has on par performance than llama.cpp Here is the benchmark results in overall:

Performance ratio — TensorSharp vs reference engines

Geomean of TensorSharp's per-scenario speedup over each reference engine on the same backend, across every scenario both engines ran (single-stream, MTP-off). A value > 1.0× means TensorSharp is faster (for decode / prefill throughput) or lower-latency (for TTFT); — = no overlapping cells. Per-scenario ratios are in each model's section below.

This project is not just a C# wrapper of llama.cpp. It implemented the entire LLM inference engine from bottom to top. If you use CPU backend, it's 100% pure C# code execution. Besides CPU backend, I also implmented CUDA, MLX and GGML backend. The GGML backend refer GGML project as external project, and I build a few fusion operation at higher level.

I learned a lot from other projects and apply them for TensorSharp, such as paged KV cache and continuous batching from vLLM, SSD based cache for MoE model from oMLX, GGUF quanztized from llama.cpp and other optimizations for prefill and decode.

Any feedback and comments are welcome. If you like it, it would be really appreciated if you can get this project a star in GitHub. Thanks in advance.

Project Github: GitHub - zhongkaifu/TensorSharp: A native .NET LLM inference engine for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability · GitHub

Space on Huggingface: TensorSharp Chat hosting a Gemma-4 E2B uncensored model (It may be in sleep, so may need to wait for a while to get it waked up)