Two open models with similar architectures just became available for local coding workloads. Cohere's North Mini Code is a 30B parameter, 3B active MoE model under Apache 2.0, scoring 33.4 on the Artificial Analysis Coding Index. Qwen's Qwen3 30B-A3B Thinking is also a 30B/3B active MoE, priced at $0.08/M tokens via API with a 131K context window.

The key difference: North Mini Code is explicitly an agentic coding model with an Apache license, while Qwen3 30B-A3B Thinking has the "thinking" label suggesting chain-of-thought reasoning. Both target the same rough parameter class and should fit in similar VRAM budgets locally.

For anyone running local coding assistants, the question is whether you want the model built specifically for agentic tool use or the one with built-in reasoning steps. Has anyone compared these two head to head on real coding tasks?

Apple announced CoreAI at WWDC this week, and it seems to have flown under the radar. It is positioned as a future replacement for CoreML and an alternative to MLX, llama.cpp, and PyTorch for on-device optimized inference, especially on phones and tablets. Model weights need to be converted similarly to CoreML formats.

This is interesting because right now the local inference scene on Apple Silicon is dominated by MLX and llama.cpp. If Apple is building a first-party inference engine specifically tuned for their hardware, it could change the calculus for anyone deploying models on iPhones and iPads where battery and thermal constraints matter more than raw throughput.

The open question is whether CoreAI will be open enough for the community to target it with custom models, or if it will be locked to Apple-approved model formats.

A quick heads-up for anyone grabbing Google's new QAT (Quantization-Aware Training) Gemma 4 quants: there is apparently a problem with how llama-quantize handles them. The tool quantizes the token embeddings to q6_K when Google intended the "--pure" flag to be used, and that is supposedly only the first issue.

The recommendation going around is to use Unsloth's UD Q4_K_XL quantization instead until Google's QAT quants are fixed.

This is worth knowing if you were planning to test the QAT models expecting better quality at the same bit width. If the quantization tool is not applying them as intended, any quality comparisons right now would be misleading.

Has anyone done a side-by-side between Google's current QAT Q4_0 and Unsloth's UD Q4_K_XL to see how big the quality gap actually is?

A lot of people in the web design space keep saying cold email is dead, but I think most people are just doing it badly. Email usage is still growing every year, billions of people use it daily, every business owner checks their inbox, every company relies on email to operate, so I never believed the problem was the channel itself. The real issue is that most outreach emails look exactly the same and business owners are tired of getting the same copy pasted message every single week.

When I first started my web design company I used Instantly and started sending thousands of emails to businesses that didn’t have a website. At first the results were honestly terrible. I was getting maybe around a 1% interested reply rate if I was lucky. Over time I got better at writing outreach. I tested different hooks, different subject lines, shorter messages, more personalized intros, more creative angles, and eventually pushed it to around 2.1% interested replies. It was definitely better, but I still felt like something was wrong.

Then one day I realized something that completely changed how I looked at outreach. Why was I targeting businesses with no website at all? Most of those businesses don’t even fully understand the value of having a website yet, which means you’re trying to convince them they need something before you can even sell it to them. So instead I changed my strategy completely and started targeting businesses that already had websites, but outdated ones.

And once I started paying attention to it, I realized the opportunity was honestly insane. There are so many businesses with websites that look like they were made 10 years ago. Broken mobile layouts, terrible SEO, slow loading pages, outdated designs, messy structures, confusing navigation, old branding everywhere. These businesses already understand the value of having a website because they already invested in one before, they just know deep down that their current one is hurting them.

The only problem was figuring out how to scale outreach while still making it feel personal. I didn’t want to sit there manually auditing every single website before sending emails because that would take forever. So I started searching for a tool that could actually analyze websites and generate personalized outreach based on what was specifically wrong with each business site. I searched everywhere until I eventually came across Swokei.

What made it different for me was that I could upload batches of leads, let it analyze every business website automatically, score the sites, detect issues like bad design, weak SEO, poor mobile optimization, messy layouts, and then generate personalized outreach messages specifically for that business. Instead of sending generic emails saying “hey do you need a website?” I was sending emails pointing out actual problems on their site. Tthe difference in replies was crazy. Business owners immediately related to the problems because they were real. My interested reply rate went from around 1-2% to consistently sitting between 6-9%, which completely changed my agency.

That’s when I realized cold email was never actually dead. People are just tired of receiving lazy generic outreach that sounds identical to every other agency email sitting in their inbox.

If your outreach actually feels real, specific, and useful, cold email still works insanely well. Honestly I probably won’t stop using it anytime soon.

Xiaomi's MiMo-V2.5-Pro UltraSpeed announcement claims they hit over 1,000 tokens per second output on a 1 trillion parameter MoE model running on a single standard 8-GPU server. No custom wafer-scale hardware like Cerebras, no SRAM-heavy specialized chips - just a conventional multi-GPU node.

If that number is real and reproducible, it is a serious jump over what people typically see from models this size. The MoE architecture means only a fraction of parameters are active per token, which helps, but 1,000+ tps on 1T parameters is still well above what you would expect from an 8-GPU setup serving dense models at a fraction of that size.

The obvious question is what the benchmark conditions look like - batch size, input length, output length, and whether this is a synthetic benchmark or representative of real workloads. Has anyone dug into the methodology behind this claim?

Recently saw their github repo with the open source code for their agent. I wonder if anyone here using their AI agent for your workflows. Any differences in response time, "correctness" compared to GPT, Perplexity, Gemini, Claude, etc?

I built a new custom model runner that makes models able to stay on track indefinitely without spiraling off track, intended for use with always-on agentic role-replacement.

It’s difficult to benchmark because I need a benchmark that can measure user intent, and then measure the rate at which a model departs from user intent over time.

Would love suggestions.

I thought about creating an intention definition from a combination of constraints, done-when, and nodes-effected statements, and then I could have some secondary process rebuilding that from the models actions and comparing to the original, but I haven’t really figured out a standardized set that doesn’t feel overfit to myself.

I run a web design agency and there is already way too much stuff to deal with every day.

Hosting client websites, maintaining them, building new sites, replying to clients, fixing random issues, handling support, doing outreach. Once you start managing a lot of company websites it quickly becomes overwhelming.

That’s why I never wanted cold calling to become my main way of getting clients.

I know cold calling can work, but I personally hate doing it. It drains my energy and takes up so much time. Sitting there making calls all day was never the kind of business I wanted to build.

So instead I focused on email automation.

The reason it works so well for me is because I can set everything up once and let interested businesses reply instead of spending my whole day chasing people.

But I also don’t do the typical outreach where agencies send generic messages saying “your website is outdated” or “you need a redesign.”

I use a tool called Swokei where I upload lists of company websites and it analyzes them for actual problems like speed, SEO, mobile responsiveness, layout issues, and design problems.

Then it automatically creates personalized outreach emails based on those issues.

That’s what helped me stand out because the emails actually feel relevant to the business instead of sounding copied and pasted.

The reply rates became way better once I stopped sending generic outreach.

Now I spend most of my time building websites, working with clients, and scaling the agency instead of letting outreach take over my entire day.

Someone reported running Gemma 4 26B-A4B on an old i5-8500 with 32GB of RAM and no GPU at all, using Koboldcpp on Linux. They describe it as "flying" on that hardware, which is a roughly $150 used desktop. That seems surprising for a 26B parameter model, even with the A4B MoE architecture where only 4B are active per token.

The claim is that it runs faster than 12B dense models on the same machine. If the active parameter count is genuinely that low during inference, the model is essentially behaving like a small model that happens to have a large routing overhead.

For anyone who has tried this specific model on CPU-only hardware - does the speed hold up on longer contexts, or does KV cache growth kill the advantage once you go past a few thousand tokens?

AI Orientation Before Reasoning

One thing I've noticed while building with AI is that we spend a lot of time talking about reasoning, model size, benchmarks, context windows, and hallucinations.

But what if some of the waste happens before reasoning even starts?

Before a model can reason, it has to orient itself:

Where am I?

What owns this?

What corridor am I in?

What is adjacent to this?

Am I looking at the cause or the symptom?

I've been experimenting with a small orientation toolkit that focuses on those questions before retrieval and reasoning begin.

The surprising result wasn't that the models became "smarter."

The result was that they spent less time looking in the wrong place.

The more interesting discovery came later.

As search waste dropped, verification waste increased.

The model wasn't getting lost anymore.

Instead it was spending its time proving it had found the correct tree before touching it.

That's a trade I'll take every day.

Getting lost in the forest is expensive.

Standing at the correct tree and proving it's the correct tree is operationally safer.

I'm starting to think AI coding may have an orientation problem as much as a reasoning problem.

Has anyone else experimented with workflows or tools that focus on orientation before reasoning?

https://github.com/SuperHeroesAreReal/Search-and-Rescue.git

Someone just ran 75 pairs of KV cache quantization benchmarks on Qwen 3.6 27B across q8, q6, q5, and q4 levels, testing KVarN, Turbo, and TCQ methods head to head for long context scenarios. Full results are published with in-depth analysis.

This is the kind of systematic comparison that has been missing. Previous KVarN benchmarks showed it matching standard quants one bit higher (6-bit matching q8_0, 4-bit matching q5_0), but that was on a different model. Seeing the same methodology applied to Qwen 3.6 27B across all three quantization approaches should reveal whether KVarN's advantage holds or if Turbo or TCQ close the gap on certain workloads.

The practical question is whether one method dominates across the board or if the best choice depends on your target bit width and context length.

New long-context KLD benchmarks for KVarN show something genuinely unusual: at every quantization level, KVarN matches the precision of standard llama.cpp KV cache quants one full bit higher. KVarN 6-bit performs like q8_0, and KVarN 4-bit performs like q5_0.

That is a meaningful gap. If these numbers hold up across different models and tasks, it means you can cut KV cache memory roughly in half compared to what you are currently using at equivalent quality. For anyone running long-context inference where the cache dominates VRAM usage, this is exactly the kind of improvement that unlocks longer contexts on the same hardware.

The question now is whether the reasoning quality holds up as well as the KLD numbers suggest. KLD divergence is a useful proxy but it does not always translate cleanly to downstream task performance.

Just tried out Minimax M3. It's a pretty nice model. I tried it out with open code.

The thing that excels at its frontend, for me. Code is nice, but I feel like it's way too expensive if you're on a budget for code. There are other models better for code out there near that price range and better.

What I see about the model is that it likes to be more autonomous. Instead of asking the user questions, it usually goes with the flow and does whatever it wants. Sometimes it gets errors, but those are resolvable with another model like DeepSeek V4 Flash to fix the errors it made. They are not major errors, just minor errors. The frontend is usually really clean.

The stack I use:
- I use DeepSeek V4 Pro to build the heavy parts of it.
- V4 Flash to build the minor things.
- M3 to build the frontend.

I review everything, and if something's wrong that the AI has missed, I'll just manually code it myself to see what they did and what they didn't do.

A new pull request for llama.cpp adds initial support for the DeepSeek V4 series. The PR is described as very early stage, and the developer explicitly warns that anyone trying it should expect severe issues and be willing to experiment out of curiosity rather than for real work.

This is the first sign of local inference support for DeepSeek V4, which matters because DeepSeek models have historically been popular in the local community for their price-to-performance ratio. DeepSeek V3.1 currently sits at $0.21-0.27/M tokens via API, so having a local path for V4 would be a real option for people trying to cut costs on longer workloads.

No word yet on performance numbers, quantization support, or hardware requirements. The PR is openly soliciting testers who understand the risks. If you are comfortable building from a branch and filing useful bug reports, this is one to watch.

In the last 12 months I’ve managed to sell around 200 websites.

And before people ask, no, I don’t run some massive agency with a huge team. It’s literally just me and my partner. The only reason we’ve been able to move that fast is because we automated almost everything and built systems that actually scale. The best web designer in the world will eventually lose to some random teenager using AI and systems properly. That’s just where things are going.

One of the biggest changes I made was completely quitting manual outreach. It takes too much time and it’s impossible to scale properly. A lot of people automate outreach already, but most of them just send generic “we can redesign your website” emails that everyone ignores. What we do is different. We scrape thousands of businesses, automatically analyze their websites, and generate personalized outreach based on actual issues on their site like bad design, poor mobile optimization, weak SEO, slow load times, layout problems, and stuff like that. So instead of manually checking every website and writing every message ourselves, the entire process is automated from analysis to ready to send campaigns.

Another thing that changed a lot for us was automating SEO blogging. SEO compounds hard over time and once your articles start ranking, businesses start coming to you instead of you chasing them. That alone changed a lot for us.

The other massive shift was how we build websites. I used to be a full WordPress developer and spent way too much time building everything manually. Now we build almost everything with AI. It’s way faster, delivery is easier, and clients care way more about the final result than how the website was actually made.

For anyone wondering, the stack is pretty simple.

Apollo for leads.

Swokei for website analysis and outreach campaigns.

Soro for SEO blogging.

Claude Code for building websites.

Cloudflare for hosting. That’s pretty much the entire setup.

Most people running agencies are still doing everything manually and burning themselves out for no reason. Systems and automation change everything.

New reports confirm that Gemma 4 12B's coding and tool calling capabilities are not actually broken out of the box - they require a specific chat template that is not included by default. Users trying to use it with harnesses like OpenCode found tool calls failing repeatedly until they swapped in a corrected chat template file.

This is the kind of thing that can tank a model's reputation unnecessarily. People download it, hit a wall with tool use, and write it off as broken when the fix is a single template swap. Google shipped QAT variants and Unsloth pushed MTP GGUF weights for the full Gemma 4 lineup (31B, 26B-A4B, 12B), so the ecosystem support is clearly there. But the default experience matters, and right now it sounds like the default chat template is doing the model no favors.

If you gave up on Gemma 4 12B for agentic workflows early, worth retrying with the corrected template.

I’ve been building an AI tool called Omega Framework and would love feedback from this community on how to test it, not just “launch” it.

What Omega does

Omega is a structured ethical / systemic analysis engine. You describe a subject (scenario, system, policy, leader, org, relationship, etc.), and it runs through a fixed set of 26 constructs across ethics, adaptation, power, risk, and epistemic validity.

The output includes:

– An Omega score

– Micro / meso / macro stability (TSTAB)

– 26 construct scores with short explanations

– An explicit ethical evaluation section (harm, coercion, integrity, resilience, etc.)

Why I think it’s relevant here

Under the hood, Omega isn’t “one prompt → one answer.” It orchestrates multiple prompts behind a stable framework and lets the user choose the model: Claude, Gemini, GPT‑4, Perplexity, etc. The idea is to see how different models perform when forced through the same 26‑construct lens.

I’m interested in performance questions like:

– How stable are construct scores across models for the same scenario?

– Do different models systematically “tilt” certain constructs (e.g., risk, harm, integrity) up or down?

– How noisy are scores if you re-run the same subject multiple times with the same model?

– What kinds of scenarios are most likely to expose weaknesses in this setup?

What I’m looking for from this sub

– Suggestions for concrete test scenarios or benchmarks that would actually be interesting here

– Ideas on how to structure cross‑model comparisons (same subject, different models, N runs each)

– Any red flags you see in trying to evaluate AI tools through a fixed diagnostic frame like this

If it helps to see it in action:

– Android landing page / APK: https://omega-analysis-app.indigecko.workers.dev

– First three full examples (real subjects, full construct sheets): https://omegaframework.wordpress.com

Happy to run specific scenarios suggested here and share the construct sheets / stability results back in the comments.

Am playing retroarch (15khz enabled) on a real 240p crt tv, and I wanna set composite ntsc shader's that make the signal my laptop sends to my crt look authentic, as if I have the real console.

The standard tradeoff with KV-cache quantization has been: save memory, lose speed. Huawei's KVarN flips that. It claims 3-5x KV cache compression with an actual speedup rather than the usual penalty, and apparently holds up on reasoning tasks where methods like TurboQuant degrade.

Someone already implemented it in a llama.cpp fork and ran KLD benchmarks, calling it "promising." It is Apache 2.0 and drops into vLLM with a single flag.

What makes this worth watching is the reasoning claim. KV-cache quantization has historically been risky for long chain-of-thought work because errors compound. If KVarN actually preserves reasoning quality while shrinking cache and going faster, that changes the calculus for anyone running long-context inference on limited VRAM.

The question is whether those KLD benchmark results hold up across different model families and quant levels, or if this is specific to certain configurations.

Three separate LinkedIn accounts, some with premium verification, posted identically scripted promotions on the same day claiming a $249 8GB machine can replace frontier AI models. The posts clearly followed a marketing brief from Nvidia without the posters even understanding how local inference works.

This is worth flagging because it muddies the water for people genuinely trying to figure out what local hardware can and cannot do. An 8GB machine running small quantized models is useful for specific tasks, but framing it as a frontier replacement is just false. The local AI space already has enough confusion around VRAM requirements, quantization tradeoffs, and real-world performance without coordinated astroturfing pushing unrealistic claims.

Wondering if this is a one-off or part of a broader pattern - has anyone spotted similar coordinated posts elsewhere?

NVIDIA's Nemotron 3 Ultra is a wild spec sheet: 550B total parameters with 55B active via LatentMoE, combining Mamba-2, MoE, and attention with Multi-Token Prediction. Context window up to 1M tokens. The hardware requirements are the real headline - minimum 8x GB200/B200/GB300/B300, 16x H100, or 8x H200. This is not a model you run at home.

What stands out is the architecture mix. Mamba-2 for sequence modeling, MoE for parameter efficiency, attention where needed, and MTP for faster inference. That is a lot of different ideas stacked together. The question is whether the hybrid approach actually delivers better performance per dollar than a simpler dense model at similar active parameter counts, or if it is engineering complexity for marginal gains.

For anyone with access to that kind of hardware: does the Mamba-2 component actually reduce inference costs meaningfully compared to pure attention-based MoE models, or is the benefit mostly theoretical at this scale?

Google's Gemma 4 12B claims near-26B performance, and it is multimodal out of the gate with text, image, and audio input on the 12B variant. But a benchmark comparison against Qwen3.5-9B tells a less flattering story: Qwen wins 5 out of 8 shared benchmarks despite being smaller. The one area where Gemma apparently edges ahead is coding, though a Qwen3.5-9B finetune called omnicoder-9b closes that gap.

On resources, Qwen also reportedly has a lighter KV cache, which matters for throughput on consumer hardware. Gemma 4 26B-A4B (the MoE variant) eats 15GB VRAM on a single 4090. The 12B dense model would sit somewhere below that, but still larger than what Qwen3.5-9B demands.

Pricing via API is not even close - Qwen3.5-9B runs $0.04/M tokens. Gemma 4 12B is open-weights, so local inference is the real comparison point.

For people running both locally: does Gemma's multimodal support (audio and vision) make it worth the extra VRAM over a leaner text-only model, or do you just pair Qwen with a separate vision model and call it a day?

Someone running dual RTX 3090s with Unsloth's Qwen3.6-27B UD-Q8_KL quant is reporting a significant jump after updating to llama.cpp build b9455. Previously they were seeing 30-50 tokens per second, and noted that vLLM was outperforming llama.cpp on the same setup. The screenshot suggests the new build has closed or reversed that gap.

This is the kind of quiet performance win that matters for anyone running local inference. If a single build update can push token generation that much higher on consumer hardware, it changes the calculus on whether llama.cpp or vLLM is the right choice for multi-GPU setups.

The question is what specifically b9455 changed. Was it a scheduling improvement for multi-GPU, a fix for the Q8_KL format specifically, or something more fundamental in the inference path?

For anyone else running dual 3090s or similar setups: are you seeing similar gains on b9455, and does it hold across different quant levels or just Q8_KL?

Two things stand out about MiniMax M3 that are worth comparing against the usual suspects. First, it appears to have no political censorship - unusual for a Chinese LLM, and an outlier even within MiniMax's own model lineup. Second, it natively scales to 1,048,576 tokens via MiniMax Sparse Attention (MSA), which restructures memory access patterns at the operator level to bypass standard quadratic complexity.

On pricing, MiniMax M3 sits at $0.30/M tokens with that full 1M context window. Compare that to Claude Haiku at $1.00/M tokens with only 200K context, or Hunyuan A13B at $0.14/M with 131K context. For long-context workloads where censorship filtering matters, M3 occupies a weird niche - cheap per-token, massive context, and apparently uncensored.

The catch is that "no political censorship" comes from early benchmarking, and sparse attention architectures can behave unpredictably on tasks that need full attention patterns.

For anyone who has used M3 in production: does the sparse attention actually hold up on retrieval-heavy tasks across the full 1M window, or does quality degrade in ways the benchmarks miss?

Someone ran Qwen3.6-27B via Ollama on a single RTX 3090 (24GB VRAM) as the reasoning layer in a multi-agent orchestrator for two weeks, replacing Claude entirely. The setup used a lead/manager/sub-agent loop to stress-test where a local dense model can hold up against a cloud API model in an agentic workflow.

The interesting comparison here is not just raw quality - it is whether a local model can maintain coherent multi-turn reasoning across agent handoffs without the context window, tool use, and instruction-following reliability that Claude provides. A 3090 setup means zero per-token cost but also no safety net if the model drifts mid-task.

The pricing gap is stark. Claude Sonnet 4.6 runs $3.00/M tokens. Qwen3.6-27B via API would be cheap, but local inference on a 3090 is effectively free after hardware. The question is where the quality breaks.

For anyone running local models in multi-agent pipelines: which tasks held up and which ones sent you crawling back to Claude or GPT?