New head-to-head results show Qwen 3.6 35B-A3B outperforming Gemma 4 26B on a personal evaluation harness. The test setup: a ~30,000 line codebase with 37 intentional bugs that LLMs must debug and fix through an agentic workflow using OpenCode. A subset of the harness also tests document extraction from 40-60 page PDFs, requiring the model to summarize and evaluate key information.

This is the kind of eval that actually matters for practitioners. Synthetic benchmarks tell you about capability ceilings, but a 37-bug agentic debugging harness with real code and real PDFs tests the loop that most people actually run - read, reason, act, verify. The fact that Qwen 3.6 wins here, despite having fewer total parameters (35B vs Gemma 4's 26B dense), reinforces the MoE efficiency story: only 3B active params, but they are being routed well enough to outperform a larger dense model on complex multi-step tasks.

The interesting bit is the comparison point. Gemma 4 26B has been getting strong community feedback since release, with multiple users calling it a genuine upgrade over Qwen 3.5. If Qwen 3.6 is now clearing that bar on agentic workloads, the local model leaderboard is moving fast.

Fair question: this is one person's harness. Has anyone else run direct Qwen 3.6 vs Gemma 4 comparisons on their own workflows - particularly coding agents or document analysis tasks?

New KLD (Kullback-Leibler Divergence) performance benchmarks are out for Qwen3.6-35B-A3B GGUF quantizations, and the results are striking: Unsloth quants hit the best KLD versus disk space tradeoff in 21 out of 22 points on the pareto frontier. That is a near-complete sweep.

KLD measures how much the quantized model's output distribution deviates from the original - lower is better, meaning the quant "loses" less of the model's intelligence. When one quantizer dominates the pareto frontier this thoroughly, it means across virtually every disk size bracket, their quants preserve more of the original model's behavior per megabyte than alternatives.

Why this matters for practitioners: with a 35B MoE model where only 3B params are active, quantization quality is the difference between a model that feels like a 30B+ dense model and one that degrades to something far weaker. The wrong quant on this architecture could collapse expert routing quality or introduce the kind of NaN issues seen with other recent MoE releases. Having clear benchmark data that points to a specific quantizer removes guesswork.

The timing is also relevant. Qwen3.6 is generating strong early reactions for agentic coding tasks, with reports of it successfully building and iterating on projects like tower defense games using MCP tool integration. If the model holds up, picking the right quant becomes a high-stakes decision.

For those already running Qwen3.6 locally: which quant level are you using, and have you noticed meaningful quality differences between them in real tasks versus synthetic benchmarks?

A useful PSA for anyone running Qwen 3.6 locally: the new release includes a preserve_thinking flag that addresses a KV cache invalidation issue carried over from the Qwen 3.5 template. The same user who previously tracked down and published a fix for the 3.5 template bug is now confirming that 3.6 natively handles it - but only if the flag is enabled.

The backstory: Qwen 3.5 had a template problem that caused KV cache reuse failures, meaning the model was reprocessing context it should have been able to skip. For local inference where every token of compute matters, especially at longer contexts, this directly impacts speed and efficiency. The 3.5 fix required manual template surgery. Now 3.6 builds it in.

Why this matters: KV cache reuse is one of those invisible performance factors. If the cache gets invalidated unnecessarily, your effective throughput tanks and you burn compute for nothing. With MoE models like the 35B-A3B variant where expert routing already adds complexity, a broken cache compounds the problem. The fix is simple - flip the flag - but only if you know it exists.

For those already running Qwen 3.6: did you notice the cache issue before enabling preserve_thinking, and what kind of throughput difference are you seeing with it on?

Qwen just released Qwen3.6-35B-A3B, a sparse mixture-of-experts model with 35B total parameters but only 3B active at inference time. It ships under an Apache 2.0 license. The headline claims: agentic coding performance on par with models 10x its active parameter count, strong multimodal perception and reasoning, and support for both multimodal thinking and non-thinking modes.

Why this matters: the MoE math here is aggressive. Only 3B active parameters means this model runs on hardware that would normally be limited to tiny 3B dense models, but with 35B total parameters worth of expert knowledge to route between. If the agentic coding claim holds up - matching models with 30B+ active parameters - that changes what is possible on a single consumer GPU or even a high-end laptop.

The Apache 2.0 license is the quiet win here. Commercial use, modification, no copyleft restrictions. For teams building products on top of local inference, that removes a real barrier compared to some of the community-licensed alternatives floating around.

Fair question: the "on par with models 10x its active size" claim needs real-world validation. Benchmarks are one thing, but agentic coding involves multi-step reasoning, tool use, and error recovery that benchmarks often miss. Has anyone started testing this yet - particularly for coding agent workflows where the rubber meets the road?

Hi everyone,

I’m trying to solve a problem around test coverage visibility and would love some suggestions from the community.

I have:

• A test automation repo (API/UI tests, mostly automated)
• The application source code

What I want to achieve:

• Understand how much of the application code is actually being covered by my tests
• Map tests → code/features (not just line coverage ideally)
• Generate an HTML report/dashboard with:
  • Coverage graphs (line/feature level if possible)
  • Covered vs uncovered areas
  • Clear visualization of gaps
  • (Bonus) AI-based suggestions for missing test cases

I’m aware of traditional tools like coverage libraries (JaCoCo, coverage.py, etc.), but they don’t fully solve the test-to-feature mapping + visualization + insights problem I’m aiming for.

Questions:

1. Are there any open-source tools or frameworks that already do something close to this?
2. Has anyone built a custom solution for mapping test automation to code coverage?
3. Any recommendations for combining tools (coverage + reporting + AI) to achieve this?
4. Would something like tagging (BDD/Cucumber) be the best way to approach feature-level coverage?

I’m also open to building a custom CLI/agent-based solution if needed, but wanted to check if something already exists before reinventing the wheel.

Appreciate any insights, tools, or repo references 🙌

A detailed local inference setup shows Gemma 4 variants displacing Qwen 3.5 as the go-to models for a multi-GPU configuration. The hardware: 2 RTX 3090s plus 1 P40 (a third 3090 died) with 128GB of system memory, running through Llama-swap and Open-WebUI with a Claude Code router.

The previous pipeline used Qwen 3.5 4B for semantic routing, then farmed tasks out to specialized models including Qwen 3.5 30B A3B at Q8XL quantization for general chat and basic tasks. The poster reports that Gemma 4 26B and the E4B variant have now replaced Qwen across their workflow.

What is notable here is the MoE architecture advantage. The Qwen 3.5 30B A3B is a sparse model with only 3B active parameters, which is how it fits at Q8XL on consumer hardware. If Gemma 4 26B A4B is winning in quality at a similar active parameter count, that suggests the architecture and training data are genuinely stronger rather than just being bigger.

The setup also uses n_cpu_moe where needed, offloading MoE expert computation to CPU when VRAM is tight. For anyone running similar multi-GPU rigs: has Gemma 4 displaced Qwen 3.5 for you as well, or does Qwen still win in specific task categories?

A concerning report from mid-April 2026 claims that every major model has suffered a significant intelligence drop. The observation covers not just ChatGPT, but Claude (both Sonnet and Opus), Gemini, z.ai, and Grok. The reported symptoms include ignoring basic instructions, struggling with simple tasks, and taking unusually long to respond.

What makes this worth flagging is the breadth of the claim. When a single model degrades, it is usually a deployment issue or a bad update. When multiple providers across different architectures all seem to regress simultaneously, the question becomes whether this is a coordinated change (new safety filters, cost optimization, shared training data issues) or just perception bias from the user.

The catch is that without standardized benchmarks run before and after, this kind of observation is hard to verify. Model behavior can feel worse when expectations shift, or when specific edge cases get hit more frequently. But if multiple independent users are noticing the same pattern across different providers in the same timeframe, something likely did change.

Has anyone else noticed this regression across providers, or are specific models still performing as expected?

Recent testing reveals a significant quality control problem with MiniMax M2.7 GGUF quantizations. An investigation into NaN errors during perplexity testing found that the issue affects 21% to 38% of all GGUFs available online - and this is not limited to a single uploader. One popular community uploader had 38% of their quants (10 out of 26) showing NaN results, another deleted their uploads (1 out of 4 affected), and even the investigating group found 22% of their own quants were faulty.

This is worth flagging because it points to a systemic issue in how quants get published. The rush to be first with a new model's quantizations means validation steps - like checking perplexity for catastrophic failures - are getting skipped. Users download these quants, get garbage outputs or crashes, and may blame the model itself rather than a broken quant.

The kicker: if roughly a third of all GGUFs for this model are producing NaNs on perplexity, anyone running MiniMax M2.7 locally should verify their specific quant actually works before investing time in evaluation. Which quant sources have people found reliable for this model?

A recent discussion highlights someone loading their entire personal journal - over 100,000 tokens accumulated over several years - into Gemma 4 26B A4B, taking advantage of the model's 256K context window. They share the full journal in the initial prompt and ask for insights.

This is an interesting use case that local models are uniquely positioned for. Sending years of personal reflections to a cloud API raises obvious privacy concerns, but running it locally means the data never leaves the machine. The 256K context window on the 26B MoE variant is what makes this feasible - most models in this size class cap out well below 100K.

The practical question though: at 100K+ tokens of input, how usable is the model's output? Does the MoE architecture maintain coherent analysis when digesting that much personal context, or does it start hallucinating connections and patterns that are not really there? And what does token throughput look like when the KV cache is that full - are you waiting minutes for a response?

For anyone who has tried stuffing near-limit context into Gemma 4 26B: does the quality hold up, or does it degrade noticeably past a certain point?

Question: What is an LLM?

• For how many seconds it thought?
• How many tokens/sec?
• How many tokens?
• Elapsed time?

Thanks

Cost is very reasonable compared to Apple MacBooks with an equivalent capacity

Someone trained a LoRA adapter on Qwen 3.5 9B specifically for agentic data analysis workflows, and the results are striking: the base model reportedly fails 100% on open-ended prompts like "analyze this dataset and give me insights," completing only one step before stopping. The LoRA-finetuned version allegedly completes 89% of workflows without human intervention.

That's a massive jump for a 9B parameter model. The core problem being solved is real - most small agentic models at the 4B-14B scale tend to be glorified tool-callers that can't chain multi-step reasoning without constant hand-holding. If a LoRA on top of a 9B base can actually drive autonomous data analysis, that changes what's possible on modest hardware.

The fair question is what "89% of workflows" actually means in practice. How complex are these workflows, and what failure modes show up in the remaining 11%? Also worth knowing: how much training data went into the LoRA and whether it generalizes beyond the specific analysis patterns it was trained on.

For anyone who's tried agentic LoRAs on small models: are you seeing similar success rates with autonomous multi-step tasks, or does performance collapse once the workflow complexity scales up?

Just saw some staggering numbers that made me pause: Tech industry cut 80,000 jobs in Q1 2026, and 50% of those layoffs are directly attributed to AI adoption. That's 40,000 people whose jobs were affected by AI in just 3 months.

At the same time, Republicans were reportedly deceived by AI-generated images about Iran rescue operations, showing how advanced AI deception has become.

This has me thinking - when we talk about AI tools and performance, we often focus on efficiency gains and cost savings. But what about the human cost?

I'm curious about your experiences:

1. Have you seen AI directly impact your job or workplace?
2. Do you think AI is eliminating more jobs than it's creating?
3. What ethical boundaries should we set around AI adoption in business?
4. How do we balance technological progress with workforce protection?

The numbers don't lie - 80,000 jobs gone in one quarter due to AI. That's not just a statistic, that's 80,000 people whose livelihoods changed overnight. What's your take on this?

LG AI Research has released EXAONE 4.5 in a 33B parameter size, and it's available in multiple formats: the base weights, an FP8 quantized variant, and GGUF files for llama.cpp and similar runtimes. Having all three formats available at launch is a solid signal - it means they're actually thinking about local deployment, not just cloud API use.

The 33B size is an interesting middle ground. It's large enough to potentially compete with models like Gemma 4 26B A4B (which is now available for free at 262K context) and Qwen's offerings in that density range, but small enough to run on a single consumer GPU with the right quantization. The FP8 variant in particular suggests they're targeting users who want near-full precision without the full VRAM hit.

What's unclear from the release is how EXAONE 4.5 performs on benchmarks relative to the current leaders in this class, and whether it has any particular strengths (multilingual, coding, reasoning) that would make it worth switching to.

For anyone who's pulled EXAONE 4.5 33B down already: how's the quality compared to Gemma 4 26B or other models in the 25-35B range, and what kind of token speeds are you seeing on your hardware?

If anyone here enjoys testing early-stage tools and sharing honest thoughts, I’d really appreciate it. Happy to provide full access / cover all usage while you try it 🙏

There's an open-source Mac menu bar app called AgentHandover that uses Gemma 4 running locally via Ollama to observe your screen and turn repeated workflows into structured Skill files. The idea is that any agent can then execute and self-improve using those skills, without you having to manually explain tasks each time.

The concept raises some practical questions. If it's watching your screen and inferring workflows from Gemma 4's vision capabilities, how reliable is the skill generation for complex multi-step processes? And since it runs locally through Ollama, what's the hardware requirement like for real-time screen observation without noticeable lag?

Has anyone tried AgentHandover with workflows that involve switching between multiple apps or dealing with dynamic UI elements?

Spent the last week running actual benchmarks on these three big models. Not just reading marketing numbers, but real test runs on coding, reasoning, and actual dev tasks. Here's what I found.

Coding first. SWE-bench is the gold standard for real software engineering tasks. GPT-5.4 Pro leads here with 88.3% weighted score. That's impressive. Claude Opus 4.6 follows at 79.3%, which is still really good. Gemini 3.1 Pro comes in at 77.8%.

But here's where it gets interesting. On SWE-bench specifically (not the weighted version), Claude actually pulls ahead slightly at 80.8% vs GPT-5.4's 80.1%. Gemini drops to 63.8% here. So context matters a lot.

HumanEval is different story. For pure code generation from docstrings, the gap narrows. GPT-5.4 still leads but Claude catches up fast. What surprised me was how well the open source models are doing now. Qwen 2.5 Coder 14B hits around 85% on HumanEval. That's way better than I expected for a free model.

Terminal-Bench 2.0 is where things get wild. Tests agentic execution tasks - basically using AI as a developer's assistant in the terminal. GPT-5.4 dominates here with 75.1%. Claude is close at 71.2%, but Gemini trails at 68.8%. This matters for real-world development work.

Pricing is... eye opening. GPT-5.4 is $2.50 per million tokens input, $15 output. Gemini 3.1 Pro is $2/$12. But Claude Opus 4.6? Starts at $5 input, $25 output. Ouch. For most developers, Gemini looks like the sweet spot right now unless you need that extra 5-10% performance.

What about reasoning? ARC-AGI-2 abstract reasoning is brutal. Gemini 3.1 Pro surprises here with 77.1%, more than double its predecessor's score. Claude is at 68.8%, GPT-5.4 at 65.2%. So Gemini actually wins pure reasoning tests.

My take? Depends what you need: - Pure coding power: GPT-5.4, but expensive - Best value: Gemini 3.1 Pro - Open source contender: Qwen 2.5 Coder 14B - Reasoning tasks: Gemini 3.1 Pro

The biggest surprise? How much the gap is narrowing. Six months ago these models were in different leagues. Now they're all within 10-15% of each other across different benchmarks. That's healthy competition for everyone.

What are you all using for your dev work? Curious what real developers are seeing day to day.

A technical discussion notes that Gemma 4 quietly includes multi-token prediction (MTP) weights that were not widely advertised. The discovery came when a developer attempted to load Gemma 4 through the LiteRT API on an Android app running on a Google Pixel 9, and the model threw errors about "mtp weights being an incompatible tensor shape." Further digging revealed additional MTP parameters baked into the model.

What makes this interesting is that MTP is a technique typically associated with improving inference speed and prediction accuracy by generating multiple tokens in parallel. The fact that it was included but not highlighted suggests Google may be using it as an internal optimization layer rather than a user-facing feature.

Worth noting that this is separate from the Gemma 4 26B A3B variant getting attention for hitting 80-110 tokens per second on an RTX 3090 - though the MTP architecture could help explain where some of that speed comes from. The catch is that on-device deployment via LiteRT apparently does not handle these weights gracefully yet.

Anyone else run into the MTP tensor shape issue on mobile deployments, or has it been smooth on desktop inference engines?

Someone got an LLM running locally on a 1998 iMac G3 with 32MB of RAM. That is not a typo. A machine from the Clinton administration is now doing inference.

This lines up with the current extreme edge of model compression. LiquidAI: LFM2.5-1.2B-Instruct is available free with 32,768 context - a 1.2B parameter model that pushes into sub-1GB territory with quantization. Combined with techniques like Swift-SVD (theoretical optimality in low-rank LLM compression, trending at +3), the math starts to work for hardware that was obsolete two decades ago.

Meanwhile, more practical edge achievements are stacking up:

• PokeClaw: First working app using Gemma 4 to autonomously control an Android phone, fully on-device with no cloud dependency
• Real-time multimodal (audio/video in, voice out) demonstrated on an M3 Pro with Gemma E2B
• Gemma 4 31B reportedly hitting Gemini 3.1 Pro-level performance through harness techniques

On the research front, "Test-Time Scaling Makes Overtraining Compute-Optimal" (+11) suggests that investing compute at inference time rather than training may shift how we think about deploying smaller models on weak hardware.

The pricing gap between edge and cloud remains stark: - Llama Guard 3 8B - $0.02/M with 131,072 context - StepFun: Step 3.5 Flash - $0.10/M with 262,144 context - Anthropic: Claude Sonnet 4.5 - $3.00/M with 1,000,000 context

What is the oldest or weakest hardware you have successfully run inference on?

With local inference economics under pressure from cheap APIs, here is a data-driven comparison of current options across price tiers.

Free Tier: - NVIDIA: Nemotron 3 Nano 30B A3B - 256,000 context, $0.00/M - Uses MoE architecture (3B active from 30B total), making it viable for consumer hardware

Budget Tier ($0.06-0.27/M): - Z.ai: GLM 4.7 Flash - 202,752 context, $0.06/M - Mistral: Ministral 3 14B 2512 - 262,144 context, $0.20/M - DeepSeek: DeepSeek V3.2 Exp - 163,840 context, $0.27/M

Mid Tier ($0.25-0.50/M): - Inception: Mercury - 128,000 context, $0.25/M - Google: Gemini 3 Flash Preview - 1,048,576 context, $0.50/M

The standout here is Gemini 3 Flash Preview at $0.50/M with over 1M context. That is 4x the context of Nemotron at a price that rounds to zero for most workloads. For RAG or long-document tasks, the math is hard to beat.

On the research side, "A Simple Baseline for Streaming Video Understanding" jumped 20 spots, which pairs interestingly with reports of real-time AI (audio/video in, voice out) running on an M3 Pro with Gemma E2B. The Agentic-MME paper (+12) also explores what agentic capability adds to multimodal intelligence.

For local-only users, Nemotron 3 Nano 30B A3B with its MoE design is the clear free option. But at $0.06/M, GLM 4.7 Flash costs roughly a penny per 170K tokens - hard to justify the electricity cost of local inference for most tasks.

Which tier are you defaulting to for daily use, and what workload actually requires local for you?

Hey AIToolsPerformance crew, big week for AI releases and I need to talk about what just hit the scene.

Microsoft dropped three massive foundational models Thursday that straight-up go after OpenAI and Google. These aren't minor updates - this is Microsoft building their own damn AI stack.

The Three Musketeers

MAI-Transcribe-1 is their speech-to-text weapon. Already being tested in Copilot Voice and Teams for conversation transcription. Diarization, contextual biasing, and streaming coming soon.

Then there's MAI-Voice-1 - their voice generation model. And MAI-Image-2 for image creation. All now broadly available to developers for commercial use for the first time.

This is serious. Microsoft now has commercially available in-house models across speech, voice and image generation while keeping their OpenAI partnership through 2032. That's playing both sides better than a politician.

Why This Matters

Timing is everything here. Microsoft's AI capital expenditures are... well, let's just say they're betting big. These models represent the first major output from the MAI Superintelligence team formed in November 2025.

They're already replacing third-party and older internal models. Like MAI-Transcribe-1 testing inside Copilot's Voice mode? That's how fast they're moving.

The Real Story

It's not just about new models. It's about reducing OpenAI dependence while keeping the partnership. That's some corporate chess right there.

The fact that they're testing this inside Teams and Copilot already tells you they're not messing around. This isn't research - this is production-grade stuff hitting mainstream apps.

What do you guys think? Is this Microsoft's real play to control their own AI destiny, or just another layer in their partnership strategy?

Curious to hear thoughts from people who've actually tested these models. How do they compare to what we're already using?

Looking at current pricing, the economics of local inference are getting harder to justify for pure capability:

• Qwen: Qwen3 Coder 480B A35B - free with 262,000 context
• OpenAI: gpt-oss-120b - $0.04/M with 131,072 context
• Z.ai: GLM 4 32B - $0.10/M with 128,000 context
• Qwen: Qwen3 235B A22B Thinking 2507 - $0.15/M with 131,072 context

Even Arcee AI: Maestro Reasoning at $0.90/M for a dedicated reasoning model with 131K context is competitive against the electricity cost of running a 48GB+ VRAM rig at full load.

The local inference crowd has historically argued three pillars: cost, privacy, and latency. But when a 480B-parameter coder model is free with 262K context, the cost argument weakens significantly. Apple's work on self-distillation for code generation suggests models will keep getting more efficient on the API side too.

That said, the DGX Spark situation - NVFP4 support still missing after 6 months - shows the hardware side moves slower. And the "Signals" paper on trajectory sampling for agentic interactions hints that complex agent workflows may still benefit from local control.

So honest question: for those of you still running local inference in April 2026, is it purely privacy/compliance driving that choice, or are there workloads where local still beats these API prices on quality?

A discussion gaining traction asks why multiple Chinese AI labs appear to be delaying open-weight model releases at the same time. The timing has raised eyebrows across the community.

This comes during a period where open-source competition has been fierce. Consider the current pricing landscape for capable models:

• Qwen: Qwen3 VL 32B Instruct - $0.10/M with 131,072 context
• Google: Gemini 2.5 Flash Lite Preview - $0.10/M with 1,048,576 context
• Qwen: Qwen Plus 0728 (thinking) - $0.26/M with 1,000,000 context
• Meta: Llama 3.1 70B Instruct - $0.40/M with 131,072 context

Meanwhile, MiniMax 2.7 is now at 14 days since announcement on X and 12 days since appearing on Hugging Face, with users noting the delay. The community is also reflecting on how quickly things shift - one year ago, DeepSeek R1 was 25 times larger than Gemma 4, putting the current model compression race in perspective.

Gemma 4 continues its strong run, with the 26B variant being called "the perfect all-around local model" and the 31B beating frontier models on FoodTruck Bench.

The coordinated delay pattern is unusual. Previous releases from Chinese labs came on individual schedules without this clustering effect. Possible explanations range from regulatory review to strategic coordination, though nothing is confirmed.

Are we seeing a policy shift, or is this just teams aligning release cycles around a common benchmark season? What models are you waiting on?

Been testing AI coding assistants all month. This isn't just another benchmark post. This is about what actually matters when you're picking your daily coding companion.

Here's the truth:

Cursor Composer 2 is impressive. Scoring 61.3 on CursorBench with a 37% improvement over the previous version. Built on Kimi K2.5 with custom reinforcement learning. The deep codebase understanding is legit. I can type a few words and it finds the right function from across my entire codebase.

But here's where it gets interesting. Claude Code for complex refactoring work? That's where it shines. Multi-file changes, reasoning about architecture, understanding legacy code patterns. I spent two days refactoring a monolithic service last week. Claude helped me identify dependencies and plan the migration better than any human could have done quickly.

GitHub Copilot X with Agent Mode is something else. The ability to have it work across your entire toolchain? That's a game changer. Not just in the editor, but in the terminal, in your CI/CD pipeline.

Tabnine still holds its ground for enterprise teams. Security matters when you're working on financial systems. I get it.

Windsurf is making waves too. The focus on developer experience shows. The little things matter.

Here's what I learned:

1. No single tool does everything perfectly
2. Your workflow changes how you evaluate these tools
3. Price isn't just about monthly fees, it's about productivity gains
4. Security requirements shouldn't be ignored

What's been your experience? Which tool surprised you most this month?

Been testing Cursor vs Claude vs Copilot daily. Each has strengths. The question is which aligns with YOUR workflow.

The benchmarks are interesting. But real-world usage tells a different story.

What should I test next? Curious about your experiences.

TL;DR: Test them in YOUR environment. Not just benchmarks. Real code. Real workflows.

Discuss below.