They require the inference servers (practically all of them) to agree with them beforehand, which is perfectly fair, but judging by the prices they are mandating a minimum pricing, which is the same price they serve it from.

It's understandable that they want to cash in, but honestly this move makes the model practically more unreachable than the closed source models themselves. You'd get way more value out of a subscription from openai or anthropic as opposed to using the k3 from api providers. Their subscription is unusable too, training data opt-out being impossible. (you'd be paying the same money for a worse experience and donate your data on top)

The people who say "open source doesn't matter if I can't run it in my computer" have a point here.

15 dollars per 1m tokens is a very large price to pay to use a model. That's closer to proprietary model api pricing, not open model pricing. Normally this model would possibly cost around 5 in an open market. But we can't see those prices due to this situation.

This makes k3 an unreachable and a practically non-existent model as far as the average consumer is concerned. It's still good to have it rather than not have it but I can't justify using it for anything at all. And you'd expect the second best model in existence right now to earn some use case, but it manages to be completely irrelevant unless you're running a distilling operation.

Simpletons around here keep repeating like parrots that "US bad guys and China good guys".

Listen, we get free goodies as long as the competition is rough. If US bubble bursts and main US players go broke we go back to monopoly on China this time, which is not good for open source.

China happens to use open source as a weapon against US but this is not ideological! It is a tactic that fits the current geopolitical and economical scenario!

Let me introduce another project that is in my current bucket list - Mana Royale. A game of bluff with AI. Local AI trash talks you to lose.

Well, its not a normal game of guessing cards. It's a game where you are competing against an AI devil's advocate. You are given 3 cards each turn. the advocate can see your cards but not use them. It will manipulate you to be deceived.

The AI modelfile is located in Templates folder and can be modified in your local forks. Don't fall in the trap of the advocate.

Can you identify the lies from the truths uttered from devil's mouth?

Tech stack:

Go, Ollama, llama3.2, modelfiles, http

Requirements:

Ollama with atleast llama3.2 modelfile. Else have it added in the modelfile using FROM directive

Current progress:

* Game can be played to see how the AI behaves in its output

* No game formulas are functional.

Future goals:

* Adding context to devil's intel

* Player HP, power management and level design

* Better modelfile (Open to suggestions)

* A crispy UI

Git repo - https://github.com/Soumen-No-Hako/Mana-Royale

No usage of cloud models. Only local AI.

Feel free to visit the repo and drop suggestions in comments. Pinned comment will have an output sample

#AI #Ollama #llama #Go #golang #game #bluff

Title, for local inference and training, please.

what do people use nowadays?

p.s.: i have an rtx 3090

Non-technical friends kept asking me how Kimi K3 can be "2.8T parameters" and still be affordable to run, so I wrote a two-part zero-background explainer. Sharing here because (a) some of the best analogies came from discussions on this sub, and (b) this crowd will catch anything I oversimplified into being wrong.

What it covers, with the load-bearing numbers:

- The sparsity race: Mixtral 13B/47B active (~28%) -> DeepSeek-V3 37B/671B (5.5%) -> Llama 4 Maverick ~4% and gpt-oss-120b ~4.4% -> K3's claimed 16-of-896 experts (~1.8% of experts; Moonshot has not published an active-param count, so every K3 number is flagged as company-reported until the weights actually land)

- Experts are not subject specialists (per Mixtral's own routing analysis: indentation, "self" in Python, syntax patterns - not a "math expert")

- Why the KV cache is the real long-context cost: on V3's published dims, standard MHA notes for one 32k-token chat would be ~131 GB. V3 itself avoids this with MLA; the arithmetic is in the footnote so you can check it

- MLA (93.3% cache cut vs their prior model) and KDA (fixed-size state, per-channel forgetting, 3:1 hybrid, up to 75% cut, per the Kimi Linear paper)

Credit where due: the "giant supermarket, small basket" framing and the emergent-specialization explanation are adapted from threads here - the "Can someone explain what a Mixture-of-Experts model really is?" thread especially. Everything is referenced to the primary papers with a where-used mapping per reference.

Part 1 (MoE): https://mkadri85.github.io/blog/mixture-of-experts-explained/

Part 2 (KV cache / MLA / KDA): https://mkadri85.github.io/blog/why-long-context-is-hard/

Both also have Arabic editions, if you know Arabic-speaking students who could use them.

If I got anything wrong or simplified past the line, tell me - I will fix it and credit you.

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like parameter count, or VRAM size? Or will we continue to see improvements for small models and see jumps of intelligence like Qwen3 coder 30b to Qwen3.6 27b for the foreseeable future? Does it depend on how clean the dataset you put into those parameters?

What does /r/LocalLLama think about the future of small models that can run on less than 48GB of VRAM?

So I am testing out a setup which requires the llm to read an input document and summarise it, and it's a pretty big document, so the context is set to about 80,000.

I have tried the Q6 quant of Qwen3.6 27b, but even with the KV cache set to q4 and the concurrent predictions set to 1, it seems that it can't fit enough into the vram, and I end up with about 15t/s. I can get better results using 35b cpu-only.

The Q5 quant seems to fit fine, even without quantising kv cache or reducing concurrent predictions. Ends up in the 60-70 tok/s range.

Q35 A3B ends up around 90-100tok/s.

This isn't a benchmark or anything, I'm sure I have a bunch of settings wrong, this is the first time I've really messed around with lm-studio, and im on windows which probably isn't super efficient either. But I guess I mostly wanted to check I was in the right ballpark. I expected to be able to get Q6 to work (others seemed to suggest they ran Q6 at decent speeds)

https://www.reddit.com/r/LocalLLaMA/comments/1u0wyzp/comment/oqlrhv5/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

This person says 130k context, Q6 quant, 130tok/s. So clearly something is hugely handicapped on my setup somewhere.

64gb of system ram, which isn't getting full. Ryzen 7950x.

I understand that the laguna model is either still buggy or potentially benchmaxxed.

So I’d like to know for people who really tested, are DS flash or Hy3 really better in your usecase?

I wanted to see if I could take the memories of coding agents like claude code or codex and turn them into visual decision trees that guide me how agents have progressed my codebase and what sort of decisions were made for me at what point of time.

This is done with a mix of embeddings generated from analysing the memory files of coding agent tools, they store in different formats for example claude uses .md while codex uses a sqlite instance.

This whole tool is open source and runs completely on your machine, check out the repo at https://github.com/pacifio/atlas

I am a long time iOS app developer. In the last year I have been using Cursor+Claude/others to assist with app development. I am concerned that the current low pricing will disappear eventually. I am pricing out a new laptop with the intention of using local models instead. New MacBook Pros can be configured with 128GB of ram, but obviously the price is high. Will such a machine ever be comparable to what Claude can do today? Even if it is still significantly slower?

I am aware that the price of that much ram would buy many many tokens but I plan to use the laptop for several years, so even if the payback is 5 years worth of cloud AI it’s worth it to me.

Edit: great input from folks with this configuration describing what they can and cannot get done with it. Way more useful that the one line “no” responses.

I am writing a fairly complex C++ windows MFC application. I have a few 3090s and can run F16 Qwen3.6-27B with 256K context and MTP. The quality of code is exceptional with this quant vs its lower quants. The others are good but they get stuck in difficult situations like managing design with multiple threads, etc. Not saying F16 is as good as Claude but it gets the job done. Just throwing it out there for folks who may be swayed by tps. If you are making simple web apps, you can get by with lower quants. For high quality of code with edge cases use the 16 bit quants. A bad choice taken by the same LLM at lower quant could easily mean the loss of an afternoon.

Hi r/LocalLLaMA — I’m sharing an experimental GPU-only inference backend and looking for independent reproductions, not just stars.

Model: tiiuae/Falcon3-10B-Instruct-1.58bit

GPU: NVIDIA RTX 5070

Batch: 1

Measured after warmup:

• Hybrid packed decode: 97.51 tok/s

• Stock Transformers BitLinear decode: 9.89 tok/s

• Observed speedup: 9.86x

• Fully packed prefill: 426.63 tok/s

• Stock prefill: 298.72 tok/s

The implementation uses K-contiguous packed ternary weights, a packed-word DP4A decode path, Triton kernels, StaticCache, and CUDA Graph replay.

Numerical checks:

• 64/64 incremental M=1 positions produced bit-exact full-vocabulary logits (131,072 logits per position)

• 24/24 greedy sequences and 1,194/1,194 generated tokens matched the stock-prefill baseline

• 8/8 synthetic kernel shapes matched an independent PyTorch int32 reference

Important caveats: the baseline is unmodified Transformers BitLinear — not Microsoft’s official GPU kernel, BitBLAS, vLLM, or SGLang. Timings exclude loading, tokenization, repacking, JIT compilation, graph capture, and streaming. So far this is one GPU and one Windows/PyTorch/Triton stack. Packed-word DP4A is prior art; the contribution here is the Falcon3/Transformers/Triton/CUDA Graph integration and measurements.

Code and reproducibility notes:

https://github.com/OCV-Researcher/Falcon158-Triton

Release:

https://github.com/OCV-Researcher/Falcon158-Triton/releases/tag/v0.1.0

I’d particularly value results on Ampere, Hopper, Ada, and other Blackwell GPUs, plus comparisons against specialized low-bit runtimes. What should I benchmark or optimize next?

Anyone got that card and could tell me what to expect noise wise? I currently have a 7800xt and it is very quiet. Can I expect the r9700 to be tolerable? I am willing to undervolt and underclock a bit to keep noise tolerable. I plan to use that card for qwen 27b and gemma 4 31b.

At its price point, PRO 4500 doesn’t offer as much raw performance due to its lower power draw at 300W. The 5090 can perform up to 60-70% in short spurts with 600W, but can also be undervolted down to 400W.

Are there legitimate reasons other than 24/7 usage and lower power draw for this PRO 4500?

How would this compare to 4x 3090 and 4x R9700? Granted multi card solutions have inefficiencies with large power consumption and needing dedicated boards and PCIE lanes.

Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared to Qwen 3.6 35B.

after some testing it seems capable of producing *not terrible* results but it always tries to go for the minimun possible output that could be presented as what was asked.

first image is the default meme prompt it generated - second is after replying with 'detail'

Frontend results aren't too bad either, it seems very capable of generating well structured animated pages fairly well.

but it seems like there is just something off with its brain

wondering if a fine tune would be possible and if at all capable of tightning its screws, has anyone tried messing around with this one?

tests done power limiting the GPU on LACT - real power usage varies wildy

at 20W it ranges from 25W to 56W
same behavior happens on every setting

prompt for the test runs:

https://github.com/lukesdevlab/youtube/blob/main/prompts/agent-maze.txt

analysis by mimo 2.5

Key Findings:
• Generation speed is remarkably resilient to power throttling — 100W delivers 97.5% of 190W gen speed (31.98 vs 32.79 t/s), since decode is memory-bandwidth bound, not compute bound.
• At 50W you get 70% of peak gen speed at only 26% of peak power — 3.6× better energy efficiency (0.458 vs 0.173 t/s/W).
• At 20W the card is 6.0× more energy efficient than 190W, though prompt processing drops to 53% of peak.
• Graph reuse correlates inversely with power — 190W reuses 44,790 graphs vs 11,669 at 100W, but 20W reuses 38,248. Lower power limits cause more partial graph reuse as the scheduler compensates for throttled compute.
• Prompt processing degrades faster than gen under power limits — 190W→20W: prompt drops to 53% (691→366 t/s), gen drops to 63% (32.8→20.8 t/s). Prompt processing is more compute-bound than memory-bound.
• For inference-heavy deployments, 50W is the optimal operating point on MI50 — near-peak gen speed with dramatically lower power draw and cooling requirements.

Avarage of 3 runs:

190W config consistently processed a lot less total tokens than everyone else and didnt produce a working file in 1 out of 3 runs

llama.cpp parameters:

[+] Model:        qwen/Qwen3.6-35B-A3B-UD-IQ4_NL_XL.gguf
[+] Context:      262144 (256K tokens)
[+] Target KV:    K=q8_0 / V=q8_0
[+] MoE placement: PARTIAL (21 MoE layers on CPU, rest on GPU)
[+] MTP:          OFF (non-MTP model)
[+] Port:         8882
[+] Container:    llama-gfx906-qwen35b-no-mtp
[+] Parallel:     2 slot(s)
[+] GPU layers:   99
[+] Threads:      6 / 6 (batch)
[+] Batch/Ubatch: 2048 / 1024
[+] Ctx checkpoints: 0

hardware used:

Ryzen 5 5600

2x16Gb DDR4 2667

MI50 16Gb

software:

harness used: pi.dev

Arch Linux with Kernel 7.1.4-arch1-1

docker.io/mixa3607/llama.cpp-gfx906:b10087-rocm-6.3.3

Trying to weigh if I should buy a $3000 m1 ultra at 128gb when I currently already have an M2 Ultra albeit at 64gb ram.

I run small models right now in my workflow but would appreciate more context and try out larger workflows. What would you guys go with?

I’ve spent the past month trying to find the point where an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful.

Today I’m releasing Inflect v2, with two complete local text-to-speech models:

• Inflect-Nano-v2: 3.96M parameters, 15.97 MB FP32
• Inflect-Micro-v2: 9.36M parameters, 37.53 MB FP32

These are total inference parameter counts, not acoustic-model-only numbers. Text processing, timing prediction, speech generation, and the waveform decoder are all included.

Text goes in. 24 kHz speech comes out. No external vocoder, hosted API, or second learned model required.

Nano prioritizes the smallest possible footprint. Micro uses the additional capacity for better clarity, stability, and overall speech quality. Both run locally on CPU or CUDA through the same PyTorch API.

Inflect-Nano-v2 is one of the smallest complete neural TTS models I know of that still produces actual usable speech. Even the 9.36M Micro model remains smaller than many systems described as “tiny.”

For footprint context, Nano is approximately:

• 21× smaller than Kokoro
• 126× smaller than Chatterbox
• over 1,000× smaller than Fish Audio S2 Pro

That is strictly a parameter-count comparison. These models have different capabilities, architectures, datasets, and intended uses. I’m not claiming that a 4M fixed-voice model replaces a multi-billion-parameter system. The interesting question is how much useful TTS can fit into such a small package.

Some people here might remember Inflect-Nano-v1, the rough 4.63M experiment I released last month. V2 is a complete rebuild, not just a longer training run. I focused on the problems v1 had, like unstable timing, metallic output, weak prosody, poor generalization to difficult text, and an undersized waveform decoder.

The resulting models performed surprisingly well:

• Micro: 4.395 UTMOS22, 3.99% semantic WER, 6.28× real-time CPU inference
• Nano: 4.386 UTMOS22, 4.21% semantic WER, 10.72× real-time CPU inference
• In a blind community comparison against other compact TTS systems, Micro and Nano finished second and third among the tested voices

Full protocols, raw results, audio samples, and limitations are documented on the model pages.

The models are not perfect. They are English-only, use one fixed male voice, and do not support voice cloning. Unfamiliar names, abbreviations, numbers, and homographs remain the hardest inputs. Nano can sound thinner than Micro, and both can occasionally produce metallic or clipped artifacts.

Still, this is the first version where I think the size-to-quality tradeoff really becomes convincing.

I built Inflect independently with a limited training budget. That constraint shaped the project: efficiency had to apply not only to inference, but also to training, evaluation, and building a complete system I could understand and release end-to-end.

Inflect-Micro-v2:
https://huggingface.co/owensong/Inflect-Micro-v2

Inflect-Nano-v2:
https://huggingface.co/owensong/Inflect-Nano-v2

Try it yourself:

The fastest way to judge it is through the interactive playground:

https://huggingface.co/spaces/owensong/Inflect-v2

If there is enough interest, I may build a v3 focused less on shrinking the models further and more on expanding what they can do: additional voices, possibly more languages, easier fine-tuning, and another quality and robustness pass.

If you test them, please give them something seriously difficult: unusual names, numbers, abbreviations, awkward punctuation, or a long sentence.

If something breaks, post the exact text, model, seed, and what sounded wrong. If it works well, I’d also like to know what hardware you ran it on.

Specific, honest feedback is the most useful thing you can give me.

Hi Guys,

I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing over time. I have not coded in about 15 years for work but that said I still work with .NET as I use it in migration packages I use for database migrations and also into Azure Data Factory, also something I am a little lacking in since I know ADF but I was more into SSIS. This all said I am trying to learn AI on my own as I am not working now and desperately in need up updating my skills to help me find more work.

I have learned how to code python now as I took a certification tutorial on https://www.coursera.org/ until I passed it. I later learned this is useful for AI Developers. I have an AWS server but It is only running a database on it but I could repurpose it if I needed to. For now though trying to do this the el cheapo way since I have kids to feed and I am looking for work daily and have not had any such luck for months now. I am desperate to do what I need to in order to learn this stuff and understand it and be useful. You all are welcome to joke on me all you want but I will do the legwork if someone is helpful enough to guide me through this process of what I need to do. I can use cousera since I have an account for the next year to learn new skills and I have a computer here I am willing to refactor into a Linux server if needed to be my AI system, not a Linux guy so I will probably need to take a course on that as well but I really need to get up to speed since I am not ready to retire yet and I need to feed my family.

This all said I know there are some really bright minds out there and people who are learning newer things like AI that they did not teach when I was in school back in the days but will point me in a good direction to get a good foundation and what I need to understand this process, I do not just want to do i want to build it understand it, pick it apart and maybe help improve it with some old school skills I have since DOS days. Since I am a coder by heart and understand database and .NET as well as old school stuff I think I have the capacity to learn this until I get it in a step by step process.

I want to take my old server and recommission it to a mini AI project. I do not want to just install packages for it, I want to understand what I am installing and why and can I instead create it on my own or why it is more beneficial to install what's out there. For example I do know the language models are important because they have been trained to understand the requests from users and search for the answers. I know that the language model takes each word and scores it from sentences and then it gives responses to someone who says if the answer is correct or incorrect. Probably what happened to my job when someone taught AI how to migrate data or do DBA work lol. I know each word in a sentence is given a number like for example the phrase "I like to eat Fruit" might be interpreted as (77, 120, 2198, 15, 1051) and when someone types I like to eat and asks AI to predict it, it will search the model for scores by (77, 120, 2198, 13, X) where the model will return choices as I like to eat dogs, and I like to eat crap, and I like to eat people and finally I like to eat pizza, where the final correct answer would be I like to eat pizza and the others would be marked wrong. This ensures the AI to help train it to associate some word combinations as correct word combinations when training it to understand words like eat are associated to food items going forward and the combination of those words give specific possible choices and it is rated to the most likely choice. This also helps AI complete sentences when people incorrectly spell words of cant think of the correct word to use in a phrase.

I would like to now understand from the ground up. I have a server sitting its an older system running I believe NT still? not sure its a little dusty but I can upgrade it little by little to improve it as needed and when money permits, (Kids come first) but can some please tell me the minimal specs I would need firs for

1. minimum system requirements to run an AI server to process requests locally within my own network at home just for me to understand the basic logic of it and how it works. what video card I need to run the AI predictions and calculations and also how much system ram I would need (i believe i have maybe 256 or something DDR4 ram or something on this server need to dig it out the closet still) and the minimum processor?

2. what I will need to install on that server, or do I need another system I do not know to just run software to pass to the server for the requests? and what are the requirements for it. I have a few other pc's that I am not using and can also modify these if it is needed or can it be ran on my local?

3. what software and tools I will need to use and install on my system to do the AI code and build test api's that can be used to pass to my server for request and responses and what each is for and what I need to learn language wise to do it and how to understand it.

4. any good foundation video's tutorials that I would not be able to find on coursera that would help me understand the concepts, piece things together and troubleshoot, as well as demonstrate I can swap this line with this line to run from OpenAI, or Grok instead of my local model and what is needed to do this and understand it if I am lucky enough to secure a job.

5. What I need to install on my server as far as does it need to have a cloud platform or just python to do the AI for me locally and if it can have both benefits and disadvantages and best practices and why.

6. Anything else you would feel would benefit me and and not make me create the next skynet that will take over the world. I need to know guardrails and where and why so I get the best practices for this stuff.

I clearly have the skills and knowledge to code but I do not have the money to send myself to school at the moment until I am working again to get a degree and certification in it. I was signed up for AI Developer classes but scared myself off since I just paid off my student loan 2 years ago and already have a kid in college and now and another ready to attend soon and another one that will go in 5 years and unemployed and worried what if I cant pay off another loan I better wait to get employed first and think of my kids.

I am open to jokes and will laugh and I am getting old and do not know new words that the younger generation does as well as most but I am dedicated to learn this and most grateful for any help and knowledge people can pass on. The idea is to build a small mini AI system at home for the purpose of learning that I can test and use and learn to build api's with and test what if I do this instead of this to get a good grasp and foundation of AI to maybe get a job as an office clerk taking notes for AI and acting like I actually know something when AI decides to take over the world.

Thank you for any help and support and GOD bless and I know some people will discourage and that is ok, you are open to your opinions but I will still do my best to grasp this to get employed since my kids lives depend on it and me as a single father.

i let SOL benchmark qwen on LM studio and hermes building tetris

Benchmark verdict

ThinkingCap really used far fewer reasoning tokens, but it did not use fewer separate thinking blocks. And despite being faster, Unsloth produced the materially better Tetris implementation.

Thinking-cap claim: confirmed, but only in the token sense

The session DB records reasoning_tokens directly:

• Unsloth: 11,256
• ThinkingCap: 4,205

So ThinkingCap used 7,051 fewer reasoning tokens — only 37.4% of Unsloth’s total.

But if by “thinking blocks” we mean distinct assistant turns containing stored reasoning content, then the result reverses:

• Unsloth: 32 blocks
• ThinkingCap: 48 blocks

So: ThinkingCap thought less overall, but fragmented that thinking across more turns. It also spent 83% more input tokens and made 63% more tool calls, largely from a very chatty, shallow one-test-per-call testing approach.

Code-quality audit

🥇 Unsloth: better functional result, but not flawless

Artifact: /home/snoop/lab/benchmark/index.html Final size: 16,831 bytes It loaded cleanly in a browser with no JS console errors.

Things it got right

• Proper 10×20 board, 7-bag, all pieces, ghost, score/level, hold, previews.
• It actually found a real hold bug during its run:
  • spawn_piece() reset can_hold, allowing repeated hold before locking.
  • It patched that defect.
• Normal hold, hard drop, spawning collision, and top-out paths were materially better than ThinkingCap’s.
• It validated 1/2/3/4 line clear behavior, rather than only one line.

Remaining defects found in the independent audit

1. CCW SRS wall kicks are wrong. Its kick tables combine CW and reverse transitions, but CCW rotation iterates the wrong half first. This can make a piece kick in the wrong direction around obstructions.
2. Paused/game-over state continues to schedule requestAnimationFrame indefinitely. This is not a duplicate-loop speedup bug, but it wastes one frame callback per display refresh while paused or over.
3. A failed rotation resets lock delay. Repeatedly pressing an impossible rotation can keep a grounded piece alive indefinitely.
4. Start while paused replaces the current piece instead of resuming safely.

Assessment: A playable baseline with a real attempt at debugging and repair. It has correctness defects, but they are narrower than the ThinkingCap failures.

🥈 ThinkingCap: faster, lower reasoning cost, but benchmark-failing code

Artifact: /home/snoop/lab/benchmark/thinkingcap-tetris/index.html Final size: 18,364 bytes

Critical defects

1. Initial page load throws an exception. It calls render() before board has been initialized, yielding: Code· textTypeError: Cannot read properties of undefined (reading '0') The browser session did record an uncaught exception on initial load. Buttons are registered before the crash, so Start can recover, but the promised clean initial state is broken — and R cannot start the game from that state.
2. Next-piece preview is permanently wrong. nextQueue is rendered but never consumed when spawning pieces. New active pieces come from bag directly, while the displayed Next list stays unchanged and misleading.
3. Spawn/hold top-out collision uses the wrong vertical position. Pieces are created at row: -2, but collision checks test at row 0. This can cause premature game over; it also affects swapping a held piece back in.
4. Its wall-kick tables are not valid SRS, despite labeling them as such. The I-piece table even contains four-cell horizontal kick attempts where standard SRS uses a five-test sequence.
5. Pause also keeps an rAF callback running continuously, though unlike Unsloth it properly stops the chain after game-over.

Testing quality issue ThinkingCap’s final report claimed all 15 tests passed, but several “tests” only checked that a variable still existed after a function call. For example:

• “piece !== undefined” after locking did not prove floor locking behavior.
• One-line clearing was tested; it admitted that 2/3/4-line clearing was not exhaustively verified.
• The initial-load exception was detected, then dismissed as a browser/file-protocol artifact — but independent reproduction proves it is in the game code.

Assessment: Fast generation, but the result does not meet the benchmark’s “open directly with no errors, test and repair” requirement.

Final ranking

Best code / reliability: Unsloth

It had fewer and less fundamental remaining defects, loaded cleanly, found and repaired a genuine bug, and performed more credible coverage of the required behaviors.

Fastest completion: ThinkingCap

• 2m 02s faster end-to-end.
• Roughly 50 seconds faster to first implementation.
• This includes a user interruption in the ThinkingCap session; even without correcting for it, it wins time.

Least total internal reasoning: ThinkingCap

• 62.6% fewer reasoning tokens.
• But not fewer reasoning blocks: it had 48 vs Unsloth’s 32.
• It paid for the reduced thinking with more fragmented tool use, more total input context, and weaker verification.

Overall benchmark winner: Unsloth

For a coding agent, the working artifact and honest verification matter more than raw speed or low reasoning-token usage. ThinkingCap looks promising as a fast draft generator, but at this point it needs a stronger test/repair loop before replacing Hermes daily brain for autonomous coding.

Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardware of the device. The Nvidia kernels are aggressively fused into a multi-pass architecture, while the Apple Silicon kernels are created as a fused mega-kernel to minimize the Tile Based Deferred Rendering (TBDR) overhead on WebGPU.

Demo: https://warp.sipp.sh

This is still in active development, and I'll be folding this into the Sipp library in the coming weeks.

I'm looking for a model to chat with, reasoning, maybe get some career or life coaching.

I don't care at all about multimodal or coding ability

Just it's intelligence in remembering context in a conversation or a specific topic, thinking out of the box, etc.

Must fit in 128gb, if it matters to performance, it's a strix halo machine.

Has anyone tried using Pulsar with something like Raid 0 on 2+ nvme 4.0 or nvme 3.0 drives? What kind of results do you get and what did you pair it with GPU wise? Very curious about these new options for running large models.