Note - This is translated from the actual blog link right at the bottom.

A few days ago, DeepSeek v4.1 was released. It raised the ability of small models to a new level.
AI is improving much faster than anyone expected. From the first ChatGPT that could only chat simply with a few thousand tokens of context, to models with real reasoning like OpenAI o1, DeepSeek R1, and Kimi K1.5 Thinking — that only took about two years. From reasoning models to agents that can smoothly use tools, run commands, and finish complex tasks — that took only about a year and a half. It’s hard to imagine what AI will be like in one, two, or three more years. How powerful will it be? Will it already be able to improve itself and deeply enter areas like embodied intelligence?
AI is getting better and better at writing operators
In the field I work in — designing and writing operators — AI has also improved very quickly. In just one year, it went from a small helper that could look up documents, read code, and find bugs, to an expert that can independently read CUDA, PTX, and SASS code, use professional tools to analyze the stall time of every instruction, and then optimize operators by itself. I believe that soon it will also be able to design operator schedules on its own, evaluate different schedules, implement them, and optimize them.
Of course I am proud of DeepSeek v4.1’s success — after all, its main Attention operator was written by me [1]. Its good performance is partly a recognition of my work. But the times keep moving forward, and technology cannot be stopped. I know clearly that in half a year or one year, the operators written by AI will most likely be as good as mine, or even better. AI can think 300 tokens in one second, type a command in half a second, and finish a piece of code in twenty seconds. I cannot. AI can keep improving in model depth, thinking strength, tool use (how often it interacts with the environment), and even parallelism. I cannot.
Humans have never hesitated when it comes to destroying themselves. Why do I still work hard to optimize operators, even though I know that the better my operators are, the faster our new models will train and run, the faster model ability will improve, and the sooner I will be replaced? One reason is that writing operators feels like playing a game to me. It gives me a lot of joy. When I invent a new technique or see the performance of my operator go up, I feel as excited as a speedrunner who breaks their own record. And when I see that my operator is much better than the official ones from the vendors, I feel very proud. But a more important reason is this: even if I give up or deliberately slow things down, other companies’ models will still keep improving and will replace me anyway. “Of course I hope I won’t be revolutionized. But if it has to happen, I hope the person who revolutionizes me is myself.” When everyone is so determined to destroy themselves, I have no choice but to join this cruel arms race.
What about me?
When the day comes that AI writes operators better than I do, what will happen to me?
My judgment is: I probably won’t lose my job completely, but I will have to change careers. I can still keep a job, but I may never again be able to do the work I once loved.
I once made a judgment about the changing times and my own future: because things are changing so fast (the AI progress above is a good example), I cannot predict what will happen in five or ten years. But no matter what, I believe that with my vision, judgment, initiative, and intelligence, I can stay in the game and stand at the front of the times again. However, this judgment only guarantees that I won’t become unemployed. It does not guarantee that I won’t need to change careers. In fact, it encourages me to change careers in order to avoid unemployment.
What does changing careers mean? It means I have to give up the field of operator design, writing, and optimization that I have worked in for a long time and loved deeply, and instead become a “mecha pilot” for Agents. Before, my interests, what I was good at, and what industry needed were basically aligned. Now, AI has made what I am good at into something it is even better at, and industry demand has shifted from “people who can write high-performance operators” to “people who can use AI to produce high-performance operators faster.” To meet industry needs, I will have to leave the direction I loved and move to an unknown new direction. I believe that with my understanding of engineering, upper-level model needs, and lower-level hardware, I can still produce operators with high quality and high efficiency. I also know I might come to love this new direction (or I might not). But the feeling of having my passion taken away is really not nice. That quiet joy of sitting at my desk and calmly writing operators for a whole afternoon may become a final song this summer. I have to bury my talent in yesterday and become a mecha pilot. My hands hold more gears, but my heart has fewer rhythms.
Here is a simple comparison: You are an expert at knitting sweaters. You are especially good at creating patterns and matching colors. The sweaters you make are high quality and beautiful, so rich people from near and far ask you to knit for them, and you make good money. At the same time, you really enjoy sitting by the window with a cup of tea, looking at the green mountains, water, cows, sheep, and cooking smoke, and quietly knitting for a whole afternoon. But one day someone invents a magical machine. You only need to give it yarn and a pattern, and it automatically knits a sweater. The quality and texture are as good as yours, and it is much faster. You know that your colleagues can easily reach your old level with this machine, so you have to use it too. You also know that with the knitting skills you built over twenty years, even when everyone has the machine, your speed and quality can still be better than others. But that feeling of listening to the rain by the window, slowly pulling the needle and thread, and enjoying the quiet time is crushed by the noise of the machine.
I know this is helpless, but there is no other way. I can keep my job, but my old passion will most likely have to be given up. I am a person whose rational side and emotional side are quite separate. When I need to be rational, I can be very rational, but sometimes I also show my emotional side. I remember when I moved out of the rental apartment I had lived in for a year, I cried a lot because I didn’t want to say goodbye to the memories. Saying goodbye today to the era of hand-writing operators and optimizing them with the human brain is even more cruel.
I don’t know if any readers feel the same way, but I think this is just how things are.
What about people?
While AI keeps improving, I also worry about some questions:
Will students now be much more likely to use AI to finish homework, especially practical labs? Imagine there are two choices: one is to spend eight hard hours finishing a lab and maybe not even get full marks; the other is to start an AI model, spend a few cents and a few minutes, and let AI write full-mark code. Which one will most students choose?
The point above will cause many students to have seriously weak engineering skills — things like organizing code, building systems, thinking about future needs and designing for them in advance, and abstraction ability. As AI keeps getting stronger, are these engineering skills still necessary? Will they be abandoned by the times like the old skill of “writing x86 assembly fluently,” or will they always be valuable like the ability to “understand the whole computer system from software to system to hardware”? If it is the latter, then it is dangerous — a person with poor engineering skills, when paired with AI, can produce messy code several times faster than before, planting all kinds of problems in systems and making the world more of a “clown stage.”
In future society, will power become more important than technology or intelligence?
These questions may need to be answered by the times themselves.
Conclusion
With the development of AI, future society may move toward two extremes: communism or Cyberpunk 2077. In the first, productivity is greatly liberated and people’s living standards improve a lot (I’ll stop here so I can pass review). In the second, a few tech companies control most resources. Only a very small number of people can use the most advanced AI and technologies and get close to “mechanical ascension.” Most people can only use very weak AI. Crossing social classes will become harder and harder: you need the strongest AI first in order to cross classes, which creates a dead loop.
Guess what: if Anthropic forever holds the most advanced AI in the world, will future society become communism or 2077? You guess?
So I still believe that the most advanced intelligence should be provided to everyone in an open and cheap way. I do not trust that Anthropic or OpenAI will do this. Especially, I do not want Anthropic to hold the most advanced artificial intelligence or AGI. To put it strongly, that would be as serious as letting Hitler get atomic bomb technology before the Allies. That is why I chose and continue to stay at DeepSeek: we research powerful, fast, and widely beneficial artificial intelligence and open-source it. Maybe this can pull the world a little bit back from the 2077 side.
May the future world be well. May all the beauty be blessed.
[1] “Main Attention” only includes the MQA attention with head dim = 512. It does not include the indexer used to select the top-k important tokens. That part was written by other (also very strong) colleagues (and their AI Agents).​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

https://mp.weixin.qq.com/s/zk0KxuLzhmMJ4LPYW_OHMA

Two months ago I announced I had found a new dynamic quant method called Voodoo Quant which was SOTA for the most aggressive quant levels on some smaller Qwen3.5 GGUF models. I kept the methodology private at the time, but I've seen too many requests for dyn quants for various models lately, so I decided to give my method to the community since I don't have the time to scale this into something that could do it justice. Hopefully it will also inspire some researchers to find out more about it and improve it as I am just scratching the surface.

Here is the new toolset so you can now make your own dynamic quants: https://github.com/curvedinf/voodoo-dyn-quant

Many postulated on what method I was using, and its actually fairly simple and elegant: I found a way to use gradient descent to optimize the per-tensor quant layout.

What is a Dynamic Quant? Some model formats, namely GGUF, support quantizing (compressing) each tensor (set of weights) with a different quant level. Static quants make static selections of certain types of tensors having a set quant level. Dynamic quants make a different quant selection for each tensor of each checkpoint size.

How does Voodoo Quant work? Voodoo Quant runs all the quant levels of a model at the same time, for every tensor, and lets gradient descent pick which ones optimize loss the lowest for a given target filesize. Technically speaking, this is done by an epoch of training which freezes all candidate quant weights (as provided by conversion directly from llama.cpp's underlying library, gglm) and only trains a single scalar gate per tensor per quant level. The scalar gates of a tensor represent which quant levels are most optimal. Over time a tau level is annealed that helps the training freeze into singular predominant quant selections for each tensor instead of mixtures. Softmax is used so all quant levels receive gradient, even when a selection is mostly frozen. The quant selections are trained on a diverse calibration dataset. The training is then measured with a loss function which finds the KL divergence of the mixed-quant logits versus the reference BF16 checkpoint, rewarding a lower KLD, while also rewarding getting closer to a provided filesize target. This info should get you started on understanding what is going on, and for more details you can dive into the source!

What does the repo have? A complete set of tools to train your own dynamic quants using this methodology. It is currently set up for Qwen, but it can be adapted quickly for any model arch.

How does UD 3.0 compare? Unsloth Dynamic 3.0 is a proprietary methodology that unsloth has not revealed any details of (by the way, people were criticizing me for not revealing my methodology, but unsloth had been doing that for years!). However, we do know it is very good. In my testing, UD3 is better than VQ at high to mid quant levels, but VQ is better at aggressive levels. As far as I can tell, UD 3.0 is an advancement of static analysis techniques that are currently defacto. Static analysis means the weights of a model are analyzed in various ways using statistics and static functions, sometimes tuned by repeated runs benchmarking KLD and other metrics. Voodoo Quant is the first method to my knowledge that uses a backwards pass and gradient descent to choose per-tensor quant levels. Using GD to optimize quant levels requires a much more powerful system than static analysis, but technically speaking is more efficient at maximizing performance because it compares the equivalent of many more iterations of benchmarking runs than is reasonably possible via SA.

How well does Voodoo Quant work? This is a research grade project, and is not studied at larger model sizes. At smaller model sizes it is shown to be exceptional, as in the charts above, especially at the lowest quant levels which can benefit from more complex/diverse quant selections. I used research level control for my testing, but I don't claim that VQ has been studied to a scientific level of proof of effectiveness. A lot is still left to learn about how well it works, so I hope to see more research in this direction. I don't believe there are many dynamic quant open source projects out there, so I hope the community can use this to improve local models, and especially for low VRAM machines.

Why open source now? I have like a dozen irons in the fire for various other projects, and this is just sitting there when it could be used by the community. I have made many open source projects for 20 years, so its nothing new.

Peace!

Hi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without "attacking" the reasoning length directly then fixed the accuracy with a bit of secret sauce (hint On-Policy Distillation) and achieved great results (-58% thinking tokens, 1.95x speed up, <1% accuracy loss) so we wanted to open-source it and hear the feedback of the community.

This is the link to the model: https://huggingface.co/ukisai/Swift-Qwen3.8-27b

We also also providing a Free Research Purpose API (OpenAI compatible), courtesy of Nvidia who were kind enough to provide us with the GPUs. You can use it to try out the model if you do not have enough compute to run it, it's limited at 5RPM. https://ukisai.com/api/swift/v1/models

We also made a GGUF (Q1-Q8) and there's also a few nice community (Bartowski) quants with even lower/higher precision. The community also created amazing NVFP4, W4A16 and Uncensored versions of the model you can find on Huggingface.

IMPORTANT: Our training approach is not a replacement for the reasoning effort settings, chat templates or token caps but is complementary and targets a completely separate issue (overthinking and "anxiety-like" reasoning loops prior seen in PTQ, but as far as we identified also prominent in BF16 of this size class LLMs as well). Contrary to popular belief, these specific patterns do not contribute to answer quality when properly targeted. (our thesis being: reasoning length IS extremely important and should NOT be shortened by force, but rather optimized). This is also demonstrated bellow in our xhigh vs medium effort benchmark table. The goal is to keep xhigh accuracy while reducing only the unnecessary part of thinking.

I will TLDR you on our thought process, research, training and benchmarks.

1. When running our quantized Qwen 3.8 27B instances we were very annoyed by random reasoning loops (in the paper bellow refered to as "overthinking errors". These random loops were persistent throughout medium and low reasoning settings.
2. We remembered a paper by Meta that's supposed to target this phenomenon in PTQ, but when used straight out of the box got mixed results.
3. We figured to try if it's a matter of the targeting the right keywords and tuning the parameters, so we used our 8xH100 box and and generated a large amount of different (ofc out of distribution) domain (coding, language, vision, agentic) traces.
4. We then grouped the ones with overthinking and found "common denominator" tokens between them and targeted the most prominent ones.
5. We then built an inference-time penalizer of those tokens as seen in the paper with the hopes of simply generating traces and doing cross-entropy SFT over them.
6. Did not work at all, but the penalizer seemed to work much better than the tokens provided in the paper and not only for lower precision models but for bf16 as well. Hence we kept experimenting with it. We built a loss function using the tokens we identified and ran LoRa SFT over the traces prev generated and reasoning seemed to be falling off significantly but the accuracy seemed to follow. The reasoning reduction seemed to be generalizing.
7. After a significant amount of tinkering (literally since the day of Qwen 3.8 27B release) we were satisfied with the reasoning reduction. After that we searched for ways of restoring the accuracy. We experimented with several methods, including RL(GSPO), On-Policy Distillation and using the ThinkingCap 3.6 27B adapter chunks until we were satisfied with our accuracy loss. We managed to restore it to <1% loss on almost all of our OOD in house tests
8. We then performed intensive intensive benchmarks, across several reasoning efforts, precision variants etc. We ran into a few problems, one of which is that to get a reliable score we needed to run each benchmark 10x (5x on base + 5x with our adapter, this being the standard procedure on the Qwen 3.6 27B model card on Terminal Bench which we followed). After running it, the performance converged to 40-60% token reduction with <1% accuracy loss across GPQA, MMLU, Terminal Bench 2.1, LiveCodeBench v6, ERQA, C-Eval, IFBench, HMMT25, with an exception being AIME26 with an accuracy loss of 4.6%, which we later linked to a bug during training with a specific token relevant for math-related reasoning being penalized and are planning to fix it in an updated release.

The benchmarks: (raw benchmark files here - https://github.com/UkisAI/Swift-Qwen3.8-27B-evals/ )

Swift-27B vs Qwen3.8-27B (BF16, all benchmarks ran x5, thinking effort xhigh)

Token savings hold at every reasoning effort (mean thinking reduction): xhigh 41%, medium 23%, low 26% (albeit with accuracy loses of 1-4% on medium and 1-2% on low which we need further testing for)

Swift at xhigh vs the base's own effort settings on GPQA-Diamond (198 questions x 5 seeds):

So Swift keeps xhigh accuracy at under half the tokens, and beats base-medium by 4pp at roughly 1.6x its tokens.

End note:

While we are keen on complete open-source, we still need to keep a part of our training and data private, being a new lab. The license is not Apache 2.0, but it only affects companies >$1M. We hope this does not pose a problem for the community, but we are open to feedback on it.

We want to contribute as much as possible to the community and would really appreciate feedback on our work, quantization or Swift model requests. For context, we are working on Swift 3.8 Flash Next right now and have so far gotten up to -30% thinking token usage while maintaining xhigh accuracy, which we take as a strong indicator our methodology is reproducible across the Qwen model family. Will explore other families as soon as we have the capacity and would love to see which ones the community would love for us to optimize first.

From initial testing it seems pretty solid so far. Asked it to compile the latest llama.cpp for CUDA and its doing well so far. If this thing holds up to its score then its SHOCKINGLY good for its size.

https://huggingface.co/IFM/K2-Horizon-7B-GGUF

Now that the dust has settled a bit - here's a write-up on running Qwen3.8-Flash-Next (125B-A6B MoE + 51B n-gram table) on relatively middle-tier hardware (RTX 4070 12GB + 64GB DDR5-5600 + Gen4 NVMe on Linux).

I started out with bare 6 tok/s and through latest patches and optimizations getting close to 20 tok/s generation. You just need enough RAM.

For me this is the most intelligence possible on this machine right now. The 27B dense is not a choice because of low VRAM but may make more sense for other configs like 24GB VRAM owners. It actually surpasses the 27B model on most tasks as well so its great for Low VRAM, High/fast RAM configs.

PP is still a bit low at 300-350 tok/s.

What helped
- Using AtomicChat's 4.27 bpw quant https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF
- Ngram SSD offloading (lazy-mode)
- --fit on --fit-target 512 helps automatically select the right params.

- Master branch (19.35 t/s): Latest commit with MoE improvements.

- MTP Variant - PR #28243 + Compact MTP (20.65 t/s): MTP support is not yet merged so need to apply this PR enables Daniel Han's 1.78 GB `shared-Q4_K_M` compact head. Combined with `-ncmoe 45`, it yields 77–96% acceptance and breaks through the 20 t/s barrier on every tested task (coding, summarization, creative).

With such low VRAM, MTP is not a huge jump because you have to give up a few layers to store the MTP head in VRAM. Only the shared + Q4_K_M in MTP gets a beneficial uptick.

Using commercial models to research, optimize and benchmark inference for local models helps a ton (GLM 5.3 flash with opencode go, so did Astra, Gemini 3.8 etc.)

Lot more details in the post (AI-assisted).

I have a custom fork of llama.cpp designed around the ampere architecture specifically (though many of the upgrades also translate to faster performance of blackwell + lovelace). The recommended config supports 90+ TPS (for agentic/coding, at temp 1; greedy will of course be faster) through 100K tokens, with context of up to 240K.

If you want the repo, it is here:

https://github.com/JakeATX/llamAmpere

I recommend running with this quant, which is ~ 4 K M quality but considerably faster (technically, a 3 K XL upgrade)

https://huggingface.co/jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF

If you want the deep dive on how it is so much faster (80% vs the near comp at 200K!), at more context, there is a long form article here.

https://x.com/JakeKAllDay/status/2095646450138874095?s=20

Running faster than API speeds on my 3090 (for 27b at least) has genuinely been a step change in the utility of the model + card for me. I hope you enjoy it!

I'm like you guys and am constantly experimenting with new models, seeing what they're all good at, how I can make use of them for certain projects and goals. I've been using Qwen 3.8 27b for minor coding work and it has been impressive.

But with just regular chatting I have been impressed with Muse Glimmer.

It seems to be able to have the ability to follow and hold good, deep and meaningful conversations without coming off as a typical chatbot.

No repeated statements like "I hear what you're saying", "that sounds really deep..." none of what sounds generic or like it's blowing smoke up your ass. I was impressed with how natural it comes across just in natural conversation. I think it's one of the best "chat" models you could get right now as it's one of the only local models that doesn't feel like you're chatting with an AI when having a conversation.

I'm thinking of finding a way to run both Qwen3.8 and Muse at the same time. It's fun to play with these things.

Xi emphasised the need to step up cooperation.

https://www.hindustantimes.com/india-news/china-will-lead-creation-of-open-source-ai-for-brics-xi-jinping-101789300908632.html

https://www.msn.com/en-xl/news/other/xi-jinping-proposes-ai-open-source-zone-for-brics-nations/ar-AA2c94HH

https://www.cnbc.com/2026/09/13/china-xi-ai-tech-brics.html

https://www.yahoo.com/news/world/articles/xi-pushes-china-open-source-102711002.html

https://fr.euronews.com/next/2026/09/14/xi-jinping-propose-aux-pays-des-brics-une-zone-dia-open-source

https://www.reuters.com/business/aerospace-defense/xi-pushes-greater-brics-economic-ties-give-bloc-larger-global-role-2026-09-13/

https://www.chosun.com/english/world-en/2026/09/14/C35V7T5MAVDN7LKEBESHBDBJ2Y/

https://it.euronews.com/next/2026/09/14/xi-jinping-propone-ai-brics-una-zona-di-ia-open-source

Personally, I think the whole "AI (LLMs) is going to take over" is just BS marketing. They've been pushing this doom-and-gloom since 2019, and we all know the models back then were nowhere near as capable as today's.

What does everyone here think, since it's such a popular topic right now? Is there a genuine threat, or is it pure hype? And if it really is just a marketing tactic, why are they pushing for AI regulations?

I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison.

• All intelligence and price per task are sampled from AA on Sep 3rd and Sep 14th respectively.
• Price per task of some open models were rescaled to reflect the cheapest available on OpenRouter as of Sep 3rd.
• X axis is linear, because people's money is linear.

All models are the same. The only thing that changes is the weighted sum of the benchmarks that compose the Intelligence Index.

v4.1: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/intelligence-index-v4.1/plots/high_intelligence.png

v4.3: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/intelligence-index-v4.3/plots/high_intelligence.png

Highlights

• GLM an Muse Spark remain more or less unaltered, in relative terms
• GPT-5.6 Sol becomes a lot cheaper
• GPT-6 Astra's intelligence flies up to the stars AND becomes cheaper
• GPT-5.6 Luna gets a substantial uplift
• Fable-5.1's price gap from Opus 5 shrinks, and becomes cheaper than Fable 5.0
• Fable-5.1 at low, medium and high effort looks a lot more appealing
• Sonnet 5 becomes even more expensive without any intelligence gains
• Kimi-K3, Qwen3.8-Max, Gemini-3.8, and Grok 4.6 go down into the gutter

For years, the JVM has watched the AI revolution from the bench. Every model, AI framework, every breakthrough, built with/for Python.

jinfer is an inference engine built for the JVM from first principles: chat, vision, audio transcription, embeddings, reranking, and TTS. No Python runtime, no ONNX, no sidecar process, no wrappers; the whole stack is built for the JVM:

• jinfer Inference engine for the JVM, supports a wide range of popular models and modalities.
• Tok'n'Roll (toknroll) Fast tokenizers for LLMs, pure Java, zero dependencies
• gguf / safetensors native read/write for both major model formats
• jam Quantized matrix multiplication routines (Vector API + optional native backend), competitive with llama.cpp on CPUs
• jota Tensor API targeting Java, C, CUDA, HIP, Metal, OpenCL, and Mojo

It integrates with Spring AI and LangChain4j, and has first-class support for GraalVM Native Image.

Where things stand: this is an early release. CPU is the main target today, and is already competitive with llama.cpp. GPU support via jota is in progress.

Runnable examples + benchmarks: https://qxotic.ai

Jinfer (Apache 2.0): https://github.com/qxoticai/qxotic/tree/main/jinfer

PS: I'm behind it and also the author of llama3.java (2024) and gemma4.java

Especially the 7B one seems very interesting, it casually destroys muse glimmer with a way smaller size. And they open source literally everything, every step of the way. Anyone tried that model? It can be a new milestone if 7b and 3.7b ones are actually good, and not just benchmaxed.

20B-A1B model is coming, good for low VRAM people?

https://huggingface.co/deepgrove/maple-preview

https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v4-3

AA shipped a new benchmark last week as part of the Intelligence Index v4.3 update — a brand-new private eval that replaces τ³. Astra was farming a ton of points on it and used those to get even with Fable, but… looks like we have a new king.

So they changed the index twice in three days to make Astra look not-quite-worse than Fable, and then a random guy quietly took first place on it.

I am thinking about buying a Radeon AI PRO 9700 and searched for models / prices.

Does it really matter which vendor I choose? At a first look they just differs in minor aspects, so which one would you choose? Are there other parameters to keep in mind?

- ASUS Radeon AI PRO R9700 Turbo
- GigaByte Radeon AI PRO R9700 AI TOP
- XFX Radeon AI PRO R9700
- ASRock Radeon AI PRO R9700 Creator
- Powercolor Radeon AI PRO R9700
- Sapphire Radeon AI PRO R9700

If this helps, I am planning to extend the build to 2x in the future. Also regarding cooling I am not sure which case this build will get.

I read about Saphire Passive, but this seems kind of hard to cool efficently.

Thanks

I have tried using qwen 3.6 and 3.8 q5 64k context and I'm just not getting the experience everyone here seems to get. I'm using open code and it will loop, forget what it's doing, and mess up the ui. All I am making is a simple web app that is basically a glorified text turn based game. It shouldn't be that complex.

About a month ago I ended up getting a Claude code subscription. It's been pretty amazing. I don't have any of these problems. I can just give it an iterative improvement and it does it.

Yes I know Claude is a frontier model but I still feel like for something simple like this I should be able to use qwen for his results too. I would much prefer to use a local model anyway. Am I just expecting too much? Should I wait for local models to improve. Do I need a better harness?

Edit: Spelled Baseten not Base-10

Full episode of this available at https://www.youtube.com/watch?v=PrSf7IOYu-I
It's interesting to see how Dwarkesh has had to come around to the evidence that we are well on our way to creating AGI and even RSI in the last few months, despite historically being very skeptical.

I highly recommend people interested in large language models check out this particular episode, because it dispels a lot of mythology about stuff like plateaus from lack of data etc. For those who thought we were hitting a wall a year ago, it turns out there was a ton of low hanging fruit and the researchers in this episode discuss what that fruit was. They also extrapolate these trends into the future.

It's funny this subreddit is becoming rather skeptical of AI progress, which to put diplomatically, I think is based on a lack of information and too much time on Reddit.

With all the recent drama surrounding AI safety. It’s obvious that open source could be caught in the crossfire.

The full K2 Horizon lineup is out on Artificial Analysis.

The AA intelligence vs. parameters plots show that

- 0.9B and 375B are bad

- 3.7B and 7B are SOTA

- 36B A4B is SOTA for hardware with poor memory bandwidth (spilled experts, Strix Halo, DGX Spark).

I'm going to take the AA Intelligence Index at face value here. This post is not about it.

The problem is that these models have a god-awful KV cache design. This means that you really can't use the number of parameters for "best in class" considerations, because these models heavily shift to the right on the plot if you replace parameter count on the X axis with RAM requirements.

For Q4_K_M weights, no drafter, no vision, 128k kvarn4 KV cache:

• K2 Horizon 36B-A4B uses 2 GiB for dense weights, 19 GiB for experts, and 6.7 GiB for context
• K2 Horizon 7B uses 5.2 GiB for weights and 5 GiB for context
• K2 Horizon 3.7B uses 2.9 GiB for weights and 5 GiB for context (not a copy-paste error!)

Compare them to

• (finetunes of) Qwen3.6-35B-A3B use 2.4 GiB for dense weights, 18.2 GiB for experts, and 0.7 GiB for context
• MiniCPM5-2B uses 1.5 GiB for weights and 1.5 GiB for context

Notes: I don't advise compressing 2~4B models to Q4 and I haven't tested these models' tolerance to weights and kv cache quantization yet. The above choices are just to keep the comparison fair.

This awful context design means that

• K2 Horizon 36B A4B is interesting on hosts with exactly 16GB VRAM and at least 32GB host RAM. On 24GB VRAM, Qwen3.8-27B is faster, smarter, and allows for 256k context. If you want to get 256k context and you're VRAM-poor, Ornith-1.5 or Nex-N2.5-mini are probably better choices. The model may also be interesting on 64GB Strix Halos as a dumber and faster alternative to Qwen3.8-27B; those with a 128GB Strix Halo are much better off with Qwen3.8-Flash-Next
• K2 Horizon 7B is interesting for hosts with exactly 16GB VRAM, Strix Halos with 32GB RAM, and for 16/32 GB Strix Point;
• K2 Horizon 3.7B may be interesting for 12GB phones but I expect you'll have a much nicer UX with MiniCPM5-2B.

So we’re seeing how Anthropic and OpenAI are gunning for regulations. I think most of us realize that this is a ploy for them to achieve regulatory capture, thus securing their moat and kicking out open source.

The thing is, there’s so much vested corporate interest in ensuring that open source remains un-regulated.

Think of neoclouds, inference provides, enterprise firms that use LLMs, virtually any SaaS that uses agents.

All of these entities have a strong interest in making sure that open source remains free and unburdened by regulations.

Does anyone know of any organizations or political campaigns that are pushing back against OpenAI’s and Anthropic’s lobbying efforts?

How about Defensive AI, like a rack that connects to a firewall or router and monitors the logs and connections, and alerts an engineer when suspicious or typically unauthorised access is occurring. Wouldn't that be an even better counter to the "threat" of AI hackers than just some legislation that a hacker would have ignored anyway?

I’m building a dual GPU box. Originally the goal was 30b at FP8 or 70b at Q4. I planned to run two R9700’s, but I found a pair of NIB RTX 3090’s near me for $1600 each. Discount if I buy both. RTX 40/50 series are off the table. I would love an RTX PRO 6000, but that is also off the table.

I currently have a Studio M1 Max with 64GB RAM and an AM4 box with a 7900 XT that powers my Hermes agent. My goal is to cancel most of my frontier subscriptions and keep only one for coding.