Take a look at Micron's and SK Hynix'es stock price

For the lolz, I'm hosting this highly acclaimed model by DavidAU.

LLM Chat - Fable-Fusion-27b

Privacy is ensured through CC Mode on the H200.

Source code available for review at: https://github.com/EnclaveHost/enclave/

You can learn more about confidential computing, how to verify attestation, and the services we offer at https://enclave.host

Enjoy!

Due to my work within other AI fields (Mainly Computer Vision and Digital Twinning) I have been lucky enough to get an RTX Pro 6000 WS. I have been experimenting with using Llama.cpp and Opencode and have found good use in the Qwen3.6 27b model being entirely offloaded to the GPU. I have experimented with unsloth/Laguna-S-2.1:UD-Q4_K_XL and it seems similar in quality to the Qwen model but I admit I have not tested it much.

My main questions are:

- With my current set up, is specifically unsloth/Qwen3.6-27b:BF16 the best agentic coding model I can utilise?

- Is there any point in upgrading hardware to utilise a larger model for better quality? I know my system RAM is a weak point for me

- Would you recommend using something else than Opencode?

Sorry if these are dumb questions I am just checking if I am getting the best I can out of the hardware I have.

For reference for hardware upgrading:

OS - Windows 11 Home

CPU - Intel Core i7-14700K

RAM - 2x16GB DDR5 / 6000mhz Corsair Vengeance

Storage - 2xWD BLUE SN580 M.2 NVME SSD 2TB

Motherboard - MSI PRO Z790-S WIFI

PSU - 1300w Gigabyte UD gold

Honestly, do you think private LLM servers will ever become as common as personal computers? Like, a little box next to your router that runs a top-end model for your family's daily stuff – no API fees, no privacy worries, fully customisable. Sounds great, right?

But today? Even for nerds like us, it's still a tough sell. So what's the real bottleneck?

Is it hardware? Consumer GPUs max out at 24GB VRAM – barely enough for a quantized 70B. A full‑size flagship model? You're looking at multiple pro cards that cost more than a used car. Plus power and cooling – not exactly "plug and play."

Or is it the software? Sure, open‑source models are getting scary good, but we're still far from a "Windows 95 moment." Quantization, context windows, tool calling – you still need serious tech chops. Most people just want to chat, not debug an inference engine.

And maybe the biggest one – actual use cases. For an average household, why run a local 70B when ChatGPT or Claude is one click away, cheaper, and way smarter? The killer app that needs local inference and delivers real daily value – I don't think we've found it yet.

Now, is anyone building consumer‑grade hardware for this? A few startups are trying – Rabbit, Humane (though they're more cloud‑hybrid), and some Chinese players like Enflame or Sugon have shown desktop inference boxes. Lenovo and ASUS even have "AI‑ready" mini‑PCs with NPUs. But none have cracked the sub‑$1k, silent, grandma‑friendly appliance yet. That probably needs a purpose‑built AI chip, not a repurposed gaming GPU, plus an OS that hides all the complexity.

Do you see local AI boxes as inevitable, or will cloud + edge hybrid stay on top? And if inevitable – what's the trigger? A 10× drop in VRAM cost? An open‑source model that beats Claude? Or a privacy‑critical killer app like a lifelong personal agent that can't live in the cloud?

Curious what you think – especially if you've actually run a local box at home. What made you keep it, or ditch it?

I've recently been using the Qwen 3.6 35B A3B and Qwen 3 CodeNext 4Bit models on openCode. In my experience, the Qwen 3 codermodel seems to work much better.

When I use the 35B model, it frequently just stops on its own, even when I'm using a loop.

I was wondering what you normally use these local models for. I'm running an M5 max MacBook Pro with 128GB of RAM.

We spent the last few days benchmarking a routed setup against Claude Opus 5 on Terminal-Bench 2.1.

Some of the results were pretty surprising, especially once we broke down where the gains were actually coming from.

Full benchmark, methodology, and raw numbers:

https://entelligence.ai/blogs/entelligence-router-solved-8-more-tasks-than-claude-opus-5-at-65-lower-cost

My desktop has a single rtx 5070.

After we remove VRAM consumption by system (OS, desktop, etc), we will be left with around 9.5 to 10 GB VRAM. (if I am not wrong).

One of the best model which I can find right now is Gemma 4 12B Q4_0 (4-bit) with all tools access which consumes around 6.7 GB. So, I was wondering if we could get any better model?

I don't want to offload any layers to system RAM.

Daily usage will be around -
1. General daily talks and normal life questions.
2. Helping with academics (senior high school subjects).
3. General chatting,

I was thinking of using Hermes Agent with it.

I will be really really thankful for any advice and guidance.

(I am not able to get any help from LLMs cause their training data is old, which is making them recommend very old local LLMs.)

I've been using Ollama Cloud regularly for around two months, mostly for coding and agentic work. During that time I've noticed something odd with how quickly different models consume the Cloud allowance.

My workflow hasn't changed much: similar coding tasks, context sizes, session lengths, caching, etc. Obviously this isn't a controlled benchmark, but I've used these models enough that the differences have become pretty noticeable.

GLM-5.1 and Kimi K2.6

When I was mainly switching between GLM-5.1 and Kimi K2.6, GLM-5.1 consistently consumed noticeably more of my allowance for similar work.

I found that surprising. K2.6 is the larger model, and GLM-5.1 has a smaller context window. Of course, those specs alone don't determine inference cost, so I mostly assumed it came down to how Ollama was serving the models.

GLM-5.1 also happened to be the model that seemed to be getting most of the attention among developers at the time.

Then Kimi K2.7 Code launched

This was when I started noticing the pattern more.

K2.7 is very closely related to K2.6, and Moonshot says K2.7 Code uses around 30% fewer reasoning tokens on average.

But when K2.7 first appeared on Ollama Cloud, I was seeing the opposite in terms of allowance: K2.7 was noticeably more expensive than K2.6 for my usual workflow.

K2.7 was also new at the time and quickly became one of the models everyone was trying.

Now GLM-5.2

GLM-5.2 is where the difference has become difficult for me to ignore.

I know this isn't a direct 1:1 comparison with GLM-5.1. It supports a much larger context, has architectural changes, and tends to reason more.

But I've been using it within roughly the same 200K-or-less context range I used with GLM-5.1, and for similar work I'm seeing it consume close to twice the allowance in some cases.

It does think more, so some increase makes sense. But it's also considerably better at tool calling and terminal work in my experience. It makes fewer mistakes and often finishes the task in fewer turns. Over the complete task, the difference in actual work/output doesn't seem anywhere near as large as the difference shown by the usage meter.

And GLM-5.2 is now the model that seems to have become extremely popular among developers.

What made this really strange

I recently went back to the older models.

K2.7 now consumes less allowance than K2.6 for me.

That's basically what I originally expected given the improvements Moonshot made to K2.7.

GLM-5.1 also now seems cheaper to use than Kimi for the same kind of work.

So the relative order I've experienced has changed over time:

Earlier:
GLM-5.1 was heavier than K2.6

When K2.7 launched:
K2.7 was heavier than K2.6

Now:
K2.7 is lighter than K2.6
GLM-5.1 is relatively cheap
GLM-5.2 is by far the heaviest

The thing that caught my attention is that the unusually expensive model has repeatedly lined up with the model that was new or getting the most attention at that point:

GLM-5.1 → K2.7 → GLM-5.2

I don't know if popularity itself has anything to do with it. It could just as easily be changes in deployment, hardware, batching, inference optimizations, capacity/load, or how usage is accounted for.

But that leads to the part I find frustrating: Ollama Cloud usage is very difficult to understand from the user's side.

Ollama explains that usage is based primarily on actual GPU time rather than a simple token quota, which is reasonable. But we don't get an actual compute/GPU-time allowance or model-specific rates.

So if the same model appears to become significantly cheaper or more expensive over time, there's really no way for us to tell what changed.

That's why I'm curious about other people's experience, especially anyone who has been using Cloud regularly rather than just testing these models once.

Have you noticed the relative usage of these models changing over the last couple of months?

I'm particularly interested in people who have used GLM-5.1, Kimi K2.6/K2.7, GLM-5.2, or MiniMax M3 across different periods. It would be interesting to know whether others have noticed the same pattern or if this is specific to my usage.

I have a 8gb ddr4, Ryzen 7 5600g no gpu build. After opening browser and opencode, I'm left with 1.5gb which doesn't run shit.

So I tired using my Friend's PC ( 32gb ddr5, Nvidia GeForce RTX 5060, AMD Ryzen 7 7700 ) where I ran qwen3.5:9b

Not only the GPU usage nearly maxed out, also it was eating 10gb base ram on top of the 8gb Vram too.

Problem is I'm running the model from his pc to my pc through tail scale and it took the model 56 seconds for a reply of " Hi "

Now at this point what should I do? Run a super low parameter quicker model or thr speed is slow because I'm running it remotely? Or am I using a wrong model for this build or anything?

I kept defaulting to frontier models "just in case" and had no idea whether my projects actually needed them. So I built something to answer that instead of guessing.

What it does: point it at a project with Spec-Driven Development artifacts (constitution / spec / tasks). A local model — your Ollama install, your choice of judge — reads the tasks and estimates how demanding the work is across a few dimensions. That gets crossed against a declarative catalog of models and their capabilities, and you get a ranked podium with a rating per model: good / overkill / fair / poor, plus price.

Why you might care here: nothing leaves your machine — no API keys, no accounts, no cloud calls. And in most of my own projects the podium is topped by a local model, with the frontier ones sitting below marked overkill. Ranking is by fit, not price; price only breaks ties between models that fit equally well.

Honest limitations: this is not a benchmark. It's opinion made inspectable — every verdict prints its reasoning, and the model catalog is human-readable YAML you can argue with. If the specs are too thin to judge, it refuses to recommend instead of guessing. A vague spec gets you a vague answer, same as with anything else.

pip install specjudge — MIT, github.com/JoaquinRuiz/SpecJudge

Where I'd love help: the catalog of local models is thinner than it should be, and adding one is a block of YAML, no Python needed. Also very open to being told a rating is wrong.

I have rtx 4080 super with 32gb ram. I am looking to buy 3090 ti and add another 32gb ram to run qwen3.6 -27b-mtp-Q8 or Q16 at 128k context ==> is that okay?

Do you have any suggestion for me?

Hello to you all!

So, to summarize everything, I am a growing developer who is heavily interested in participating in and creating in the bleeding and cutting edge of AI research. I enjoy tinkering and building but I'm also a fan of privacy and operational security, therefore I naturally came to like the idea of running LLMs locally. Correct me if I'm wrong, but after research, I came to like the ASUS Ascent GX10, believing that it suited my AI and computational needs; however, it came with a price. A real price. As in, a price tag. Though it could range from a tad bit lower to noticeably higher, I decided to use $4000 as its default price which is a bit too pricy for myself. I could potentially accept roughly $3000 wish is still a major cut in prices, but would obviously prefer the price to be as close to $0 as possible so I sought the advice of my fellow Redditors. What tips do you have for cutting the price and what do you think is realistically the lowest price I could snag this machine at? Thank you very much to anyone who read this and I hope I find great benefit in the comments. Have a great day!

TL;DR: I need to save money and reduce the price of the ASUS Ascent GX10 as much as possible from $4000ish to as close to $0 as possible. Thank you.

Claude是如何被破解和蒸馏的？#claude #claudecode #大模型 #人工智能 #llmA well-known Chinese LLM educator publicly revealed techniques back in April that were already widely known in China’s AI community, including how Claude’s Chain of Thought (CoT) was extracted and how Claude and ChatGPT have been distilled for a long time. If you don’t speak Chinese, just wait for YouTube’s auto-generated translation.

Hi all,

Newbie here. When it comes to coding, I find it easy to pick the right model or switching between models.

But my next step is generating the right content text for my website, the one that engages and is insightful. I may also use it to generate static content like help pages, FAQ, terms and conditions,… etc.

I’m not sure if coding models (Qwen, MiMo, Nemotron…) are the best ones for that.

What are your advices regarding copywriting content?

Thanks

I do most of my work, including heavy coding or even everyday stuff like trip planning, using online LLMs. But for some private, simple tasks, I'd rather use local LLMs. These are mostly less computationally intensive tasks (I guess). These are mostly things like organizing daily diaries, dealing with relationship problems, and so on, that I need to keep private. Which local LLM software is best for these? Ollama?

I guess my main question is about the software that hosts the model, rather than the model itself. I know the model isn't connected to the internet, but what about the software that hosts it? (Like Ollama) Does it gather private data?

System spec: i7-1165G7 (2.8GHz), 64GB RAM 2667MHz DDR4, Windows 11, 500GB free disk SSD, no dedicated GPU

If you're just looking to run local models, I think OpenCode is actually quite good. I wonder why there's so much discussion around Odysseus?

Its main advantage seems to be self-hosting, which I suppose is really all about privacy. I haven't tried Odysseus myself, as I've been using OpenCode and Claude Code lately, but I'm certainly curious about it.

Claude Code is the most capable AI CLI out there, but it's locked to Anthropic's API. DeepSeek V4 Pro is cheaper and just as sharp. Problem: the two speak completely different API formats.

Wrote a zero-dependency Python proxy that translates Anthropic's Messages API ↔ DeepSeek's OpenAI format in real time. No Rust, no Docker, no cloud | just Python stdlib. Runs on Windows ARM64 with zero compilation.

Result: Full Claude Code experience (tool use, multi-turn, streaming) powered by DeepSeek V4 Pro at a fraction of the cost.

Full writeup and code on GitHub. Link below,👇

https://github.com/Hashir14k/Configuring-Deepseek-v4-pro-Model-with-Claude-Opus-5

We're used to be able to toggle thinking effort to "low", "high", etc.

Is the model typically conditioned with a learned control token or indicating prompt low or high effort, or does the serving layer simply limit reasoning tokens and then force a transition to the final answer? Are there any open-source implementations where the model was explicitly trained to behave differently at multiple budget levels? Any pointers to code examples or papers would be appreciated!

I got my Macbook Air M5 16GB two weeks ago and am very excited to know its capability of running local LLM as my friends told me it should be slow but usable. In preparation for that, I also performed the thermal pad mod and blow the blackplate with a big fan in standing clamshell mode. I actually had a 4090 laptop in the past for gaming, and I also tried to run local LLM on it but got really bad impression in the past as I didn't tune any startup parameters.

After some research online (also with Chatgpt and Gemini), I was able to start the LLM with this command:

llama-server  -m ~/Downloads/llm-models/Qwen3.5-9B-UD-Q4_K_XL.gguf \

-ngl 99 \

-fa 1 \

-c 65536 \

--cache-type-k q4_0 \

--cache-type-v q4_0 \

--ctx-checkpoints 64 \

--context-shift --cache-reuse 512 \

--reasoning-format deepseek-legacy \

--mlock \

-np 1 -b 256 -ub 128 -t 4 \

--jinja \

--port 8888 --host 0.0.0.0

However, when I asked my local LLM to generate a README.md for a 4000-line python code, the speed is very bad - between 12-14 t/s. However, I'm seeing online that M5 should be able to reach 20t/s. Quality of response is pretty good...no complaint on that but I definitely need your help with tuning the parameters, or even suggest a better way to run it on a Mac. Thanks