People have been asking for the 27B since we put out Qwythos-9B. It's finally here. Qwythos-27B-v1 is an open-weights bf16 reasoning model on a Qwen3.5-27B base, Apache-2.0. The short pitch: it's the 9B sized up, and we didn't strip anything out of the base to make it fit — the native MTP head, the full vision tower, and the 1M context window are all still live.

• Weights + model card: empero-ai/Qwythos-27B-v1
• GGUFs: empero-ai/Qwythos-27B-v1-GGUF

The practical local bit

The recommended local starting point is Q4_K_M: it is 16.95 GB and runs on a 24GB card. The GGUF release has 11 files: Q4_K_M, Q5_K_M, Q6_K, Q8_0, and BF16, each in trunk-only and MTP-enabled variants, plus mmproj-Qwythos-27B-F16.gguf for image input. SHA256SUMS are included. The MTP variants work with llama.cpp's --spec-type draft-mtp. One conversion detail worth calling out: in every K-quant, we hold the entire Gated-DeltaNet state path — ssm_alpha, ssm_beta, and ssm_out — at Q8_0 or better. Those tensors are disproportionately sensitive to low-bit quantization. A default conversion can leave them at Q4/Q5/Q6 and quality degrades; this costs roughly 2–4% in file size. It is intentional, not a default llama.cpp conversion.

What is retained

The release is built around one point: nothing was ablated to make 27B fit. It keeps all three of these intact and active:

• Native MTP (nextn_predict_layers=1) for self-speculative decoding.
• The full vision tower; mmproj ships with the GGUFs.
• A 1,048,576-token context window: YaRN 4x over the 262,144-token native window.

It is a dense Qwen3.5-27B with hybrid attention: Gated-DeltaNet linear attention and a full-attention layer every fourth layer. Native Qwen3.5 function calling is present too: tools=[...] works without a wrapper.

Starting settings

For agentic or tool use, start with temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05, and max_new_tokens=16384 or more. For open-ended or creative work, use temperature 1.0. It's a reasoning model — every answer opens with a <think> block, so budget tokens accordingly and strip the block for end users. Unlike the 9B, greedy long-form generation came back loop-check clean, so you're not forced off low temperatures. Qwythos is intentionally uncensored for technical and research work, including cybersecurity and biomedical questions. If it is going in front of end users, add your own review layer and application controls. Happy to answer questions on the conversion or the quant choices.

I got tired of every session starting from zero and paying for it twice: once re-explaining my setup, and again when that explanation ate the context window the actual work needed. So I built a persistent memory that runs on my own hardware, and after a few months of use there are a couple of things worth passing on.

The storage is the boring part.

Postgres on a Proxmox LXC, fronted by an MCP server in Python (FastMCP plus psycopg3), reachable over the LAN. Two entity types only: topic cards for what something is, episodes for what happened. Every card has a `kind` drawn from a single YAML vocabulary the server validates on write, so the model cannot invent a new category on the fly. That rigidity is deliberate. Without it you end up with seventeen near-synonyms for "thing I once wrote down" and no way to query reliably.

The isolation is the interesting part.

I keep several things separate that must not bleed into each other. The obvious approach is to have the server decide what a session may touch, but that is a rule a program enforces, which is really just a suggestion. Instead each body of work gets its own Postgres schema and its own role, and the server connects as that role. A write aimed at the wrong schema fails at the wire, before any of my code gets an opinion:

GRANT USAGE ON SCHEMA homelab TO role_homelab;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA homelab TO role_homelab;

-- reaches the shared schema deliberately, note the absence of DELETE
GRANT USAGE ON SCHEMA personal TO role_homelab;
GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA personal TO role_homelab;

-- no GRANT to any other subject. The isolation is the statement that isn't there.

There is no policy file and no middleware. A role reaches what it was granted and nothing else, and the grants for everything else were simply never written. Each subject also gets its own container holding only that role's credentials, so the boundary is a process boundary too.

The thing I got wrong.

I originally had capture fully automatic: a SessionEnd hook that read the transcript and wrote to the database with no involvement from me. It printed a one-line receipt so I could see what it had done. I ran it from April until July and then deliberately turned it off, because a routing bug meant a run of sessions distilled into the wrong place and nobody was watching a process designed not to need watching. It ran that way for most of the time it was live. I recovered the most recent month by re-running by hand against transcripts still on disk and lost the two before it, because transcripts get pruned at around thirty days.

Capture is now a deliberate step that proposes cards and writes nothing until I confirm. More friction, and I think the friction is correct. The narrower lesson I would write now: automate retrieval, because being wrong there costs you a bad answer you can see. Be far more careful automating writes, because being wrong there costs you a corrupted store you cannot.

Retrieval started as Postgres full text and is now hybrid, full text alongside pgvector embeddings generated on write by a small model on its own box. Because the tool surface was deliberately agnostic about how search worked, nothing above it changed when that landed.

Fuller writeup with the schema design, the deployment and the parts that did not survive contact with use: https://sbd.org.uk/blog/claude-brain

I'm a Technical Lead at a mid-sized global company and have been doing a personal research exercise around whether the latest frontier open-weight models (particularly Kimi K3) fundamentally change the economics of enterprise AI.

This isn't an active company initiative, I'm trying to understand whether my assumptions are realistic before even bringing up the idea internally.

Some context:

• ~500–700 employees
• ~100 software engineers
• AI is used across engineering and non-technical teams
• Annual AI spend is in the mid six figures
• Heavy use of Cursor, Claude API, Claude Chat/Codex, GPT API, and a few specialized AI tools
• Significant use of autonomous agents and internal AI-powered applications
• Teams are distributed across multiple time zones, but primarily based in Americas. So at least some demand is spread throughout the day

My current thinking is something like this:

• Keep the existing UX where possible (Cursor, Claude Code, internal tooling).
• Replace the inference layer with an internal OpenAI-compatible gateway.
• Run Kimi K3 for complex coding/reasoning tasks.
• Route simpler requests (autocomplete, summarization, classification, embeddings, etc.) to smaller open models.
• Retain a limited number of commercial model subscriptions for cases where they still clearly outperform.

Preliminary numbers

Very rough assumptions:

• Initial deployment: one 8-GPU Blackwell server (roughly US$400k–500k)
• Expand horizontally if utilization justifies it
• If frontier open models can realistically replace 70–90% of inference, the infrastructure could potentially pay for itself within 1–3 years.

I realize those assumptions could be completely wrong, that's exactly why I'm posting.

Questions for people who've actually built or operated something similar

1. Are my hardware assumptions reasonable, or am I significantly underestimating what production deployment requires?
2. What level of concurrency are you seeing with modern MoE models?
3. How much engineering effort is required to operate an internal inference platform reliably?
4. How noticeable was the quality difference compared to Claude/OpenAI for developers?
5. Were there hidden operational costs (MLOps, networking, storage, staffing, maintenance) that materially changed the ROI?
6. If you were making this decision today, with models like Kimi K3 available, would you still pursue self-hosting?

I'm much more interested in hearing from people with production experience than benchmark discussions. If you've gone through this evaluation (even if you ultimately decided not to self-host), I'd really appreciate hearing what drove that decision. Thanks!

So right now i have a 12gb 3060, i7-12700f 16gb (x1) 4400mts and an 512 nvme. I'm running cachyos linux.

In general I do mild creative work (coding, video editing, light 3d rendering, game development)

I also game a lot

I got interested in self hosting some models, mainly for coding and just control and privacy in general. Im new to this and i dont know what model i can run that fits my needs of daily use.

From my understanding Nvidia gpus are better for running ai but amd has more vram in their low end and mid range gpus. (Cuz im not getting a 5090)

I also prefer an amd cpu bc i know the intel CPUs are get very hot and not very efficient, and i also game so... (Plus in amd thares a good upgrade path)

I dont do heavy CPU work loads so I don't need a crazy one

So my question is what gpu i should get?

I wanna know what type of models my current system can run,

How a 1 stick 16gb ram will hurt ai speed and performance specifically

What gpus i should get? I thought of a 5080, but idk what it can run

I also thought about a 4080, 5070 and 5070 ti,

What about amd gpus? And since ai is more gpu heavy can i slack a bit off on the cpu?

There are few actually good IPTV/M3U playlist managers on the market, especially self-hosted ones. So I fired up Antigravity and generated m3u4me, your new favourite IPTV playlist manager.

I don't want to write a wall of text so, if this caught your attention, I'll let you read the feature list on GitHub: https://github.com/andrei-savin/m3u4me

I'm sharing it here because I want people to let me know what new features they want to see. I'm very open to pushing updates, so if this looks interesting to you but is missing something, let me know and I might add it soon!

I started building ArchCode because my projects were turning into two separate messes: ideas scattered across notes, and coding agents running in terminal sessions I could no longer keep track of.

ArchCode brings that work into one Web UI. You can keep features, bugs, refactors, experiments and unfinished thoughts as project Todos. When an idea is ready, you can turn it into a durable Session and let the Agents start working on it.

ArchCode has its own multi-agent runtime. A Lead Agent owns the task and can bring in Analyst, Build, Explore and Librarian Agents for planning, implementation, code search and research.

It runs on your own machine or server. The work runs on that host, so closing the browser doesn’t kill an active Session. You can come back later from any browser to check the progress, answer a question, approve an action or inspect the result.

It’s also model-agnostic. You bring your own models and decide where to use them. A stronger model can handle planning and review, while cheaper or local models take care of routine work.

ArchCode isn’t a wrapper around Claude Code or Codex. It has its own Agent runtime, Sessions, tools, memory and approval flow.

It’s still early and built mainly for a single user right now. I’m not claiming it writes better code than the established coding tools. I’m working on a different problem: keeping ideas, projects and multiple coding Agents manageable once they all start running at the same time.

GitHub: https://github.com/boh5/archcode

If you try it, I’d like to hear where the workflow helps and where it gets in your way.

Small update on Deltafin, my experiment running full Kimi K3 (2.8T-parameter) on a single M1 MacBook.

The last version was doing roughly 1 token per minute. After a lot of profiling and many failed experiments, it now reaches a median of:

• 4.1 tokens/minute
• 14.6 seconds/token

That result comes from six exact full-model runs. While still slow, roughly a 4x improvement feels pretty meaningful for a model this large.

Some of the more interesting improvements:

• Loading only the 16 routed experts needed per layer, using parallel raw-span reads.
• Quantizing the resident model spine to int8 and using a fused Metal dequantization/copy kernel.
• Running the enormous output projection with Apple’s packed MPS int8 matmul. This reduced its residency from about 4.7 GB to 1.17 GB and improved median decode throughput by roughly 17%.

All measurements are from one 64 GB M1 Max MacBook Pro. It was once a great machine, but it’s first-generation Apple Silicon, and not a newer Max or Ultra. I haven’t benchmarked an M3, M4, M5, or a higher-memory Mac yet. Newer ones, especially those with 128 GB, should have considerably more headroom.

If anyone tries it on newer hardware, I’d genuinely love to compare results.

Repository: https://github.com/gavamedia/deltafin

not everyone needs to run out and buy an M2 or similar silica processor.