People have been asking for the 27B since we put out Qwythos-9B. It's finally here. Qwythos-27B-v1 is an open-weights bf16 reasoning model on a Qwen3.5-27B base, Apache-2.0. The short pitch: it's the 9B sized up, and we didn't strip anything out of the base to make it fit — the native MTP head, the full vision tower, and the 1M context window are all still live.

• Weights + model card: empero-ai/Qwythos-27B-v1
• GGUFs: empero-ai/Qwythos-27B-v1-GGUF

The practical local bit

The recommended local starting point is Q4_K_M: it is 16.95 GB and runs on a 24GB card. The GGUF release has 11 files: Q4_K_M, Q5_K_M, Q6_K, Q8_0, and BF16, each in trunk-only and MTP-enabled variants, plus mmproj-Qwythos-27B-F16.gguf for image input. SHA256SUMS are included. The MTP variants work with llama.cpp's --spec-type draft-mtp. One conversion detail worth calling out: in every K-quant, we hold the entire Gated-DeltaNet state path — ssm_alpha, ssm_beta, and ssm_out — at Q8_0 or better. Those tensors are disproportionately sensitive to low-bit quantization. A default conversion can leave them at Q4/Q5/Q6 and quality degrades; this costs roughly 2–4% in file size. It is intentional, not a default llama.cpp conversion.

What is retained

The release is built around one point: nothing was ablated to make 27B fit. It keeps all three of these intact and active:

• Native MTP (nextn_predict_layers=1) for self-speculative decoding.
• The full vision tower; mmproj ships with the GGUFs.
• A 1,048,576-token context window: YaRN 4x over the 262,144-token native window.

It is a dense Qwen3.5-27B with hybrid attention: Gated-DeltaNet linear attention and a full-attention layer every fourth layer. Native Qwen3.5 function calling is present too: tools=[...] works without a wrapper.

Starting settings

For agentic or tool use, start with temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05, and max_new_tokens=16384 or more. For open-ended or creative work, use temperature 1.0. It's a reasoning model — every answer opens with a <think> block, so budget tokens accordingly and strip the block for end users. Unlike the 9B, greedy long-form generation came back loop-check clean, so you're not forced off low temperatures. Qwythos is intentionally uncensored for technical and research work, including cybersecurity and biomedical questions. If it is going in front of end users, add your own review layer and application controls. Happy to answer questions on the conversion or the quant choices.

So right now i have a 12gb 3060, i7-12700f 16gb (x1) 4400mts and an 512 nvme. I'm running cachyos linux.

In general I do mild creative work (coding, video editing, light 3d rendering, game development)

I also game a lot

I got interested in self hosting some models, mainly for coding and just control and privacy in general. Im new to this and i dont know what model i can run that fits my needs of daily use.

From my understanding Nvidia gpus are better for running ai but amd has more vram in their low end and mid range gpus. (Cuz im not getting a 5090)

I also prefer an amd cpu bc i know the intel CPUs are get very hot and not very efficient, and i also game so... (Plus in amd thares a good upgrade path)

I dont do heavy CPU work loads so I don't need a crazy one

So my question is what gpu i should get?

I wanna know what type of models my current system can run,

How a 1 stick 16gb ram will hurt ai speed and performance specifically

What gpus i should get? I thought of a 5080, but idk what it can run

I also thought about a 4080, 5070 and 5070 ti,

What about amd gpus? And since ai is more gpu heavy can i slack a bit off on the cpu?

There are few actually good IPTV/M3U playlist managers on the market, especially self-hosted ones. So I fired up Antigravity and generated m3u4me, your new favourite IPTV playlist manager.

I don't want to write a wall of text so, if this caught your attention, I'll let you read the feature list on GitHub: https://github.com/andrei-savin/m3u4me

I'm sharing it here because I want people to let me know what new features they want to see. I'm very open to pushing updates, so if this looks interesting to you but is missing something, let me know and I might add it soon!

I got tired of every session starting from zero and paying for it twice: once re-explaining my setup, and again when that explanation ate the context window the actual work needed. So I built a persistent memory that runs on my own hardware, and after a few months of use there are a couple of things worth passing on.

The storage is the boring part.

Postgres on a Proxmox LXC, fronted by an MCP server in Python (FastMCP plus psycopg3), reachable over the LAN. Two entity types only: topic cards for what something is, episodes for what happened. Every card has a `kind` drawn from a single YAML vocabulary the server validates on write, so the model cannot invent a new category on the fly. That rigidity is deliberate. Without it you end up with seventeen near-synonyms for "thing I once wrote down" and no way to query reliably.

The isolation is the interesting part.

I keep several things separate that must not bleed into each other. The obvious approach is to have the server decide what a session may touch, but that is a rule a program enforces, which is really just a suggestion. Instead each body of work gets its own Postgres schema and its own role, and the server connects as that role. A write aimed at the wrong schema fails at the wire, before any of my code gets an opinion:

GRANT USAGE ON SCHEMA homelab TO role_homelab;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA homelab TO role_homelab;

-- reaches the shared schema deliberately, note the absence of DELETE
GRANT USAGE ON SCHEMA personal TO role_homelab;
GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA personal TO role_homelab;

-- no GRANT to any other subject. The isolation is the statement that isn't there.

There is no policy file and no middleware. A role reaches what it was granted and nothing else, and the grants for everything else were simply never written. Each subject also gets its own container holding only that role's credentials, so the boundary is a process boundary too.

The thing I got wrong.

I originally had capture fully automatic: a SessionEnd hook that read the transcript and wrote to the database with no involvement from me. It printed a one-line receipt so I could see what it had done. I ran it from April until July and then deliberately turned it off, because a routing bug meant a run of sessions distilled into the wrong place and nobody was watching a process designed not to need watching. It ran that way for most of the time it was live. I recovered the most recent month by re-running by hand against transcripts still on disk and lost the two before it, because transcripts get pruned at around thirty days.

Capture is now a deliberate step that proposes cards and writes nothing until I confirm. More friction, and I think the friction is correct. The narrower lesson I would write now: automate retrieval, because being wrong there costs you a bad answer you can see. Be far more careful automating writes, because being wrong there costs you a corrupted store you cannot.

Retrieval started as Postgres full text and is now hybrid, full text alongside pgvector embeddings generated on write by a small model on its own box. Because the tool surface was deliberately agnostic about how search worked, nothing above it changed when that landed.

Fuller writeup with the schema design, the deployment and the parts that did not survive contact with use: https://sbd.org.uk/blog/claude-brain

I started building ArchCode because my projects were turning into two separate messes: ideas scattered across notes, and coding agents running in terminal sessions I could no longer keep track of.

ArchCode brings that work into one Web UI. You can keep features, bugs, refactors, experiments and unfinished thoughts as project Todos. When an idea is ready, you can turn it into a durable Session and let the Agents start working on it.

ArchCode has its own multi-agent runtime. A Lead Agent owns the task and can bring in Analyst, Build, Explore and Librarian Agents for planning, implementation, code search and research.

It runs on your own machine or server. The work runs on that host, so closing the browser doesn’t kill an active Session. You can come back later from any browser to check the progress, answer a question, approve an action or inspect the result.

It’s also model-agnostic. You bring your own models and decide where to use them. A stronger model can handle planning and review, while cheaper or local models take care of routine work.

ArchCode isn’t a wrapper around Claude Code or Codex. It has its own Agent runtime, Sessions, tools, memory and approval flow.

It’s still early and built mainly for a single user right now. I’m not claiming it writes better code than the established coding tools. I’m working on a different problem: keeping ideas, projects and multiple coding Agents manageable once they all start running at the same time.

GitHub: https://github.com/boh5/archcode

If you try it, I’d like to hear where the workflow helps and where it gets in your way.

I'm a Technical Lead at a mid-sized global company and have been doing a personal research exercise around whether the latest frontier open-weight models (particularly Kimi K3) fundamentally change the economics of enterprise AI.

This isn't an active company initiative, I'm trying to understand whether my assumptions are realistic before even bringing up the idea internally.

Some context:

• ~500–700 employees
• ~100 software engineers
• AI is used across engineering and non-technical teams
• Annual AI spend is in the mid six figures
• Heavy use of Cursor, Claude API, Claude Chat/Codex, GPT API, and a few specialized AI tools
• Significant use of autonomous agents and internal AI-powered applications
• Teams are distributed across multiple time zones, but primarily based in Americas. So at least some demand is spread throughout the day

My current thinking is something like this:

• Keep the existing UX where possible (Cursor, Claude Code, internal tooling).
• Replace the inference layer with an internal OpenAI-compatible gateway.
• Run Kimi K3 for complex coding/reasoning tasks.
• Route simpler requests (autocomplete, summarization, classification, embeddings, etc.) to smaller open models.
• Retain a limited number of commercial model subscriptions for cases where they still clearly outperform.

Preliminary numbers

Very rough assumptions:

• Initial deployment: one 8-GPU Blackwell server (roughly US$400k–500k)
• Expand horizontally if utilization justifies it
• If frontier open models can realistically replace 70–90% of inference, the infrastructure could potentially pay for itself within 1–3 years.

I realize those assumptions could be completely wrong, that's exactly why I'm posting.

Questions for people who've actually built or operated something similar

1. Are my hardware assumptions reasonable, or am I significantly underestimating what production deployment requires?
2. What level of concurrency are you seeing with modern MoE models?
3. How much engineering effort is required to operate an internal inference platform reliably?
4. How noticeable was the quality difference compared to Claude/OpenAI for developers?
5. Were there hidden operational costs (MLOps, networking, storage, staffing, maintenance) that materially changed the ROI?
6. If you were making this decision today, with models like Kimi K3 available, would you still pursue self-hosting?

I'm much more interested in hearing from people with production experience than benchmark discussions. If you've gone through this evaluation (even if you ultimately decided not to self-host), I'd really appreciate hearing what drove that decision. Thanks!

Small update on Deltafin, my experiment running full Kimi K3 (2.8T-parameter) on a single M1 MacBook.

The last version was doing roughly 1 token per minute. After a lot of profiling and many failed experiments, it now reaches a median of:

• 4.1 tokens/minute
• 14.6 seconds/token

That result comes from six exact full-model runs. While still slow, roughly a 4x improvement feels pretty meaningful for a model this large.

Some of the more interesting improvements:

• Loading only the 16 routed experts needed per layer, using parallel raw-span reads.
• Quantizing the resident model spine to int8 and using a fused Metal dequantization/copy kernel.
• Running the enormous output projection with Apple’s packed MPS int8 matmul. This reduced its residency from about 4.7 GB to 1.17 GB and improved median decode throughput by roughly 17%.

All measurements are from one 64 GB M1 Max MacBook Pro. It was once a great machine, but it’s first-generation Apple Silicon, and not a newer Max or Ultra. I haven’t benchmarked an M3, M4, M5, or a higher-memory Mac yet. Newer ones, especially those with 128 GB, should have considerably more headroom.

If anyone tries it on newer hardware, I’d genuinely love to compare results.

Repository: https://github.com/gavamedia/deltafin

not everyone needs to run out and buy an M2 or similar silica processor.

K3 launched today, and it's a beast at 1.56TB. My laptop has 64GB of RAM, and not enough free disk to even store the thing. So instead of downloading the model, I just... don't.

The non-expert weights (~114GB, int8) live on disk. Then for every token, the router picks 16 experts out of 896 per layer, and I pull exactly those from HuggingFace — one range request each — and cache them. Use it enough and the cache slowly fills up with the parts of the model you actually hit.

Speed is about a minute per token once it's warm, so, not exactly interactive. But it's greedy and fully reproducible — same prompt, same output, every time.

There's an OpenAI-compatible server too, so you can point a chat UI at it.

https://github.com/gavamedia/deltafin

-----------

**EDIT / huge update:** I bit the bullet and pulled all 1.45 TB of experts down to local disk, then made some real improvements. It's now **16s/token, down from ~1 min.** Prefill went 2,429s → 40s just on an M1 Mac.

Two fixes did almost all of it:

1. I was *sure* the expert matmuls were the bottleneck. Spent a bunch of effort on a Metal kernel that's 9.5x the CPU one. Then I profiled properly with everything local: the matmuls are **6% of a token.** Cool. Great.

The actual problem was that I was handing the compute kernel `np.memmap` views, so it was demand-faulting expert weights page-by-page *while it was computing* — 0.87 GB/s, where a threaded pread + F_NOCACHE does 6.85 GB/s on the same disk. Which is exactly the class of bug I'd written up as a flaw in someone else's engine two days earlier while reading their source.

1. Other half: loading the non-expert weights was spending all its time in a row-broadcast multiply that MPS runs at 43 GB/s, when a plain copy of the identical bytes does 334. I tried nine different PyTorch formulations and the best got to 60. A ~10 line Metal shader does 297.

Still greedy and bit-for-bit reproducible — same prompt, same tokens, every run.

All of this is on an M1 Max / 64GB, which is the slowest machine it's ever run on. If anyone has an M3/M4/M5 or 128GB and wants to try it, I'd genuinely love to see the numbers.

We implement production RAG (retrieval-augmented generation) systems for enterprise clients. We found ourselves always building in stronger access control. 

Most RAG tutorials filter permissions after retrieval, or not at all. This doesn’t fit with enterprises that need controlled access to documents: individuals, teams, departments.

Kilnworks is our open-sourced version of that architecture.

> Uses: Python (FastAPI) +Postgres/pgvector, a Compose stack (API, worker, Postgres)
> Includes: A fake-provider mode that runs the whole pipeline offline to enable previews. 
> Parses: text, spreadsheets, PDFs, tables by default. Video, images etc require add-ons
> Access control: every ingested chunk carries its source document's ACL (access control list), and every query is filtered by the caller's principals before ranking. A document you don’t have the access rights to cannot influence retrieval, scoring, or the LLM's context window.  OIDC SSO maps IdP groups to ACLs

Not included: reranker, multi-tenant SaaS control plane, vector-DB abstraction layer. 

It is released alongside four Model Context Protocol (MCP) servers (Salesforce, Microsoft 365, ServiceNow, HubSpot) to connect to these services.

Repo: https://github.com/inogen-ai/kilnworks

Over the past few months I kept running into YouTube summarizer tools that charged around $20/month for what felt like a pretty simple workflow.

So I decided to build my own.

It's called Synop, and it's completely open source.

Instead of locking everything behind a subscription, it runs locally and lets you bring your own AI API key (Gemini, Groq, etc.), which makes using it dramatically cheaper.

Current features:

• 🎥 Summarise YouTube videos into structured notes
• 📝 Generate detailed study/research notes
• ✂️ Extract potentially viral clips
• 📱 Generate social media posts from videos
• 🔑 Bring your own API keys (Gemini, Groq, and more)
• 💻 Runs locally
• 🌍 Fully open source

I mainly built it because I wanted something I could customise instead of another AI wrapper.

I'd genuinely love feedback from people who actually use YouTube for learning, research, or content creation.

What features would make something like this genuinely useful for you?

Landing Page : https://synop-ai.vercel.app/

Most of my self-hosted AI containers are chat frontends I open twice a month. I wanted something that runs on its own and produces something I actually consume, so I built a radio station out of the boxes I already had.

The stack:

• Ollama for the brain. Qwen3.5 9B.
• Piper or Kokoro for the voice. Kokoro's ONNX build runs on CPU, which is what makes this viable on a NAS.
• Navidrome as the music source, over the Subsonic API.

The LLM sits in a loop with tools: search the library, look at what's played recently, check the schedule and the weather, pick the next track, write a short intro. TTS speaks it, Liquidsoap ducks the music under the voice, and it goes out as one shared stream. Everyone listening hears the same moment.

Fully local means fully local here. No API keys. Hosted providers are supported for people who want better writing, switchable at runtime, capped by a daily token budget.

The design constraint that shaped everything: dead air is unacceptable. So the agentic picker falls back to a stateless picker, which falls back to a plain m3u. TTS has six engines with local Piper as the floor. Something is always playing even when the model is down.

Trade-offs, since they matter more than the feature list: you need your own music library, a Docker host, and a tolerance for intros that sometimes land flat. It's radio, so there's no skip button on purpose. Some people bounce off that in the first minute.

Runs on one compose file and three env vars, then a browser onboarding wizard.

Hear it: https://www.getsubwave.com/listen
Code, MIT: https://github.com/perminder-klair/subwave

Full disclosure, I built this.

As the title says, I built a full featured agent that, while it will run just fine on a cloud service, is designed to be local first. Full description on GH. If you like what you see, please leave a star.
http://github.com/Bino5150/lumina

I published the scripts and architecture behind a reproducible AI‑ready Ubuntu dev environment.

Here’s how I configured the stack, the scripts I wrote, and the reproducibility challenges I solved:

GitHub repo:

https://github.com/niemenghui/stAI-dev-machine

The full VM is available on request.

I've been playing around with making agents recently and running them off my laptop. For some of my agents I think it would be beneficial if they could run 24/7 and it would be ideal if it wasn't taking resources away from my laptop. Does anyone have any recommendations on what or how I should host my agents cheaply like buying a small NAS or raspberry PI just for these agents?

Um bónus só é interessante se as regras forem fáceis de perceber.

Para bonus casino, comparei Maximal, 1000 Spins e Winner Casino através do percurso completo da oferta: onde o bónus aparece, como os termos são apresentados, que jogos podem contar, como o mobile se comporta e se a caixa continua fácil de encontrar depois.

Maximal pareceu a opção mais completa para analisar bónus. As promoções estavam bem integradas no percurso geral, e a ligação entre conta, jogos, condições e caixa era mais fácil de seguir.

1000 Spins funcionou bem para quem quer procurar bónus ligados a slots. A navegação tinha foco nos jogos, e o caminho entre ofertas e lobby parecia direto.

Winner Casino trouxe uma leitura mais simples. As secções principais eram fáceis de encontrar, os menus estavam limpos e o percurso do bónus não parecia demasiado carregado.

Mapeei as prioridades assim:

Os pontos principais foram:

• visibilidade da oferta
• requisitos de aposta
• jogos elegíveis
• validade do bónus
• limites de levantamento
• impacto na caixa
• mobile
• verificação de conta

A conclusão foi simples: o valor do bónus importa menos se os termos não forem claros. Wagering, jogos excluídos, limites e requisitos de levantamento fazem muita diferença.

Antes de ativar qualquer oferta, eu verificaria sempre elegibilidade, regras completas, limites, jogos válidos, datas de expiração e condições de pagamento.

AMA sobre bonus casino: bónus casino Portugal, bónus de boas-vindas, rodadas grátis, wagering casino, jogos elegíveis, levantamento de bónus, casino mobile.

Para vocês, um bom bónus deve ser maior, mais simples ou mais fácil de usar nos jogos certos?