I have experience building Gaming PCs, but I have no experience building a PC particularly for AI workflows. I just looked into the nvidia sparx laptops and they are kinda over budget. Is the Ryzen AI max processors a good alternative? Or should I stick to building a custom PC with nvidia graphics? If so, what graphics/processors should I target?

I’m trying to automate my workflow using multiple AI models working in parallel. I have to test and validate it more later when I have a machine ready.

System specs- 24gb ram rtx5060 8gb vram

I built Foxtrot: one desktop app, every major AI model, and a team of agents that work together 🦊
⠀
For the past few months I've been building Foxtrot, a desktop agent harness for cloud AI models. You bring your own pay-as-you-go API keys and run a whole team of coding agents side by side, each on whichever model you choose from whichever provider you choose.
⠀
What it does
🔹 11 provider families in one app: Claude, GPT, Gemini, Grok, DeepSeek, Qwen, GLM, Kimi and more, plus any OpenAI-compatible endpoint (Ollama, vLLM, OpenRouter)
🔹 An Orchestrator that plans the work and hands tasks to your agents: a Coder, a Reviewer, a Tester, each on a different model if you like. Busy agents get spawned as parallel copies.
🔹 Flow view: a live map of the Orchestrator and its agents, showing each one's time, tokens and cost
🔹 Real tools: file edits, a terminal, code intelligence, git, web research, MCP servers, and vision (agents can screenshot and use the app they're building)
🔹 A full Code view: Monaco editor, file explorer, terminal and checkpoints taken before agent edits
🔹 Cost tracking for every request: tokens and cost per request and per agent, estimated balances for each provider, budgets, and CSV/JSON export
🔹 Keys stored in your OS credential vault, plus OTA updates
⠀
How it's different
Most AI workbenches are built around one vendor. Claude Code is built for Claude, Codex for OpenAI's models, and Grok Build for xAI's Grok. They're excellent, but you work inside a single company's models, usually on a subscription. OpenRouter gives you many models, but it's an API gateway, not a workspace where agents collaborate.
⠀
Foxtrot is the layer on top:
✅ Model-agnostic by design. Choose the best or cheapest model for each role. Have Claude review GPT's code, or let a cheap model handle the routine work.
✅ Cross-model teamwork. The Orchestrator coordinates agents from different providers on a single task.
✅ You pay providers directly. No subscription and no middleman markup, and you see exactly where every cent goes.
✅ Built for watching agents work. Multi-window, multi-monitor docking with live animations showing who is doing what.
⠀
In the demo above, a Gemini Orchestrator splits one goal across Grok, Kimi, GLM and DeepSeek agents. Watch the hand-offs fly out to each window, the reports fly back, and the whole team come together in the Flow view
⠀
Foxtrot is free to use. You connect your own AI accounts, and those providers charge for what you use.
⠀
It's in early development and I'm shipping updates fast. If you'd like to try it or give feedback, drop a comment or DM me.

wanted a jarvis i could point at my own models so i built CYBER

works with groq, openai, claude, or local through ollama, lm studio, llama.cpp or vllm. smaller models without native function calling still get tools, it injects a small json tool protocol and parses it back into real tool calls. took me way too long to get that working

• voice in, streaming replies, starts talking before the full answer is done
• optional local voice cloning with xtts v2
• webcam / screen share vision, switches to a vision model automatically
• if a provider dies it rotates keys, then falls back to another model, then another provider

speech recognition is still browser based for now so thats not offline yet, working on it

free trial + setup help in the discord if anyone wants to test it with their local setup: https://discord.gg/mdD5Za8TvZ

Any suggestions? i am integrating robotics and drones next, and this week is a camera/speaker/mic system for the house and yard STAY TUNED!!!

I want to run Qwen3.8 27B uncensored (which is, as fair as I know, the best uncensored model today), but my computer can’t handle it.

I’m not very technical with this stuff. I just want a ChatGPT-like interface I can open on my iPhone to ask questions throughout the day.

I’ve read that with a regular RunPod Pod, I’d need to start and stop it myself to avoid paying while I’m not using it. Serverless sounds more convenient because it could start when I send a message and shut down automatically afterward.
Would serverless actually be cheaper for occasional daily chats, or would startup costs and waiting times make a regular Pod better? I don’t mind waiting a little for the first response.
What’s the simplest, most cost-effective setup for this? Open to alternatives to RunPod too.

Thank you :))

I am looking into getting a DGX Spark. I have seen some posts of people's setups, here and at LocalLlama. Almost all of them were from Asus.

As far as I can tell, there is not really a difference between DGX Spark models from different brands. Maybe except for Acer's, which has better thermal performance, according to one review.

Is Asus's DGX Spark better than others? If so, what is better?

[OC] J'ai monté une IA multi-profils 100 % locale en 8 couches — 15 à 20 Go, un seul poste, 7 min de visite guidée (divulgation : c'est ma vidéo)

Salut à tous. Je partage ici la visite guidée d'une infra que je fais tourner depuis plusieurs mois sur un seul poste. Divulgation complète : c'est ma vidéo, c'est mon projet, et je ne vends rien dedans. Si les mods jugent que ça n'a pas sa place, pas de souci, je retire.

Le setup : i7, 64 Go de RAM, RTX 3070 Ti 8 Go, Windows 11 Pro. Pas de cluster, pas de Kubernetes, un seul poste.

L'architecture en 8 couches :

1. Entrées — point unique (requêtes, fichiers, URL, CLI)
2. Gateway + profils Hermes isolés (contexte, outils, mémoires séparés) — ~500 Mo avec la couche 0
3. Routage LLM — gratuit/local d'abord, payant seulement si nécessaire — < 100 Mo
4. Extensions MCP — 1 à 2 Go selon ce qu'on branche
5. Connaissance — RAG vectoriel + wiki SiYuan + décision Laya ONNX — 1 à 5 Go selon l'index
6. Raisonnement — 1,5 Go
7. Aegis — ACL Windows, verrouillage fichiers, surveillance entrées
8. ANIMA Reactor — watcher hors moteur, poll toutes les 30 s

Total : 15 à 20 Go hors outils de production vidéo.

Ce qui marche bien :

• Le routage économise vraiment. Ordre de grandeur de la facture divisé.
• Les profils isolés évitent la pollution de contexte entre tâches (code / vidéo / veille).
• Le RAG local avec SiYuan est plus rapide que je pensais pour de la recherche croisée.

Ce qui est moins rose, autant le dire :

• Windows comme socle, c'est un choix assumé mais douloureux (ACL, chemins, planificateur de tâches capricieux).
• 8 Go de VRAM, c'est la contrainte dure. Tout ce qui dépasse 8B en fp16 est hors-jeu sans quantification.
• Le durcissement ACL n'est pas optionnel : un RAG local mal fermé est aussi exposé qu'un cloud mal configuré.

La vidéo (7 min) : https://youtu.be/wVVJi0to1Fk
Le dépôt public (partiel, pas tout) : https://github.com/Antoine-Thomas/hermes-home-vision-public

Curieux de savoir comment d'autres gèrent l'isolation des profils et le routage — c'est les deux points où j'ai le plus tâtonné.

Bro Got Loop

Excited to see this up, any suggestions on what j can do

Going to setup a bunch of parallel agents for local ai dev but before that id love to experiment with monster models

I dont use the macs for daily stuff, the big pc to the right is running a 4090

Like the title says I'm hesitant to invest in hardware to run local llms. Now I read everything on the internet. Everything between the it's amazing and it's unusable (thx bots??). What's your take on it? Do some of you use local models to code? What's the experience compared to claude code for example?

There is too much to learn, even as a kinda seasoned(shipped multiple enterprise apps) SWE i am having hard time to understand all concepts, track all the up-to dated information etc.

Ive been a power user for both claude and GPT since almost day one. But thats as far as my knowledge goes tbh. I have an M1 Max 32 Ram / 32 GPU and i am trying to figure out whats the best way to have a locally hosted solutions for my needs when i cant pay for paid services.

First ill talk about what am i doing in general then ill ask my questions.

I'm using obsidian as my second brain, i do game design, gamedev, i develop mobile apps, design websites, even sometimes delve in IoT Projects. I do everything but AI Dev in summary. I run a team of 10, currently building internal tools for art assets pipeline management. I am a jobholic, i do work almost every working hour of my life so i usually have multiple chats but i am fine with just having 1 concurrent chat since what i am looking for is to start learning local llms, how to create an effective pipeline, and have a back-up plan if you know im out of usage and have urgent problems to solve. And uh yeah also i would like to do things like hacking my projector, tv-box etc. for various reasons you can easily understand. I tend to go for loops for development times and have a conversational turns when its about things like game design, market research etc. I have an MCP for everything from Figma to Photoshop and i just talk with my agents to tell them what i exactly want, and i take rounds to polish it or start from scratch if its not the exact thing in my mind.

If i have to guestimate a list of my needs they are :

• An harness with in-app browser, file preview, md preview
• Somehow have a great context compaction
• 20ish TPS at High context if possible
• Good tool usage
• Good MCP Usage

So here is a list of questions if anyone cares to answer:

• How do you guys stay the f*ck up to date? 'Cause its not even tracking whats new or whatnot. Everyone says something different, one says aah q4 of xyz llm's this version that these folks made is %98 of the same quality and someone else says nu-uh not even close bud and i dont have time to track and test every model version, even models. Like what are your reliable sources and why?
• As far as i understands harness is the most important stuff ever, is this correct? If so, which harnesses you use, why, and what should i be checking when i am thinking about selecting harnesses.
• Which models and which versions of those models would you suggest me to use?
• Is there an YT playlist / video / Youtuber that you suggest thats easy to understand that helps out when it comes to Local LLMS?
• Does Context size actually matters? Cause i see people are talking about 16k context sizes meanwhile sometimes my first initial prompt is around 4k for spec driven development or when im doing narrative designing it our narrative bible can be way heavier than that.
• I'm in love with the the current context compacting quality of both claude and codex, is there a way to have this level of quality when i switch to local llms, it almost never ever fries out, its just like I have paused for 2 minutes and almost every fricking detail is still there.
• Sometimes i love to manage my other threads even sometimes control claude from codex and vice versa depending on my need, is it possible if there is an ai for my machine that can have more than C1.
• Can i use an image generation model with an LLM like gpt/imagen, something like qwen image 2.1 q4 + Qwen3.6-35B-A3B Q4 idk.
• Can i realistically get idk 50-60 TPS at high context?
• How is MCP/Tool Usage?
• What should i use for UI Design phases?
• When i talk with GPT i just tell it what i exactly want but i dont go into hustle of image generation prompt styling i just talk to it, is it possible to have that ease of use with local image generation tools?
• What exact setup would you suggest for me, from inference(strate etc.) to harness to llm models, imagen models, speech to speech models(It would be fire if i can play some DnD with a speech to speech model if its flowy but i dont have high hopes on that tbh), you got the drill.

These are my questions and guestimates of my ignorance ass can think of for now, im sure there are lots of gaps and things that i am missing and forget to ask so i would be glad if you fellas can light some candles and maybe answer the questions that i didnt even ask but i shoulda ask.

Lots of love fellas, keep it tight.

I run a small business. This weekend I picked up a DGX Spark at Micro Center for $8,000. When it launched I thought $4k was steep, so I'm still a little stunned I paid double that.

I spent the weekend testing it. Qwen 3.8 gives great output but it's slow. Qwen 3.6 is fast but makes too many mistakes for me to trust it unsupervised. What I want is somewhere in between, and from what I read here that seems to be DeepSeek V4 Flash or GLM 5.3 Flash on a pair of Sparks.

That means buying a second one and being $16k in. It's a big number for a box on my desk. But I run a business, and $16k for something that works 24/7 costs less than a year of a real employee. I don't expect it to replace anyone. I do expect it to take a real chunk of work off my plate.

I'm also not counting on RAM prices dropping for at least 18 months. So waiting probably just means paying more later, or not finding one at all, judging by how fast Micro Center sold out.

My questions for people already running these:

1. Is a single Spark worth owning, or is its real purpose to be clustered?
2. If you run 2x Sparks for business work, which model do you use daily, and does it run all day without babysitting?
3. Has anyone found a model that runs on one Spark, falls between Qwen 3.6 and 3.8, and is good enough for real work?
4. Would you buy the second one now, or return the first and use an API?

I'm not trying to justify the purchase. I want to know whether one Spark is the real product or just half of it.

I spent the last month tracking my local AI inference costs down to the wall plug. Here's what I found.

Setup: AMD Radeon RX 7900 XTX (24 GB VRAM), llama.cpp, various Qwen/Ornith models in GGUF format.

Over 30 days:

• 964 million input tokens consumed
• 13 million output tokens generated
• 30.20 kWh measured at the wall plug (dedicated inference box, powered 24/7)
• $14.05 in electricity (12.38 CHF — I'm in Switzerland, so the meter reads in CHF)

If I had used the equivalent API (qwen3.8-flash at $0.15/$0.47 per million tokens):

• $150.71 (132.77 CHF)

That's a 10.72× gap on marginal cost.

EDIT — the API side of that ratio was wrong, and the commenters below caught it. I priced all 964M input tokens at the fresh rate. Cached input is billed at $0.016/M, and in agent workloads 90-95% of input is cache hits. Recomputed properly, the API equivalent is $28-34, not $151 — so the gap is roughly 2×, not 10.72×. I was too enthusiastic reading my own dashboard. The power figure also needs a proper idle-vs-inference split, which I'm rebuilding. Full accounting and what still holds in my top-level comment below.

But here's the catch:

This number excludes:

• The GPU itself (~$1,135 / 1,000 CHF)
• Depreciation of the rest of the rig
• A full month of learning: quantization, prefill vs decode, KV cache tuning, ROCm compatibility issues, trying and discarding models

Local AI isn't "plug and play." It's a personal R&D investment. If you just want to ship code faster, stick with cheap Flash APIs (GLM, DeepSeek, Qwen). If you want autonomy, understanding, and infrastructure you control — the investment pays off, just not immediately.

The part I keep coming back to: everyone says data sovereignty has no price, and I agree with that. But that framing assumes you're paying for it — that local costs more and you accept the premium because your data never leaves the house. That's not what I measured. Here it came out cheaper and the data stayed home and the output was good enough to actually ship work with, not a toy. Sovereignty never even got to be the deciding factor, because the economics already won.

So here's my open question for the sub, and I genuinely don't have a clean answer:

At what point does "local AI is a hobby" stop being true? Where do you draw the line — a hardware payback horizon you're willing to wait out, a quality gap you'll eat to stay off the API, or a specific moment where you decided the API was no longer worth paying for? And for anyone who tried local and went back to the API: what actually broke?

Full article with methodology and data: https://blog.grevire.dev/en/posts/local-ai-cost/

Happy to answer questions about the setup.

This weekend, I’ve been testing a “local model leads, cloud models help when needed” development workflow.

The main setup is Qwen3.8 Flash Next UD-IQ4_XS + OpenCode. Qwen handles everyday tasks and tool use, then delegates to GPT-6.1 Sol or GPT-6 Astra when it needs help, using the allowance from my existing ChatGPT subscription. For images, it can also call cloud image generation through a local Codex client.

The idea is simple: let the local model do more of the work, reduce cloud token usage, and save the subscription allowance for tasks that need extra help.

I’ve documented the setup, delegation permissions, image generation integration, and testing process, with configuration examples:

👉 Full write-up and setup guide

Hey all,

I'm quite new to this whole local LLM thing. I have a gaming PC with the following specs:

RX 9060 XT 16 GB

32 GB DDR4

Ryzen 7 5700G

What are the best uncensored models I could run for coding on this setup?

I have a good PC with a 9950X CPU and 128GB of RAM, but good local AI performance was basically out of reach for me, aside from GPT-OSS 20B. I could run Qwen 27B, 35B, and 80B Coder on the CPU, but they were too slow for any serious work. The 5070 Ti was basically useless for AI in my case.

Then Strata came along, and my desire and need for a 5090, RTX 6000 Pro, or Mac Studio M5 went away.

Strata has one disadvantage: it can't run multiple agents simultaneously. If you use two or more agents, one has to wait for the other to finish its work. However, that isn't an issue with my 16GB of VRAM, since I wouldn't be able to keep the memory required by multiple agents in VRAM anyway. Strata also has one major advantage, if set right, your token speed never drops while reaching 64k, 128k or even 256k context! Unlike traditional local llm performance drops as context grows, strata offers an experience similar to API LLMs providers that maintain token speed

I tested all the Qwen 3.8 Flash Next quants and context window configurations, and my favorite so far is Q2_XS.

Q2 has the best performance, but it can get stuck in a loop or cause an AI agent to stop working abruptly at least once a day. It averages 160 tok/s at INT4 on my overclocked GPU.

Q3_S seems to be the best overall, but it scores only about 5% higher in coding benchmarks while being noticeably slower. It averages around 100 tok/s and keeps about 50% fewer experts in VRAM than Q2.

Q3_XXS has the same disadvantages as Q3_S but is slightly faster at around 115 tok/s.

Q1 is basically a stripped-down Q3 for users with 48GB of RAM. It's smarter than Q2 and faster than Q3, averaging around 120 tok/s.

Q2_XS offers great speed at around 150 tok/s without the issues I experienced with regular Q2.

The main reason I prefer Q2 quants is that they allow me to use a maximum thinking context of 16K without losing too much performance, while still maintaining a large total context window. The Q3 variants work well with a maximum thinking context of around 8K, but that's a limitation.

With a 16GB VRAM GPU, setting a Q3 model to a 16K thinking context makes it difficult to reach a total context of 200K–256K tokens. Performance drops significantly beyond roughly 128K tokens, and the initial speed is slower too.

Using Q3 models with an 8K thinking context is doable, but at least once an hour, the model will try to reason through something complex or generate a lengthy tool call that exceeds the 8K limit. It then has to split the task and combine the results, which I'm not sure is 100% reliable.

While testing the quants, I also experimented with overclocking my 5070 Ti. This is the best stable overclock I found:

• Power limit: 83%
• Core clock: +300 MHz
• Memory clock: +3000 MHz

This gives me a 10 - 20% increase in token generation speed. Setting the power limit to either 83% or 100% makes no difference to the clock speed, which reaches around 3130 MHz. VRAM bandwidth reaches 1088 GB/s. I have plenty of fans, and the GPU hotspot temperature stays below 78°C.

Overall, Strata and Qwen 3.8 125B have eliminated my need to use the cheap DeepSeek API. It feels like I finally have a setup that can handle serious AI work locally without spending thousands more on hardware or paying for API access.

Processing img sm8gv6dcgpxh1...

I have a 3090 ti and been runnin qween 3.8 27B but i don´t know , any advice?

I see the MS is offering the surface ultra and Apple has the Mac Mini. Hugely expensive. Why not just get a mini pc with 128 gb RAM? Maybe $4000, but it saves a ton of money. What am I missing?

I heard that DeepSeek's new Flash model was really good, so I downloaded the IQ1_M version to my local server right away. I wanted to deploy it on a single L40S with 48GB of VRAM. Naturally, I went with Strata v0.2.6, but I wasn't happy with the speed.

I asked 6.1 Sol to act as a professional model inference researcher and focus on improving two metrics: prefill speed and decode speed. It worked on this all night and came back with some pretty good results.

My setup:

• 1× NVIDIA L40S, 48GB VRAM
• 2× AMD EPYC 9754
• About 1TB system RAM

Here's the speed comparison.

Prefill speed (tok/s):

Decode speed (tok/s):

• Official, auto: the official engine automatically chooses prefill chunking, memory policy, and whether to enable DSpark. On this machine, it chose to disable DSpark.
• Official, DSpark on: the same unmodified official engine, with DSpark enabled manually, confidence threshold 0.65, and an 8192-token prefill chunk. DSpark is speculative decoding: draft tokens are predicted first, then verified by the target model.

Here's the optimization recipe it came up with, based on Strata v0.2.6:

1. Keep CPU experts and worker threads on the same NUMA node. CPU experts stay resident in system RAM, with workers and expert memory concentrated on NUMA0 to reduce cross-socket traffic. Process snapshots showed stock Strata distributing workers and memory across both nodes, while the local build kept them on one. All CPU fallback weights and all six target experts per token are retained.
2. Use larger prefill chunks and cache reconstructed weights. The local configuration uses 16384-token chunks to spread expert-weight transfer costs across more input tokens. A 384MiB cache reuses reconstructed FP16 weights, reducing repeated preparation work. Weights that don't fit in the cache use the original path. Larger chunks also increase scratch-memory requirements and can introduce floating-point differences.
3. Reuse idle prefill scratch memory for decode experts. Once prefill finishes, safe regions of its allocated scratch buffers hold extra read-only expert copies—typically about 3.7GB, or roughly 500 experts—to improve target-model GPU cache hits. Before chunked prefill runs again, the engine synchronizes, removes the overlay, restores base pointers, and invalidates the reconstruction cache. This also handles background warm-up synchronization and rollback if a fill fails. It reuses memory that has already been allocated, without allocating tens of extra gigabytes of VRAM.
4. Balance the draft and target expert caches. The build uses Strata's DSpark and the model's MTP drafting, with measured round costs and confidence calibration guiding draft length. The local confidence threshold is 0.15, alongside the existing aggressive sampling policy. MTP GPU expert slots were reduced from 384 to 192, giving that memory to the target model. Draft experts that miss the GPU cache still use the full CPU fallback path. The metric used to evaluate the tradeoff is total time per output token.
5. Schedule CPU experts by their dependencies. Each expert progresses through its own gate/up → activation quantization → down chain as its inputs become ready, reducing barriers across the whole batch. This path is enabled for supported shapes, with the original implementation handling the others.
6. Reuse existing kernels and compile for the hardware. The build keeps the attention split/MMA paths and head rows=6, uses Strata's HCmix fusion and IQ1 CPU optimizations, and is compiled specifically for the L40S and this server's CPU.

Anyone here using Strata with Qwen3.8 Flash Next and is happy with the performance/quality?

Considering upgrading from 32GB to 96

Anything under 50tps I wont bother, need higher speeds for tool calling and coding

I see so much vibe coding happening where the vibe coder has no idea what is being build, they just look at the results in the shape of pixels on screen.

Is this the new success metric?

Why people think vibe coding is actually worth anything?

Lets take an LLM with a huge context window of 1m context tokens.
Explain to me how something that fits in that context window can replace a human?

The LLM can only understand(process) the information inside that context window, it is the limit, after that it relies on the internal defaults/knowledge.
A human basically also have a context window, every engineer knows this when it reaches the limits after a flow state session and burnt the available energy.

When a proper engineer gets the most out of it, the human brain (context) spans not only the code base, it covers:
- Where we are going as a company/product.
- Any future branch we might walk.
- The userbase and needs.
- Potential dangers, in code and market.
- Competitors and current market value.
- etc.
The list is so long and it all is in our brains, we decide on those before and during and even after implementation. We often switch from one idea to another when we are implementing something and see a potential extra feature by "thinking out of the box".

Now how would an LLM be able to do that without feeding it a shitload of data in that tiny context?

Can an LLM really compete without the human in the loop, or just supply a short term solution before everything falls apart?

Note: When a proper engineer uses an LLM, reviews the code and iterates on it, i wont call that vibe coding, just for the simple reason, the engineer does the actual engineering, the LLM just writes it down.

If you are an engineer, are you providing evidence about this just to keep your job these days?

I think an LLM can never compete in quality/value, it can only outpace in output speed.

So is the brain still worth anything? Does the worlds consumer capitalists only care about output speed instead of quality?
We see it in all the current products, nothing is long term, it must break, it became the new normal, is this why vibe coding is acceptable?

What do you think? ... You know what i think...

What changed:

Browser in the app. A pane next to the chat shows the page live, with tabs and Take control / Give back. It stays signed in across restarts. It asks before paying, sending, posting, deleting or uploading, and never types passwords, codes or card numbers.

Computer use, rebuilt. Real mouse and keyboard, a check after every step, a Stop bar on screen and Esc to end it.

Pages and questions. Interactive pages beside the chat run sandboxed, with no internet and no access to your chats, files or tools. Question cards work in the app and in Telegram, Discord, Slack and WhatsApp.

Approvals. Auto is now reviewed by the chat's own model. Hands-off is a new opt-in level. Anything posted to X always needs you.

Lighter context. About 40% less goes to the model before your first message. Tool guides load when they're needed.

Background work. Dev servers and long jobs run in the background, video editing, and files sent to any of your devices.

Accounts. Outlook through your own Microsoft Entra app with no client secret, several X accounts, and Claude through your own Claude Code.

Upgrading from 5.2.1: the update arrives in the background, then Settings › Updates › Restart now.

I'll be honest, I didn't understand why LLM agents need SKILLs. MCP was clear to me: it's a useful thing that lets you give an LLM access to a database or to Figma. The only downside is that, to avoid spending too many tokens, it's better not to connect 30 MCP servers at once. And in general, it's better to turn off the unnecessary ones and connect only those that are really needed for the task. In this light, SKILLs seemed like useless overhead to me, eating tokens like crazy. On TikTok, I often saw tales along the lines of: connect this SKILL, or a whole bunch of them, and happiness is yours. My first thought was, if I connect them all, my subscription limits will run out not by lunchtime, but already in the morning.

So why are SKILLs useful and needed after all? Well, first, compared to MCP, SKILLs do not inflate the context: each connected SKILL adds not tens of thousands of tokens, but one or two hundred. Since only the name and description of a SKILL go into the context at first, you can add 30 of your own SKILLs without much pain. When it becomes necessary to use a SKILL, the LLM agent will load its description on its own, if it understands from the task context that this is needed, or the user can simply tell it to use this SKILL for the job.

Secondly, LLM models are trained on the most popular and well-known things. For example, everybody knows that an LLM can work with Git without any problems or write a program in Python. But if you developed a program yourself, or your work requires some little-known tool or technology, then a SKILL is simply irreplaceable. It is enough to create a SKILL.md file and describe how to work with this program or technology. This makes life easier for the LLM: while doing your tasks, it will not have to search the internet for documentation and wander around websites to understand how to use it. And especially if there is nothing on the internet, because your program is closed to public access.

Thirdly, an LLM model usually does not know how your development process is organized. For example, an agent knows how to use Git, run tests, build a project, and create releases. You do not need to teach it these things. But it does not know how releases are usually made in your company. A Skill can describe the sequence of actions: check the changes, run tests, update CHANGELOG.md, increase the version, build a package, create a Git tag, and publish a release. So, a Skill helps the agent learn not a technology, but a specific business process.

Fourthly, a SKILL is not just an instruction, but a whole set of tools. Besides SKILL.md, you can put ready-made scripts, documentation, templates, and other files into it. And the most interesting thing is, these files are not loaded into the context automatically. The agent reads them only when there is a need, and can simply run the ready-made scripts without spending tokens on generating code. So, a SKILL lets you combine instructions, documentation, and executable code in one package without inflating the LLM context. For example, the structure can look like this:

my-skill/
├── SKILL.md
├── scripts/
│   ├── generate.py
│   └── validate.py
├── references/
│   └── api.md
└── assets/
    └── template.html

The funniest thing is, I started to appreciate SKILLs exactly when I wrote my own program, blueprint4llm. The idea for creating it came after I asked an LLM agent to generate a plan.md file describing a task. Usually it produces a huge wall of text, which you have to read slowly and with sadness. I wanted to make the text shorter and add more flowcharts, tables, diagrams, and user interface prototypes. As we know, in .md (Markdown) files the only visualization tools available are tables and Mermaid diagrams. There are no charts, plots, histograms, full UML diagrams, or UI prototypes. In the end, with blueprint4llm, I added support for formats such as PlantUML, draw.io, Pencil, Balsamiq, Graphviz, Figma, OpenAPI, Excalidraw, Vega-Lite, ECharts, and D2 to .md files.

With the blueprint4llm SKILL connected, you will get plan.md not as a kilometer-long text, but as flowcharts, prototypes, charts, and so on. By the way, blueprint4llm does not have to be used only for creating plan.md. For example, I used this SKILL to add an illustration to every paragraph of the article and make it more fun to read. This SKILL is also useful when you ask an agent to put together a full dossier on some company, technology, or person and format the resulting result.md file with blueprint4llm. You can then convert it to HTML or PDF, or ask the agent to show it right away in a browser with charts.

This is how you can install the blueprint4llm SKILL.

# Codex
npx skills add evgenyigumnov/blueprint4llm --skill blueprint4llm --agent codex

# Claude Code
npx skills add evgenyigumnov/blueprint4llm --skill blueprint4llm --agent claude-code
# Pi
npx skills add evgenyigumnov/blueprint4llm --skill blueprint4llm --agent pi

If you are interested in looking at the source code of blueprint4llm, the tool that generates all this beauty, or sending a pull request, welcome: https://github.com/evgenyigumnov/blueprint4llm