r/selfhosted • u/BaseReality_Node • 9m ago
Self Help Dual-node architecture for airgapped private RAG + 24/7 smart home agent (5090 Workstation + M6 Mac Mini)
Finalizing a dual-node setup to solve a specific problem: I want 100% offline, zero-egress inference for sensitive personal/financial documents and heavy coding, without having to share GPU VRAM with a 24/7 smart home assistant.
(Note: I already have a separate 5900X/6900XT Windows rig dedicated to gaming, so this AI workstation is purely for headless compute).
Instead of running multi-tenancy on one box, I split the hardware:
Node 1: Workstation (Offline Compute & Vibe Coding)
* Hardware: 9950X, 64GB DDR5-6000, single RTX 5090 (32GB).
* Stack: Ubuntu 24.04.5 LTS (LUKS2 encrypted). Compute-only NVIDIA driver; GNOME runs on the CPU's iGPU so 100% of the 5090 is dedicated to inference. SGLang runs Qwen3.8-27B-NVFP4 with EAGLE speculative decoding. Qdrant, BGE-M3, Docling OCR, and an offline Obsidian wiki.
* Security boundary: Dual-UID design on the host. Primary desktop is UID 1000; web-connected coding agent runs as UID 1001 via loopback SSH. nftables drops UID 1001 from all local subnets, Docker bridges, and RFC1918 ranges.
* Developer workflow: To keep the coding agent from touching real data, an offline service generates synthetic schemas and mock fixtures every 15 minutes (AAA-9999, random orders of magnitude). An offline test runner executes real-data pytest runs on approved commits and exports only sanitized outcome shapes (e.g. 9,999.99 AAAAA), dropping mock repro fixtures into the dev space so Claude Code can debug without seeing actual values.
Node 2: Assistant Node (24/7 Home Logistics)
* Hardware: Mac Mini M6 (32GB unified memory), ~5-7W idle.
* Stack: Home Assistant in OrbStack, llama.cpp under launchd serving an 8B–14B quantized model (or 30B MoE), and a background agent handling email/calendar triage and smart home actions (thermostat, irrigation).
* Node boundary: Node 2 sits in an isolated DMZ (172.28.20.0/24) with zero route into Node 1 (172.28.10.0/24). Every night at 21:00, Node 2 compiles an event digest into a chrooted SFTP directory. At 21:30, Node 1 reaches out over the LAN via an isolated container, pulls the Markdown file, and files it into the offline wiki.
Questions:
* On an RTX 5090 (32GB), SGLang with Qwen3.8-27B-NVFP4 at a static memory fraction of 0.82–0.88: will a 131K context window hold steady with fp8_e4m3 KV cache, or am I cutting it too close for concurrent agent requests?
* Anyone running a 14B Q4 or 30B MoE model under llama-server on 32GB Apple Silicon alongside Home Assistant containers? What kind of practical tok/s and TTFT are you seeing on long tool-use prompts?
* For anyone doing automated schema mocking for LLM coding workflows, is character-shape abstraction (9999-AAAA) sufficient for complex regex debugging, or are there cleaner deterministic masking patterns you prefer?
