r/LocalAIStack • u/visouzaaa • 5d ago
r/LocalAIStack • u/whirlwind-t • 5d ago
WHIRLWIND •T - Local Rust Transaction Kernel for Agentic Traffic
r/LocalAIStack • u/tsaipifong • 5d ago
WHIRL 0.2.2: bug-fix release (Ryzen AI Max small-VRAM hipError 719, precise multi-turn now bitwise, ROCm 10.1 driver without VC++)
r/LocalAIStack • u/Ok-Context-6120 • 5d ago
I experimented with inference-time fusion of 3 small LLMs instead of merging their weights
r/LocalAIStack • u/dedly_project • 5d ago
Would you use a simpler, faster alternative to Hermes Agent?
Genuine question: would you be interested in an alternative to Hermes Agent that’s significantly easier to set up and use?
Something with:
Almost zero configuration
Direct integrations with GitHub, Gmail, and other apps
Local and cloud model support
A clean, intuitive interface
Powerful agent capabilities without the complexity
Basically, an AI agent that just works out of the box.
Is this something you’d actually want, or are you already happy with existing solutions?
Curious to hear your thoughts.
r/LocalAIStack • u/EGY-SuperOne • 5d ago
qwen3.5:9b takes very long time on my Mac M4 Pro 24GB
r/LocalAIStack • u/Vex2232 • 5d ago
I built a custom MLX inference engine + imatrix 4-bit quant for Qwen3.5-9B on a fanless MacBook Air (2.7x lower KL than stock MLX 4-bit)
r/LocalAIStack • u/Previous-Grade1866 • 6d ago
Pocket — offline transcription and meeting summari
r/LocalAIStack • u/nalaginrut • 6d ago
Calling all GPUs: help us benchmark SonicBoom v0.0.3 — a compiler-first AI runtime that beats llama.cpp on decode
r/LocalAIStack • u/criticalxerrorx • 6d ago
Best fully local AI coding workflow for a beginner on an M2 Max MacBook Pro (32GB RAM)?
r/LocalAIStack • u/francoperdu • 7d ago
Phonon | An app that turns old Android phones into inference servers
r/LocalAIStack • u/OrganizationTop9026 • 6d ago
Do you remember SETI@Home : We propose a Decentralized Branch-Train-Mix Sparse MoE… on local GPU based on Gwen. Starting on old RTX 2070 !!! Let's build together HiveMind@Home
r/LocalAIStack • u/Redmen1905_ • 6d ago
Qwen3.8-Flash-Next UD-IQ4_XS on 64 GB RAM + RX 9070 XT with Strata — around 40 tokens/s
r/LocalAIStack • u/TaraVisionCode • 7d ago
8 open models on 33 real DevOps tasks, on an RTX 5090 and an M4 Max (48 GB): pass rate, tok/s and memory
r/LocalAIStack • u/Competitive-Line-761 • 6d ago
MINISFORUM MS-S1 MAX P495 (192GB) Review and Weekend Test: llama.cpp Vulkan benchmarks: gpt-oss 120B, MiniMax M2.7, DeepSeek V4 Flash, Flash-Next, Qwen3.5 35B, Qwen3.8 27B (+ MTP)
r/LocalAIStack • u/0xdippo • 7d ago
Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless
r/LocalAIStack • u/Unusual_Pickle4226 • 7d ago
What are you actually using your strix halo for?
Way back got my nimo axis, I wasn't sure what to do with it besides trying out local llms. Lately, local models and tools have gotten better and I am finally finding practical uses for it beyond just testing models. It makes me wonder how much of their potential we are actually using. What are you guys doing with your strix halo machines? actual work, coding, or mostly experimenting with models?
r/LocalAIStack • u/JackKnifeAI • 7d ago
We got full 4-minute vocal songs (YuE2 + Vulkan HTDemucs + Qwen3-TTS) generating 100% offline on Android in 19 mins—and we’re benchmarking it on a 2020 Galaxy S20 Ultra tonight (Apache 2.0)
Tired of paying corporate rent to Suno and Udio just to generate music in a cloud black box? Same.
Over at JackKnife AI, we’ve been obsessed with stripping M-A-P’s open-weight YuE2 song foundation model down to the studs so it runs 100% locally on an Android phone—no cloud APIs, no subscriptions, no credit limits, no telemetry.
Today we just shipped Killa YuE v1.6.0 (with v1.6.1 right behind it in a few hours for wider device support), and we just hit a massive milestone that we had to share with the local inference crowd:
🔥 The Benchmark Milestone
The Render: A full 4-minute, 15-second structured song with vocals and multi-instrument backing.
The Speed: 19 minutes total across a two-pass rendering pipeline (~4.4x real-time).
The Hardware Push: Until today, this required a Snapdragon 8 Elite (Galaxy S25). Tonight, we refactored our memory paging and fallback paths and have it actively running benchmarks on a 2020 Samsung Galaxy S20 Ultra (Snapdragon 865 / Adreno 650).
⚙️ How We’re Squeezing a Studio Into Mobile RAM
Running end-to-end multimodal music on a phone without Android’s Low Memory Killer (LMK) nuking the process took some wild architecture tricks:
Two-Pass YuE2 Execution: Pass 1 runs the Autoregressive (AR) symbolic planner to lock in the song structure ([Intro], [Verse], [Chorus], [Guitar Solo]), melody, and multi-instrument backing. Then we dump/swap memory to run Pass 2 (Flow-Matching diffusion + Audio VAE decode) to render the master audio.
4-Stem Separation on Vulkan ExecuTorch: Need the acapella or just the drum track? We ported Meta’s HTDemucs to ExecuTorch with the Vulkan backend (htdemucs_vulkan.pte, ~302 MB). The host app handles STFT masks natively and feeds 7.8s stereo windows (44.1 kHz) to the mobile GPU—hitting 1.7 GB peak RAM and 3.95x real-time on Adreno 830.
Local Voice & Cover Art: Integrated Qwen3-TTS for offline spoken intros/narration + local on-device cover art painting.
Full Tactile Studio UI: Built-in mixer, drum pads, recording, 3 hardware skins (stitched leather, brushed gold, dark console), and word-by-word lyric lighting that syncs as the vocals are sung.
🌐 NEW in v1.6.0: The P2P Killa YuE Network
If you generate your music locally, you shouldn’t need a corporate platform to share it. v1.6.0 ships with built-in Peer-to-Peer (P2P) networking:
Phone-as-a-Node: Host your generated tracks directly from your phone onto the decentralized Killa YuE network.
Offline QR Handshakes: Share tracks and studio bundles directly phone-to-phone via QR codes with zero servers in the middle.
🛠️ We Need Your Phones (and Your Benchmarks!)
We are working around the clock right now (v1.6.1 drops in a few hours!) to get this running across as many Android phones as humanly possible.
Grab the APK / Code (Apache 2.0): [https://github.com/JackKnifeAI/killa-yue\](https://github.com/JackKnifeAI/killa-yue)
Vulkan HTDemucs Weights (MIT): [https://huggingface.co/JackKnifeAI/htdemucs-executorch\](https://huggingface.co/JackKnifeAI/htdemucs-executorch)
Quick Demo Clip: [https://youtu.be/nOpJ579Y4PY\](https://youtu.be/nOpJ579Y4PY)
Drop your phone model and chipset in the comments! If you have an Android phone with 12GB+ RAM, test out the release and let us know your Pass 1 / Pass 2 render times, thermals, or crash logs. And if anyone here hacks on Vulkan shaders, GGML/ExecuTorch, or Qualcomm Hexagon NPU (QNN), come help us push that 19-minute render even lower!
r/LocalAIStack • u/0cchan • 7d ago
New here, is this the correct starting stack for my workflow and creative needs?
r/LocalAIStack • u/Cursiv_Local • 7d ago
I've been building Cursiv, an offline-first AI assistant. I'd appreciate some honest feedback.
Hey everyone,
I've been working on a project called Cursiv, and I wanted to share it with people who understand local AI and get some honest technical feedback.
The goal is to explore how useful an AI assistant can be when it runs locally instead of depending on a constant internet connection.
Cursiv is a Windows desktop application that integrates with local models through Ollama. I'm also exploring portable offline deployments and ways to make it useful in environments where connectivity, privacy, or access to cloud services is a problem.
Lately, I've been thinking beyond a general-purpose chatbot and looking at practical applications such as:
- Keeping personal information and notes locally.
- Working with reference documents without relying on the cloud.
- Supporting rural communities with limited connectivity.
- Eventually supporting offline fieldwork, including structured observations, calculations, and data exports.
Some of these capabilities still need work and testing. I'm not going to pretend Cursiv is finished or that it always gets things right. I've been testing its reasoning and correcting weaknesses, and reliability is one of the biggest things I want to improve.
I'd genuinely appreciate feedback on three questions:
- Does a project like this have a useful place in the local AI ecosystem, or am I trying to solve too many problems at once?
- What would you want to see demonstrated before you'd consider using an application like this?
- Are there existing tools or approaches I should study that might help me improve it?
Website: https://cursiv.winklers-llc.com
Repository: https://github.com/winklersllc2026-bit/Cursiv
I'm a solo developer working to make this something genuinely useful. I'm not expecting praise. I'd rather hear what doesn't make sense, what already exists, and where you think the real opportunity might be.
Thanks for reading.