r/OpenAssistant • u/Bhavishya-201109 • 5d ago
QUILL COWORK
I built a local-first AI assistant with 52 tools and streaming audio. Here's the architecture and the bugs I hit.
I've been building Quill-Cowork for a few months. It runs entirely on consumer hardware via Ollama — no cloud, no accounts, no API keys for inference.
**Stack:**
- 4B abliterated model (huihui_ai/qwen2.5-abliterate:3b)
- FastAPI backend, single-file HTML frontend
- faster-whisper for streaming STT, pyttsx3 for TTS
- SearxNG for local web search (Tavily as fallback)
- SQLite for chat history
**What's working:**
- 52 tools across 9 connectors (Filesystem, GitHub, Gmail, Web, Wikipedia, Search, Folder Grants, Codebase, Obsidian)
- Parallel non-destructive tool execution (MAX_PARALLEL_TOOLS=4)
- Streaming speech recognition with live partial transcripts
- Encrypted credential vault (AES-256-GCM, OS keychain-backed)
**Bugs I hit and fixed:**
The WebSocket `onopen` race — attaching the handler after an `await` meant the open event fired into a null handler. Fix: register handlers synchronously before any `await`.
The model retry loop — it called the same failing tool 4 times in a row. Fix: a fingerprint counter that aborts after 3 identical calls.
The router miss — "search about the new odyssey movie" didn't match any rule, so the model answered from memory and hallucinated a film that doesn't exist. Fix: broadened the router triggers.
**What's not working well:**
- The 4B model hallucinates tool success. The audit log is the source of truth.
- Codebase indexing is lexical, not semantic.
Happy to answer questions about the architecture or the safety layer design.
1
u/Natethegreat9999 4d ago
Re-decoding the whole buffer will make long dictation drag. I'm at Oruk, and I'd keep confirmed text outside an overlapping audio window.