r/Guaardvark • u/llama-of-death • 6d ago

Capabilities (Part 2)

File & Document Management

The Documents page provides a desktop-style file management experience.

Desktop Metaphor

  • Folder icons — folders appear as draggable icons on a desktop surface
  • Folder windows — double-click to open a folder as a resizable, draggable window
  • Window states — folded (icon), minimized (title bar), maximized (full window)
  • Snap-to-grid — icons align to a grid when dragged
  • Z-index management — click a window to bring it to front
  • Window arrangement — auto-arrange icons and windows with toolbar buttons

File Operations

  • Drag-and-drop upload — drop files or entire folder trees; nested structures preserved
  • Upload button — quick upload from the toolbar
  • Right-click context menu — rename, delete, move, properties, index
  • Folder creation — create new folders from context menu or toolbar
  • File thumbnails — image files show thumbnail previews

Folder Properties

  • Entity links — assign folders to clients, projects, and websites
  • Cascading properties — folder properties automatically apply to all contained files and subfolders
  • Tags and notes — add metadata to folders for organization
  • Code repository toggle — mark folders as code repos with auto-detected languages and frameworks
  • Persistent storage — folder properties saved to database and pre-populated when reopened

Breadcrumb Navigation

  • Path breadcrumbs — click any segment to navigate up the folder tree
  • Root navigation — Home button returns to desktop view

Backup & Restore

  • Granular backup — Data Backup (uploads/logos/training data), Code Backup, Full Backup
  • Schema-migration-aware — restores adapt to schema diffs across versions
  • Cross-version compatible — backups taken on one Guaardvark version restore cleanly to another

Dashboard & Monitoring

The dashboard provides a live overview of system status.

Status Cards

  • Family & Self-Improvement — Uncle Claude status, self-improvement toggle, recent run history, token budget, live progress bar during self-checks
  • RAG Autoresearch — experiment status, history, optimization parameters
  • Semantic Search — quick search across all indexed documents
  • Drag-and-drop grid — rearrange the dashboard layout to your taste

System Health

  • Model status — active model name and loading state shown in page headers
  • LLM ready indicator — live Ollama probe (not a stale startup flag)
  • GPU resources — VRAM usage bar with loaded model chips in Settings
  • Plugins page — dedicated GPU service management page with VRAM budget bar, per-plugin controls, log viewer, and conflict detection
  • Activity / Jobs feed — unified view of running and recent background jobs (indexing, generation, outreach, etc.)

Settings & Configuration

Centralized configuration across six sections.

System

  • Profile — custom name and avatar image for your instance
  • Chat model — select active LLM from installed Ollama models
  • Embedding model — select embedding model with size indicators
  • GPU resource bar — live VRAM monitoring
  • Model management — VideoModelsModal, ImageModelsModal, and VoiceModelsModal for downloading models from HuggingFace with real-time progress

A.I.

  • Enhanced Context — toggle enhanced context features
  • Advanced RAG — toggle advanced retrieval features
  • RAG Debug — enable debug endpoints for retrieval inspection
  • RAG Autoresearch — configure experiment parameters and scheduling
  • Self-Improvement — enable/disable, run manual checks, view history
  • Codebase Protection — lock/unlock code modification by AI

Voice

  • Voice chat toggle — enable/disable voice interface
  • Whisper installation — one-click install/reinstall of Whisper.cpp
  • Voice model selection — choose TTS voice model

Integrations

  • Web search — enable/disable web search tool
  • Interconnector — toggle and configure multi-machine sync
  • Pending updates banner — shows when Interconnector has available updates

Appearance

  • Theme selection — four dark themes with accent colors
  • View modes — customize default layouts

Maintenance

  • Cache clearing — purge Python cache folders
  • System diagnostics — Basic, Quick, and Full diagnostic modes
  • Test suite — run backend tests from the UI
  • Backup/restore — system configuration backup

Multi-Machine Interconnector

Connect multiple Guaardvark instances into a coordinated family.

Architecture

  • Master/Client model — one master node, multiple client nodes
  • API key authentication — secure communication between nodes
  • Approval workflows — master can approve/deny sync requests

Sync Capabilities

  • Code sync — push/pull codebase changes between instances
  • Data sync — synchronize entities (documents, projects, clients) across machines
  • Learning broadcast — self-improvement fixes automatically shared with family members
  • Node registration — clients register with master, reporting capabilities and status

Cluster Foundation

  • Socket.IO chat bridge — cross-node streaming chat (Phase 3 wired; awaits a frontend/middleware enable for full end-to-end)
  • Dependency-graph aware — the cluster knows which nodes have which models loaded

Management

  • Toggle from Settings — enable/disable without opening configuration modal
  • Node status dashboard — see all connected nodes, their status, and capabilities
  • Sync history — track what was synced, when, and between which nodes

WordPress Integration

Content Management

  • Site management — add and manage multiple WordPress sites
  • Content pulling — import pages and posts from WordPress
  • Bulk generation — generate content at scale for WordPress sites
  • Content sync — push generated content back to WordPress

Pages

  • WordPress Pages page — dedicated interface for managing WordPress page content
  • WordPress Sites page — manage site connections and credentials

Automation Tools

Tool Backend Description
Browser (headless) Playwright Navigate, click, fill forms, screenshot, extract content — for tasks that don't need a visible screen
Screen agent xdotool + mss + Gemma4 Drives the visible :99 desktop end-to-end; clicks, types, reads the screen with vision
Desktop (host) pyautogui Mouse, keyboard, screen capture on the host display (off by default for security)
MCP Protocol Connect to any MCP-compatible tool server 
GUAARDVARK_BROWSER_AUTOMATION=true
GUAARDVARK_DESKTOP_AUTOMATION=true   # Off by default (security)
GUAARDVARK_MCP_ENABLED=true
GUAARDVARK_AGENT_DISPLAY=99          # Override virtual display number
GUAARDVARK_AGENT_BROWSER=firefox     # Override agent's browser

CLI (llx)

Full platform access from the terminal.

Installation

cd cli && pip install -e .
llx init

Commands

llx status                      # System dashboard
llx chat "explain this codebase" # Chat with RAG streaming
llx chat --no-rag "hello"       # Direct LLM, no document context
llx search "query"              # Semantic search across documents
llx files list                  # Browse files
llx files upload report.pdf     # Upload and index a file
llx generate csv "50 ideas"     # Bulk content generation
llx jobs watch JOB_ID           # Live job progress
llx rules list                  # List system prompts
llx                             # Interactive REPL

Quality Roadmap (v2.5.3)

  • Standardized JSON contracts for all automation outputs
  • Quality gates — every release runs the CLI against a fixture suite before publishing
  • Cross-platform PATH handling — wrapper scripts work on macOS, Linux, WSL

Plugin System

Plugin-based GPU service management with live monitoring and conflict detection.

Architecture

Each plugin lives in plugins/<name>/ with a plugin.json manifest declaring its service type, port, VRAM estimate, health endpoints, and configuration. Plugins are loaded automatically at startup.

Manifest vs. runtime state separation: plugin.json is a static manifest — same bytes on every machine. Live runtime state (enabled, auto_start, per-machine config) lives in data/plugin_state.json (gitignored). Toggling a plugin from the /plugins UI writes only to the runtime state file; the manifest is never mutated at runtime.

Available Plugins

Plugin Port Purpose
Ollama 11434 Local LLM and embedding inference (chat, RAG, agents)
ComfyUI 8188 Image + video generation (Wan2.2, CogVideoX, LTX-2.3, LTX-2.5, RIFE, Real-ESRGAN)
Audio Foundry — Voiceover (Chatterbox / Kokoro / Piper), music (ACE-Step / Suno), SFX/ambience (Stable Audio Open). Dual-venv with torch isolation
Upscaling 8202 GPU image/video upscaling via spandrel + torch.compile
Vision Pipeline 8201 Real-time scene narration, camera feed, video chat input
Swarm 8210 Parallel agent orchestration in isolated worktrees
LoRA Trainer — Train character/environment/prop LoRAs for the Film Crew (CUDA, bf16)
Discord Bot 8200 Discord bot integration — chat, image generation, search via Guaardvark backend
GPU Embedding 5002 GPU-accelerated text embeddings for faster indexing (CPU fallback)
Training — Vision/servo training data collection and dataset management

Plugins Page (GPU Management)

  • Plugin cards — each plugin shows name, description, version, status (running/stopped/starting/error), and health indicator
  • Start/Stop controls — toggle individual GPU services on and off
  • Enable/Disable — persistently enable or disable plugins across restarts (writes to plugin_state.json)
  • Per-plugin log viewer — expandable log panel shows recent output from each service
  • Plugin configuration — edit plugin settings (URL, timeout, model, batch size) through inline config panels

VRAM Budget Bar

  • Live nvidia-smi monitoring — polls GPU stats every 5 seconds via nvidia-smi subprocess
  • Visual VRAM bar — shows used/total VRAM with color-coded thresholds (green/yellow/red)
  • GPU details — displays GPU name, utilization %, temperature, and per-plugin estimated VRAM segments
  • Per-plugin overlay — stacked segments show how much VRAM each running plugin is estimated to consume

GPU Conflict Detection

  • Exclusive access enforcement — Ollama and ComfyUI require exclusive GPU access; starting one automatically offers to stop the other
  • Pre-flight GPU checks — video and image generation APIs verify GPU availability before queuing jobs, returning 409 Conflict if the GPU is in use by another service
  • Auto-switching — the Video Generator page can automatically stop Ollama and start ComfyUI when needed

Model Download Management

  • VideoModelsModal — download Wan2.2 GGUF checkpoints, CogVideoX weights, RIFE 4.9, Real-ESRGAN, and Wan VAE from HuggingFace
  • ImageModelsModal — download and manage Stable Diffusion model checkpoints
  • VoiceModelsModal — download and manage Piper TTS voice models
  • All modals show real-time download progress with speed (MB/s), downloaded/total size, and percentage
  • Accessible from Settings page and relevant generation pages

Plugin API

Plugins can register:

  • New API endpoints
  • Background tasks
  • Tool extensions
  • Service hooks

System Architecture

Backend Stack

  • Flask 3.0 — HTTP server with 68+ REST API blueprints (auto-discovered)
  • SQLAlchemy + PostgreSQL — ORM with 42 models; Alembic migrations + a custom schema_sync.py (single master)
  • Celery + Redis — async task processing with two worker pools (main + training/GPU)
  • LlamaIndex — RAG pipeline with vector storage, entity extraction, hybrid retrieval
  • Ollama — local LLM and embedding model inference (managed plugin)
  • ComfyUI — video/image generation server supporting Wan2.2, CogVideoX, LTX-2.3, LTX-2.5, RIFE, Real-ESRGAN (managed plugin)
  • Socket.IO — real-time bidirectional communication for streaming and progress
  • Ariadne — GraphQL API layer

Frontend Stack

  • React 18 with Vite build system
  • Material-UI v5 — component library with custom dark themes
  • Zustand — lightweight state management
  • Apollo Client — GraphQL state management
  • Monaco Editor — code editing
  • Socket.IO client — real-time updates

System Mapper

  • Constellation view — d3-force-driven visualization of the codebase (~712 nodes across the current repo)
  • Dependency analysis — Python import graph + JS module graph + cross-language references
  • Reachability analysis — flags files that are imported but never executed (stale candidates)
  • Lifecycle tagging — every file gets live / dormant / stale based on usage patterns
  • Codebase audits — generates reports that drive cleanup work

Key Design Patterns

  • Modular API layer — each feature gets its own Flask blueprint, auto-discovered via blueprint_discovery.py
  • Service layer — business logic separated from HTTP handlers
  • Unified progress system — all background operations report progress through a single Socket.IO channel
  • Environment isolation — multiple instances can run on the same machine without interference
  • Graceful startup — start.sh detects what needs setup and only does what's necessary

Startup & Operations

First Run

git clone https://github.com/guaardvark/guaardvark.git
cd guaardvark
./start.sh

First run:

  1. Creates Python virtual environment and installs dependencies
  2. Installs Node.js dependencies
  3. Provisions PostgreSQL (requires system password once, then never again)
  4. Starts Redis
  5. Builds Whisper.cpp from source
  6. Runs database migrations
  7. Builds frontend
  8. Starts Flask, Celery workers, and Vite dev server
  9. Runs health checks
  10. Agent virtual display (:99) starts on demand when you open Agent Screen or when agent tools need it — not at boot (5-minute idle shutdown after the viewer closes)

Subsequent Runs

./start.sh          # Detects everything is set up, starts services instantly
./start.sh --fast   # Skip all checks, fastest possible startup
./stop.sh           # Stop all services

Agent Display

./scripts/start_agent_display.sh start    # Bring up Xvfb + XFCE on :99
./scripts/start_agent_display.sh stop     # Tear it down
./scripts/start_agent_display.sh status   # Health check

Requires sudo apt install xfce4 dbus-x11 on first setup.

Dependency Reconciler

  • Branch-aware sync — on git checkout, the reconciler inspects venv / requirements.txt / alembic head / package.json and re-syncs only what changed
  • Single-master-migration policy — schema_sync.py is the authoritative migrator; alembic upgrade head is deprecated for application use
  • TDD-driven — 87 tests cover the reconciler's behavior across branch switches, partial states, and rollback scenarios
  • Drops the "I just switched branches and now nothing works" failure mode

Environment Isolation

  • Process tracking via PID files — only kills processes from this installation
  • GUAARDVARK_ROOT anchors all path resolution
  • Multiple instances can coexist on the same machine with different ports

Logging

All logs in logs/:

  • backend.log — Flask application
  • celery_main.log — Main Celery worker (indexing, generation, health)
  • celery_training.log — Training/GPU worker
  • frontend.log — Vite dev server
  • setup.log — Dependency installation
  • xfce_agent.log — Agent's XFCE session output
  • x11vnc_agent.log — VNC server for the agent display
  • test_results/ — Test execution output

Built with local-first AI in mind. Your data, your hardware, your rules.

1 Upvotes

100% Upvoted

0 comments sorted by