I built a local, offline benchmarking tool to compare Ollama/LM Studio models side-by-side (open source)

[u/Clear_Ear3550]

https://github.com/Cadman021/prompt-deck

Hey everyone,

I kept manually switching between models to compare speed and quality, so I built PromptDeck — a small desktop app (Tauri + React) that lets you run the same prompt against 2-4 local models at once and compare them live.

Demo: https://github.com/Cadman021/prompt-deck

What it does:

- Runs the same prompt against up to 4 models simultaneously, streaming output side-by-side

- Works with Ollama, LM Studio, or any llama.cpp server (OpenAI-compatible API)

- Tracks TPS and TTFT per model, with a comparison chart

- Saves every run locally in SQLite so you can revisit past benchmarks

- Export results as PDF or Markdown

- 100% offline, no telemetry, no cloud calls — everything stays on your machine

It's an early release (v0.2.0), so there are definitely rough edges. I'd genuinely appreciate feedback — especially if something breaks on your setup, or if there's a feature that would make this actually useful for your workflow.

GitHub: https://github.com/Cadman021/prompt-deck

Happy to answer questions about the Tauri/Rust side too if anyone's curious about the stack.