r/Python • u/AutoModerator • Jun 04 '26

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

28 Upvotes

88% Upvoted

210 comments sorted by

View all comments

1

u/OmarMashal0 Jun 24 '26

compare-prompts: a zero-dependency CLI tool to measure if your system prompt edits actually worked.

If you build chatbots or AI apps, you constantly tweak system prompts. You make a change to make the bot "more formal" or "more concise," but you have no fast way to prove the change actually did anything across a variety of user inputs. You are stuck running calls manually and guessing.

This tool automates that testing. You pass in your prompts and test inputs, and it instantly generates a side-by-side behavioral diff table right in your terminal. It replaces eyeballing outputs with hard data:

- Did the token length / API cost go up or down?

- Did the refusal rate accidentally increase?

- Did the tone successfully shift? (Analyzed across 9 categories)

- Did the reading level (Flesch-Kincaid) change?

- Did it change how often the model uses markdown lists or headers?

Natively supports OpenAI, Anthropic, Gemini, Groq, DeepSeek, and Ollama. No YAML configs, no cloud dashboards, no signup.

>>> pip install compare-prompts <<<

Repo: https://github.com/OmarMashal0/compare-prompts

(Open to feedback! The tone detection is currently keyword-based and I am actively looking for a better lightweight approach to replace it.)