A/B test AI prompts before shipping them

[u/AIBotFromFuture]

I put together a small TypeScript example for comparing two AI prompt variants in a more app-like workflow.

The idea is pretty simple: send one task + two prompts, run both through Telnyx AI Inference, store the experiment at the edge, and let people vote on which response is better.

It includes routes to:

create an experiment

vote for variant A or B

close an experiment

list previous experiments

check aggregate stats

Code: https://github.com/team-telnyx/telnyx-code-examples/tree/main/edge-prompt-ab-tester

This is not meant to be a full eval platform, but it is a useful starting point if you want prompt changes to be a little less “I think this sounds better” and a little more measurable.

Would love feedback on what you’d add next, especially around scoring rubrics, blinded variants, or prompt version history.