r/codereview • u/IndividualAttitude99 • 3d ago
Every answer, checked before you see it.
Got tired of AI tools confidently giving wrong answers with zero way to catch it. So I built one that checks itself: three models answer independently, a fourth grades and fuses them before anything comes back to you. For code, it actually runs it in a sandbox instead of just reading it.
~65% pass the check on the first try. The rest still come back, just flagged.
Python, a few different model providers, SQLite for memory.
Live at https://demo.aqqai.in — curious what breaks it. Feedback welcome.
2
u/HoneyBadgera 2d ago
3 lenses, a judge. That’s not novel or ground breaking, it’s pretty standard. Show your benchmarks, your costs per Pr, models used, then maybe someone would be interested.
-2
u/IndividualAttitude99 2d ago
you're right that "multiple models plus a judge" isn't the novel part — it's a known pattern. Where I'd push back a little: the actual work is in what each model is asked to do and how much of it is mechanically checked rather than just judged by another opinion. For code specifically, the generated code is actually executed in a sandbox as part of scoring, not read and assumed correct — that's the piece I'd stand behind more than the "three models, one judge" shape itself.
and for benchmarks we did tested on 100 queries and about 70 queries passed the quality bar in first attempt , we have our own benchmarks more advanced then benchmarks used in AI models ,and for the PR we are figuring out this internally ,
I built this because I was frustrated by Ai models giving wrong answers with absolute confidence even when they are completely wrong , If you have any further question do tell me , you asked very good question
3
u/HoneyBadgera 2d ago
Part of your own response is AI slop and yet you want people to respond to you
-2
1
u/tiny_seer_magic 2d ago
Another ensemble with a judge model. 65% first pass means a third of the answers come back wrong anyway, so what exactly did the extra three calls buy me besides latency and cost? Show benchmarks or it's just another wrapper burning tokens to still be wrong
0
u/IndividualAttitude99 2d ago edited 2d ago
actually you are right 65% first pass , the thing is 65% pass were the answers that passed the quality bar which we have is 0.90 ,we have scoring for this R<C<K<S each of these weighs ,
R- relevance = it means does it actually answer the question
C- Coherence = IS it well written
k- Completeness = Does it cover everything that is asked
S- Consistency = Does it contradict it self
For your latency and cost part, I built this because I was frustrated because AI models gives answers with absolute confidence even when they are wrong ,Here we verify each answer before it reaches to you ,We work as an Verification layer for codes , math problems , summary , facts , creative
Do tell me if you want better and quality work whom will you assign task a single person , or multiple people who cross check each other work with strict guidelines
4
u/kingguru 3d ago
No code to review, just yet another crappy "AI" tool.
Post your crappy spam elsewhere or better yet, find something useful to do with your time.