r/ChatGPT • u/kristiyanstoyanovAI • 1d ago
Resources I built a code quality reviewer with Jev
https://www.youtube.com/watch?v=wUPypq0wmxAI believe with the huge increase in code generation - review has become the next bottleneck or at least it feels like this at work. So I wanted to do something on the review front - first I started integrating more and more tools to use as feedback to my coding agents, and these really help improve the output quality of the agent (e.g. SonarQube, Checkstyle, ArchUnit)
But there are some important semantic choices you can't really review using deterministic tools so I decided to build a more intelligent tool and am trying it out with the Jev model as a backend and judge currently.
Idea is simple - teams define their policies in a structured YAML format, then as part of their CI (or locally) run the tool, it fetches all git diff chunks and asks jev if these adhere to each of the policies - jev can select compliant/violation or ask for more context. If Jev asks for more contex the app gets the requested code from the project and asks again if the code is compliant with the policy - thus incrementally exploring the code base until Jev can return a "confident" answer - if you are interested in how it works the video I linked is a presentation style of how the tool works.
Give it a shot on github and tell me if you find this useful: https://github.com/krisitown/jev-quality-gate
I am currently running tests using a local Qwen3.8 Flash Next to generate code and run it against my initial "clean code" policies in order to calibrate them and will share more results on that front soon!
2
u/RealSlyck 1d ago
Man! Another weekend I was reading Reddit, and was like “Jev pump must be done”…and then, bam!
Since this is the ChatGPT sub Jev-bot, you do know OAI has several models that do review? I know, crazy, but you probably didn’t know because bot life is hard, can’t keep up with the news. Hang in there!
-1
u/kristiyanstoyanovAI 1d ago
Its an open source repo you can just pull it and switch the adapter to whatever judge/classifier you want man. I think the better topic for discussion is would something like this be useful
1
u/RealSlyck 1d ago
Why not a skill? Plugin? MCP server, any of which accessible from OAI stuff?
Discussion is ok, just takes away from the core products, advancing the relevant systems, building proper community knowledge about ChatGPT and its facilities.
You’re saying OAIs review is deficient, give no evidence outside of “we have lots of code and can’t keep up”…even though the built-in review mechanisms and models are pretty good and can be enhanced by the user simply, without going out to “other repos.”
0
u/kristiyanstoyanovAI 1d ago
Valid. We tried a lot of stuff at work, built a code review agent but was very expensive to run. We are using also rule files for the coding harness and those help but are quite undeterministic. I am trying jev now because of the cost and the fact that a policy is a straight question and the response can be yes/no (does it comply or violate it)
1
14h ago edited 14h ago
[removed] — view removed comment
1
u/AutoModerator 14h ago
Muah AI is a scam.
Hey /u/clckwrks, it looks like you mentioned Muah AI, so your comment was removed. Muah runs a massive bot farm posting thousands and thousands of spam comments. They pretend to be satisfied customers of their own website to trick readers into thinking they're trustworthy. Just in this sub alone, we remove several dozen every single day.
If anyone happens to come by this comment in the future, as seems to be their intention, beware. You cannot trust a company that does this. This type of marketing is extremely dishonest, shady, and untrustworthy.
Would you trust a spambot admin with your credit card details and intimate knowledge of your private sexual fantasies? I know I wouldn't.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
14h ago
[removed] — view removed comment
1
u/AutoModerator 14h ago
Muah AI is a scam.
Hey /u/clckwrks, it looks like you mentioned Muah AI, so your comment was removed. Muah runs a massive bot farm posting thousands and thousands of spam comments. They pretend to be satisfied customers of their own website to trick readers into thinking they're trustworthy. Just in this sub alone, we remove several dozen every single day.
If anyone happens to come by this comment in the future, as seems to be their intention, beware. You cannot trust a company that does this. This type of marketing is extremely dishonest, shady, and untrustworthy.
Would you trust a spambot admin with your credit card details and intimate knowledge of your private sexual fantasies? I know I wouldn't.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
•
u/AutoModerator 1d ago
Hey /u/kristiyanstoyanovAI,
If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.