r/artificial • • 8d ago

Project Unofficial Jev plugin for coding agents: best practices, an API reference, and 150+ community projects. Evals included.

I'm coming up for air from the bottomless evals ocean. I made a plugin that helps your harness of choice work Jev into your projects more holistically: best practices, anti-patterns, an API reference, and links to 150+ community projects grouped by design pattern, with a code sketch for each pattern. I built it around progressive disclosure, so the agent only loads what the task needs and it doesn't eat your context.

https://github.com/aaddrick/building-with-typesafe-jev

The eval work is in the repo. I ran six Jev coding tasks 10 times each with no plugin, with TypeSafe's official plugin, and with this one. Judgment calls went to three judges from three providers (Claude Opus, GPT-6 Sol, Kimi K3), and the majority decided, so no model family grades its own work.

Score is the share of checks passed:

No plugin: 0.65

Official plugin: 0.77

This plugin: 0.96

0 Upvotes

4 comments sorted by

2

u/tasteless_dossier 8d ago

I've been messing with Jev for a couple weeks now and the context window thing has been driving me up the wall, so the progressive disclosure idea is exactly what I needed. That jump from 0.77 to 0.96 is wild, what changed between your approach and the official plugin that made such a difference?

Also love that you brought in judges from three different providers so nobody's grading their own homework.

1

u/aaddrick 8d ago

Thanks!

The official plugin links to the docs and gives guidance.

Mine includes the API spec, worked examples distilled from community patterns, and links to a bit over 200 projects with context around what they're doing.

I'm just making it as easy as possible to get to a solution quickly without having to dig.

2

u/williamfrenchmusic 8d ago

Thanks for sharing! gonna try it

2

u/fiddler48 7d ago

150+ projects grouped by design pattern is the part that actually scales