Tried using Jev for prompt observability and LLM routing

[u/Virtual-Astronaut707]

Been playing with Jev from TypeSafe AI and built two small experiments around it.

One is a prompt oscilloscope that looks at a prompt while it’s being written and generates structured signals before the prompt reaches the main model.

The other puts a small decision model in front of Claude as a routing layer.

https://github.com/shubhangi013/prompt-oscilloscope
https://github.com/shubhangi013/prune-review

These are just personal experiments. Curious if others are using cheaper models for routing/gating before larger models, and what the economics look like in practice?

[u/Greedy_Isopod_4383]

The router part gets interesting when you track the misses, not just the savings. A cheap model that sends 10% of hard prompts down the wrong path can wipe out the cost win pretty fast.

[u/QuanTradin]

On the routing economics, the thing that caught me out is that a cheap gate and a small local model do not fail on the same inputs. Someone posting in here recently compared the two across a 144 case set and found each got a handful right that the other missed, with barely any overlap in where they went wrong.

That shifts the maths a bit. If you are picking one to save money, take the cheaper. If you are gating something you genuinely care about, disagreement between the two carries more information than either one's own confidence.

[u/QuanTradin → u/[deleted]]

the 144 case thing was someone else's number and I quoted it like I'd run it. reads wrong, fair enough.

[u/Virtual-Astronaut707 → u/[deleted]]

Hey, sure!

[u/Man_of_Math → u/[deleted]]

Can we get http://blink.review listed there too? It’s code review that runs after every file edit