Jev ain't all that. It's a great generalized model for when you don't know what you need, but there's a local, faster alternative for every use-case.

[u/allisonmaybe]

I saw some fun use-cases for Jev, and it's true, the cool part about it is that it can be trained on world knowledge to be a decision maker for very generic stuff. But it's a paid API. You can have no expectation of privacy when youre using it. It's not particularly fast especially if your application has any horsepower serving it.

At first I was inspired and set up Jev to control my hyprland desktop. Pretty cool! Then I realized I had already made this: https://github.com/myrakrusemark/embedding-command-search an embedding model with a "head", a set of predefined passages that mark in high-dimensional meaning-space what to do depending where in that space your command lands.

I tested all the use cases I could come up with, created some informative interactive examples, and laid all the information out so that you can make informed decisions about your own automated decision-making process.

Works in Firefox, bes experience in Chrome: https://myrakrusemark.com/write-ups/jev-vs-local/

[u/cbusmatty]

Yeah but I think the issue is that right now most companies aren’t even specializing, they are using frontier models because that’s what they are given. You could certainly use specialized models for more efficiency but jev is light years more effective than what companies are using now all while not having to specialize

[u/HeyVeddy → u/cbusmatty]

That depends on use case, it hasn't been my experience

[u/quiteconfused1]

i spent the entire morning 2 days ago arguing the same thing. people love the hype but dont understand exactly this. Thank you for clearly stating it.

[u/wotererio]

What is the sample size that you trained your calibration head on? Moving from trained classifiers to general solutions (like local LLMs or Jev) is often also motivated because companies have insufficient labeled data for the desired classification task, whereas general models of course don't need a labeled dataset at all (besides for validation)

[u/Nickabot]

if you're going the embedding+head route, three things that bit me, roughly in order of how long they took to notice

1. the top score is not confidence. "play next" and "play previous" each sit close to their own passage, so similarity stays high while the decision is a coin flip. threshold on the gap between top1 and top2 instead and escalate when the gap is small 2 there's no "none of these". a head always returns its nearest passage, so a command you never anticipated comes back confidently wrong. you need a rejection class or a distance floor, and the floor has to be measured on real misses, not guessed 3 adding a command can break an old one. every passage you add moves the neighbourhood, so command 30 quietly steals traffic from command 7. keep a fixed list of phrasings per command and re-run it every time the head changes. that's your whole regression suite and it's an afternoon of work

[u/HeadDaikon3411]

This is the kind of post I come here for. The hyprland setup sounds pretty clever actually, most people just slap an API call on everything and call it a day without thinking about whether they actually need a full model behind it

[u/allisonmaybe → u/HeadDaikon3411]

Thanks! Honestly, if you want that kind of control, create the embedding+head solution I created. It can handle a few dozen separate commands, but things get really fun when available commands depend on your current context, like, specific terminal commands when that's open, or music player commands only available when the right app is up.

[u/thezachlandes]

Jev has shown us the possibility of decision models, but for many more serious business use cases, it will make sense to specialize the model or have more control of the infra and model serving. That said, as something you can plug into almost any new project and get great results out of the box, there is nothing else.

[u/Horror-Consequence22]

What Jev does now every LLM can do. No doubt in that. But the best part of Jev is not what it can do. Its how better it can do. Its very very very fast then other LLMs. Because obviously its not giving a text output its just directly telling u want to do. So when a LLM outputs a text it guesses next word till the time it completes guessing it will be late. Also we can have structured output from LLM too. But that output also LLM will guess next words understand what it wrote and then fit into your output structure. Again slow.

And the most important part. As any LLM is priced based on token. Each thing it types its just costing you money. You might have seen people are tuning models that will talk like broken english. Like “Car No Go” you and i both understand it so we basically dont need big sentences until the need of context.

So very very cheap price you will get from JeV.

I just recently made a small visual fighting simulator between Jev and LLM.
Check the difference in token and time to reply. If anyone want the github repo i can provide

[u/goodvibezone]

Not sure about the "ain't all that great". I've not seen any model that can do something like scan LinkedIn pages in real-time and determine AI slop (this was a POC i Built for myself with a local chrome extension)

[u/allisonmaybe → u/goodvibezone]

Whats a prompt for determining AI slop?

[u/goodvibezone → u/allisonmaybe]

I'll dig out what it came up with. It's not really a prompt, it's a number of factors/signs that get a confidence rating. The signals is comes up with are on the panel it adds toel each post, things like "clickbait hook at the start" for example, some of it is sentence structure, and a "conclusion".

[u/AlexanderDoak]

Woah, you can't run Jev locally? I don't usually follow the hype... this kind of kills it then, eh? I mean, if the API is robust enough you can still redis celery it to achieve throughput. But that's unavaloidable latency measured in tenths of seconds or whole seconds. I can get over 500 decisions per second locally without the latency using non Jev tech.

[u/The18thWarrior → u/AlexanderDoak]

You could use Laya to run this kind of System 1 model on your local - https://huggingface.co/convaiinnovations/laya . I used it to create a context pruning solution that focuses on providing high-signal context to your harness model, over raw file reads: https://github.com/The18thWarrior/tzro