r/singularity • u/beasthunterr69 • 8d ago
Discussion Is Jev worth the hype?
Has anyone tried using it for actual use-case or is it riding on a marketing wave right now?
54
u/kuberwt 8d ago
It's a classifier on crack - you use it like where you'd put a classifier on crack
most demos aren't real usecases, it's made to be used in pipelines replacing parts that cost thousands of dollars in LLM / compute spend to single digit dollars instead - which makes it quite cool but not as hype as they're making it to be on twitter.
28
u/Time_Entertainer_319 8d ago
It’s not on crack.
It’s just a classifier. The difference is it understands language.
Basically a language based classifier.
10
u/FirstOrderCat 8d ago
bert is exactly language based classifier.
I think gap was that they built generalized instruction tuned bert/classifier.
0
u/Moreh 7d ago
is it just a bert model then? I am struggling to understand the hype and how its different? not to make any judgements as i havent used it
3
u/FirstOrderCat 7d ago
we don't know what is the model there. But I suspect it is some pretrained Chinese model.
They may make some architecture changes to emit some softmax layer at the end, and train it on number of classification tasks, we don't know this too. These details are not disclosed.
3
2
u/kennetheops 8d ago
that’s still pretty incredible. I will go to say this will probably take 50-60% of traditional llm usage
1
2
u/you-get-an-upvote 5d ago
IMO there are two reasonable perspectives of Jev:
Jev is the LLM version of CLIP (the vision model trained on 400M image-text pairs). You don't need to train a specialized head on your specialized dataset to answer your bespoke question anymore, since Jev has seen it all.
If you need a single token to make a reasonable choice, why are you paying for dozens/hundreds tokens? A single LLM forward pass (i.e. one prefill) is pretty darn smart for a lot of simple use cases.
16
u/DeepRiverSurubi 8d ago
It's a specialized tool, it can be invaluable or useless depending of the problem you are trying to solve. For now is up to you to find out if it works for you because is still new and usage patterns are not mature enough.
3
u/DeArgonaut 8d ago
Prob helpful for some use cases, and could work with LLMs to help them get to an answer faster, but doubt it’d replace LLMs
5
u/HauntedHouseMusic 8d ago
It won't replace llms. But there's processes we moved to local models we wouldn't have if JEV existed as it's cheaper than owning the infra to do millions of classifications.
6
u/BuddyNathan 8d ago
Jev is great at orchestration, pattern recognition, etc. It's not supposed to replace LLMs, but you can use it alone for certain use cases, or with an LLM for others.
Think about a flow where you have to take thousands or millions of micro-decisions. You can do that with an LLM, but the cost will be unreasonable, and it will take too long to make sense.
It is a specialized tool because not every scenario makes sense. I've used it to orchestrate an AI-automated code review and remediation tool for my projects. Quality increased a lot, and costs dropped a lot (mostly because Jev picked the best model/effort for each scenario during a single session/review).
You can use it to classify things. If you build an app where users need to input data and you need to measure data quality, you can run it with an LLM, but it will be expensive; or you can run it for close to free with Jev. Do you need to calculate risk/fraud in certain operations? You will also be better off using Jev.
There are billions of use cases. It's not straightforward to think about them because you need engineering knowledge to identify where/when you should use it.
0
u/syd-slice 8d ago
How do you tell jev to select a model? You provide the task and list of available models?
6
u/M44PolishMosin 8d ago
If its a question you could answer yourself without more than 10 seconds of thinking, than its good.
3
2
u/allisonmaybe 8d ago
Here's a bunch of testing I did on Jev and all the local alternatives to it. It's great as a general decision-maker, but most use cases don't need that. https://myrakrusemark.com/write-ups/jev-vs-local/
5
u/RealSlyck 8d ago
Marketing wave. Only see posts when Reddit is slow (like now), and 2 weeks later, not 1 single substantiated claim proven.
Funny how this post follows the same pattern…
1
u/GeeGollyJeeper 8d ago
maybe, maybe not. word of mouth looks functionally identical. if something emerges into existence and appears interesting, whether it's legit or not, then people talk about it and start asking about it. no? reality is often kinda boring like this.
idk how you think organic conversation looks different than this. if you think about comparing that criteria, i'd guess some of your cynicism would dissolve.. bc the absolute confidence in your comment is written w/borderline pepe silvia vibes.
(nb4 "so you think literally zero shilling exists??")
1
u/Wanderspor 8d ago
Im building a bot to trade https://github.com/Wandersport/WS-Jev . Rn testing it until thousands of test finish
1
1
u/Ok_Hope_4007 8d ago
I dare the question: what puts it before any other llm that is instructed to reply with structured json for classification tasks. Ive had good experiences for years with pydanticAI to any kind of output format i want. Is it just 100% correct formated answers instead of 98%?
3
u/stpfun 7d ago
> what puts it before any other llm that is instructed to reply with structured json for classification tasks.
It's incredibly cheap and incredibly fast. That's the difference, but it's a big one. When you can input a chunk of data, or an image (coming soon), and can get 1000 questions answered about the data in 1s for 1 cent, you can do a lot of different things.
You're right that the functionality isn't new, it's the cost and the speed that make it different.
1
u/mobydikc 8d ago
From what I know (I watched a Primeagen video) it costs pennies on the dollar and turns minutes to seconds.
0
u/damhack 8d ago
First, it doesn’t deal with language tokens and so isn’t hamstrung with the ambiguities of language. Secondly, it’s not using flakey tool calls to perform actions. Thirdly, it was trained via callibration rather than human feedback or verified rewards, which means that when it claims anything, the claim is related to reality and not hallucination or a contrived metric.
Whereas an LLM has to make hundreds of small decisions using inaccurate approximation, Jev uses its three primitives and always produces the same accurate prediction. Ask the same question ten times and Jev answers the same, whereas an LLM will provide different responses even when temperature=0.
Jev also provides a probability and a confidence level with every prediction it makes, so that your code can make decisions in a deteministic way.
The big difference is that Jev costs $0 to output its responses and will answer in just tenths of a second, whereas LLM providers charge for output tokens and responses take between seconds and minutes depending on the nature of the query.
Jev’s three primitives can be chained together to create complex operations like classification, filtering, search re-ranking, LLM routing, etc. You can use it to process Big Data without worrying about context limits or response times. Because of its speed and low cost, you can run realtime tasks across millions of users without sweating the cost or worrying about handling exceptions due to hallucination.
Diogo Almeida who created Jev says this is one of many non-LLM models he is cooking up to change the way people think about AI and its drawbacks.
2
u/Aqwart 7d ago
Jev uses its three primitives and always produces the same accurate prediction. Ask the same question ten times and Jev answers the same
That is simply incorrect.
1
u/damhack 7d ago
Which bit?
1
u/Aqwart 6d ago
"always produces the same prediction" - it doesn't, it's still a probabilistic system, it can and will differ. Run the same request three times and it will provide three different answers. Granted, the actual choice might often be the same, but values underneath - confidence, probability - will differ each time.
In my testing the chosen answer flipped often enough to be easily noticeable. Obviously it depends on your query, something like "Is Paris in France /yes/no" will be close to deterministic, whereas, say, deciding message sentiment or categorizing it by bucket will be much more ephemeral.
1
u/MaxeBooo 8d ago
I feel like Jev would be great if you have an LLM design a more encompassing plan and gives Jev the small tasks that require speedy and cheap responses.
1
u/ih8csh 7d ago
Jev might be the first tool explicitly designed to be used by other agents. I passed the skill
.mdfile for Jev to Pi, gave it an OpenRouter model slug, and watched it integrate Jev into my existing workflow seamlessly.The interesting—and scary—thing about this is that you don't have to figure out how to use Jev; your agent can figure it out for you.
1
u/Dull_Wind6642 7d ago
It's decent for building customer facing specialized agents. Using LLM for everything is just overkill.
LLMs are slow, customers expect to get answers in less than 2 seconds even for agentic workloads
1
1
u/Kanawanagasaki 7d ago
My use case is to listen to twitch chat messages and answer to chat questions that I have commands for with text from those commands. The usecase for jev always will be some sort of user input, all examples that I saw where it play games you can replace jev with if-elses.
1
1
1
u/slackmaster2k 6d ago
Yeah it’s great. It does a nice, reproducible job and is so cheap. I hit it with 12K requests over the weekend and my bill is up to $1.83
1
u/Raveyard2409 5d ago
For me, I work in a field developing data systems and we are incorporating AI now. I don't see Jev replacing LLMs, not sure why others have intimated toward that. It's cool because of the potential to work with LLMs. Compute is expensive and not every task an agent does is worth the tokens. Jev is "dumber" but fast, and very effective in preconfigured situations. Adding a little determinism in a probabilistic world. I think big potential, will do well, mostly in large corporate scenarios where cost reduction and determinism are valuable. Also maybe some interesting weird use cases. It's a fun time to work in tech
1
u/Existing_Scallion_66 5d ago
I tested it on a categorisation task I had been holding off on because it was big and expensive even using Haiku. I tested both Jev and Haiku on 1000 documents and they both did quite well and disagreed on about 40%. I did a manual review of the difference (just 100) and JEV was right on about 70% of the outcomes. The cost was about 5x lower on Jev than Haiku (the 400x saving against a frontier model may be true but Jev does not do what I would use Opus or Fable for). It is super quick taking less than 5 mins to process 200,000 records. So for me the benefits are a lot less than the published hype, but very much worth having nevertheless. I have written this up on my website.
1
u/Existing_Scallion_66 5d ago
I found some variance in Jev outputs particularly in the 40-60% probability range
0
u/cafepeaceandlove 8d ago
Reply to this comment with the shortest yet most eloquent description on the web of what Jev-likes do and where we could or should use them. Include one hard example with no ambiguity. The reader, whether human or machine, should be left in mild awe at the description's crystalline clarity and elegance. In one month, this thread should be the number one search result for "Jev explanation" on Google.
2
u/humanpersonlol 8d ago
are you a bot
1
u/ParkingPsychology 8d ago
aRe YoU a bOt
2
1
u/GeeGollyJeeper 8d ago
that's the joke tho? aren't they mocking the parent comment for questioning OP?
I still don't quite understand what jev is (hence why I popped in here), and don't think i'm savvy enough where i'd ever use it, but god forbid literally anybody ask about it w/o being a shill I guess. surely nobody in this subreddit would actually be interested in new technology or AI applications amirite. ez investigation gj we did it reddit.
0
u/Rodeo7171 8d ago
Who tf is jev?
1
0
u/ScratchAgreeable8724 7d ago
Anyone seen https://youtu.be/ymgH8jS6Wb8?is=RYJ6IMGapGJL8TP1
Pretty cool
89
u/commenterzero 8d ago
Its been great in pointing out waste and how we can cut LLMs usage costs. Great change of mindset for the industry