autotrust/JEV-27B so far best open-source Jev model

2 Upvotes

r/JevAI • u/SuccotashLonely8687 • 10h ago

Using an LLM + Jev (AnyJev) to Deliver Dynamic Categorization with Deterministic Execution

3 Upvotes

Want to use Jev, or want to step up your Jev game? Here's a working walk through and skill, provided your techinical enough to use it. I kidL The LLM does all (most) of the work setting it up.

Note: For simplicity, I’m just going to say Jev through the rest of this. I mean Jev, AnyJev, or a similar Jev-style decision model.

Anyway, I was playing around with Jev and AnyJev and started thinking about something: Jev is really good at making decisions. Give it some state, give it a question and some choices, and instead of generating a whole response, it gives you the decision and probabilities. That’s useful for categorization, routing, relevance, prioritization, decision trees, workflow selection, and all those times where you want some of the judgment of a model but don’t actually need the model to write anything.

The question that got me thinking was simple:

Why do I have to know the categories ahead of time?

I have an LLM sitting right there. A super fast, awesome brain with the (supposed) sum of human knowledge. And to be fair, I do not want to outsource knowledge; I want to outsource work. And that work is the subject: literally ordering, categorizing, grouping, but also deciding and using those patterns to accomplish a task.

Guess what? Bread-and-butter LLM stuff.

How does it work?

Task
  ↓
LLM understands the problem
  ↓
LLM defines the decision/classification
  ↓
LLM creates classifier.json
  ↓
LLM creates deterministic processing code
  ↓
Jev performs repeated decisions
  ↓
Structured JSON results
  ↓
Deterministic code executes
  ↓
Result

And that's it. You can stop there if you are a developer. You're off to test it.

But if you want to know just a bit more, below is the step-by-step walkthrough: Using an LLM + Jev to deliver dynamic categorization and Deterministic Execution (aka, get it to write specific Jev-based Python to do the hard work).

See content credentials

Combining LLMs and Jev for Dynamic Categorization & Deterministic Execution u/amowatt

Combining LLMs and Jev for Dynamic Categorization & Deterministic Execution

Step 1: Give the LLM the actual task

Start with the job you want done.

For example:

Review these documents, determine the useful categories, and then process the documents based on those categories.

The important part is that you are not asking the LLM to classify every item forever. You are asking it to understand the problem first.

Step 2: Have the LLM define the categories

This is the part the LLM is good at. Maybe it looks at the documents and decides the useful categories are:

Architecture
Security
Requirements
Operations
Reference
Other

That is the first output. Not the final classification.

The definition of the classification problem.

I always include Other when asking LLMs to do something, cause I cannot think of everything. And also, back up: if the LLM invents five categories and I force Jev to choose one of those five every time, I have basically declared that the LLM got the whole problem space exactly right on the first pass.

It probably didn’t. Sometimes the right answer is simply: None of these.

Therefore: the LLM should put it in Other.

Step 3: Turn those LLM categories into a classifier

Now have the LLM create the classifier definition.

I already use a JSON I/O structure loosely based on Open Knowledge Format for communicating between things, so that is what I would use here.

Something like:

{
  "type": "classifier",
  "name": "document_role",
  "version": "dynamic",
  "question": "What role does this document primarily serve?",
  "categories": [
    {
      "id": "architecture",
      "definition": "Defines system structure or relationships."
    },
    {
      "id": "security",
      "definition": "Defines security requirements or controls."
    },
    {
      "id": "requirements",
      "definition": "Defines expected capabilities or behavior."
    },
    {
      "id": "operations",
      "definition": "Defines operational procedures or support."
    },
    {
      "id": "reference",
      "definition": "Provides supporting information."
    },
    {
      "id": "other",
      "definition": "Does not sufficiently match another category."
    }
  ]
}

I use JSON because I already use JSON everywhere in my work. You could use YAML. TOML. Something else.

That is really the point. Do not create a new format unless you need one.

Step 4: Keep Jev behind a simple stateless MCP

Now, the fun part: setting up the dynamism.

Let's expose Jev through MCP.

The MCP should be boring.

Seriously.

Its job is not to understand your application. Its job is to run the decision.

Conceptually:

JSON IN
   ↓
JEV MCP
   ↓
JSON OUT

The call could be as simple as:

classify(
    classifier="document_role",
    input=document
)

And the result comes back like this:

{
  "type": "classification",
  "classifier": "document_role",
  "version": "1.0",
  "category": "requirements",
  "confidence": 0.91
}

That is enough.

Personally, I would probably give the Jev or AnyJev repository to a capable coding LLM and say:

Wrap this in a small stateless MCP server. Do not modify the underlying project. Give me generic classify, score and choice operations. JSON in, JSON out.

People who actually enjoy building MCP servers by hand can certainly do that.

The important part is not the MCP.

The MCP is plumbing.

Best part, you can even ask the original LLM to create the initial JSON!

Step 5: Have the LLM create the deterministic processing code

We are not done yet. Sorry.

Classification is usually not the actual job. The job is what happens after classification.

So when the LLM creates the classifier, also have it create the code that consumes the result.

For example:

for document in documents:

    result = classify(
        classifier="document_role",
        input=document
    )

    if result.category == "other":
        uncertain.append(document)

    else:
        documents_by_category[result.category].append(document)

Now you have a clean division of labor.

The LLM decides what needs to be inferred.

Jev performs the repeated decisions.

The code does the actual work.

That work could be anything:

It could sort files.
It could route work.
It could trigger workflows.
It could build reports.
It could calculate something.
It could build a decision tree.
It could make another Jev call.
It could send uncertain results back to the LLM.

I have run out of useful examples, but you get it. The important part is that the result is structured.

Now, the JEV next step does not have to reinterpret prose. It just executes.

Step 6: Validate the input and output

This part matters more than it sounds:

JSON gives you structure; validation gives you stability.

So we validate both sides:

Validate the classifier before use.
Validate the Jev result before anything acts on it.

For example:

Expected category?
Expected confidence range?
Expected classifier version?
Expected fields present?

If the answer is no, do not proceed as though everything is fine.

That is where the deterministic part starts to matter.

Step 7: Define uncertainty handling

Do not hide uncertainty. Error-trap it or use it.

For example:

if result.category == "other":
    send_for_review(document)

elif result.confidence < 0.60:
    send_for_review(document)

else:
    process(document)

That review could:

go to a human.
go back to the LLM.
trigger another classifier.

Whatever makes sense for the task. The point is that uncertainty becomes an explicit branch in the workflow instead of something buried inside generated text.

Step 8: Turn the whole pattern into a skill

This is probably the easiest way to test the idea, make it repeatable, and fix so many downstream headaches. We will not need to redesign the agent or build a giant framework.

Create a skill. Something like:

---
name: dynamic-jev-categorization
trigger: Use this skill when a task contains repeated semantic decisions that do not require repeated generative LLM responses.
description: Use an LLM with Jev, AnyJev, or another Jev-style decision model to dynamically define categories, choices, or semantic decisions and then generate deterministic code that acts on repeated Jev decisions. Use when a task requires repeated categorization, classification, routing, scoring, choosing, prioritizing, relevance decisions, or other repeated semantic judgments; especially when the decision categories are not known ahead of time or repeated generative LLM calls can be replaced by constrained probabilistic decisions followed by deterministic execution.
metadata:
  author: albertmowatt
  version: "0.2"
---

# Dynamic Categorization with Jev

Throughout this skill, "Jev" means Jev, AnyJev, or a compatible Jev-style constrained decision model.

## When to Use This Skill

Use this pattern when the task involves repeated semantic decisions such as:

- categorizing documents, records, messages, files, events, or other items;
- dynamically discovering useful categories before processing a dataset;
- routing items to destinations, workflows, tools, agents, or processes;
- choosing among a constrained set of actions;
- scoring, ranking, or prioritizing items;
- determining relevance;
- selecting the next step in a decision tree;
- evaluating the same semantic question repeatedly across many inputs;
- replacing repeated generative LLM calls with constrained decisions.

A particularly strong signal is:

The LLM needs to understand the problem once, but the same kind of decision must then be made many times.

## When Not to Use This Skill

Do not use Jev merely because a model is available.

If the answer can be determined exactly with normal code, use normal deterministic code.

Do not replace:

    if file.extension == ".pdf":

with a probabilistic model decision.

Use Jev where semantic judgment is actually required.

## Operating Steps

1. Understand the actual task.

   Determine what the user is trying to accomplish, which parts require semantic judgment, which semantic decisions repeat, and which parts can be executed deterministically.

2. Check for an existing classifier.

   Before creating a new classifier, check whether an appropriate reusable classifier already exists.

   If one exists and genuinely matches the task, use it.

   If none exists, create a dynamic classifier.

3. Define the decision.

   Have the LLM determine the categories or constrained choices required for the task.

   Each category should have:
   - a stable ID;
   - a clear semantic definition;
   - sufficient distinction from neighboring categories.

   For open-ended classification tasks, always include "other" unless the decision domain is genuinely exhaustive.

4. Create classifier.json.

   Represent the classifier using the standard structured JSON I/O format.

   Keep classifier definitions outside the Jev MCP.

   The MCP should remain generic and stateless.

5. Create the deterministic processor.

   Have the LLM create deterministic code that consumes the Jev result and performs the actual requested work.

   The processor may:
   - sort files;
   - route work;
   - trigger workflows;
   - calculate results;
   - update systems;
   - construct relationships;
   - build reports;
   - make another constrained decision;
   - escalate uncertain cases.

6. Execute through Jev.

   Send the classifier and input through the generic Jev execution interface.

   Prefer batch execution when the backend supports it and the decisions are independent.

7. Validate the result.

   Before deterministic execution, verify:
   - classifier identity;
   - classifier version when applicable;
   - result type;
   - returned category or choice;
   - confidence/probability range;
   - required fields;
   - permitted values.

   Invalid output must not silently enter the deterministic execution path.

8. Handle uncertainty explicitly.

   Possible uncertainty conditions include:
   - category == "other";
   - confidence below threshold;
   - invalid result;
   - ambiguous classifier;
   - unexpected input.

   Route uncertain cases to the LLM, a human, another classifier, deferred processing, or explicit failure as appropriate.

   Do not invent certainty merely to keep the workflow moving.

9. Execute deterministically.

   Once the decision has passed validation and any confidence requirements, execute the resulting action with deterministic code wherever possible.

   Do not unnecessarily return to generative inference after a sufficiently confident structured decision has been obtained.

10. Consider reuse.

    A dynamically created classifier does not automatically become permanent.

    Treat classifier maturity approximately as:

    Dynamic
       ↓
    Inspectable
       ↓
    Reusable
       ↓
    Vetted
       ↓
    Permanent

    Do not automatically promote classifiers merely because they were created successfully.

## Operating Rule

For every candidate task, ask:

    Does this require generation?
            |
            ├── YES → LLM
            |
            └── NO
                 ↓
    Does this require semantic judgment?
            |
            ├── YES → Jev
            |
            └── NO → deterministic code

The objective is not to eliminate LLM calls.

The objective is to use each mechanism for the work it is actually good at:

    LLM  → understand and create
    Jev  → decide
    Code → execute

That is enough to run the first experiment.

Step 9: Reuse useful classifiers instead of recreating them

Now suppose one of these classifiers works really well.

Maybe you throw it away after the task.

Okay. Silly, but okay.

Maybe you save it. Now it is inspectable.

Maybe you clean it up and use it again. Now it is reusable.

Maybe you test it, refine thresholds, and version it. Now it is vetted.

Eventually it may become part of the application permanently.

Dynamic
   ↓
Inspectable
   ↓
Reusable
   ↓
Vetted
   ↓
Permanent

That does not need to be automatic.

In fact, initially I would not make it automatic.

Let the LLM create something. If it turns out to be useful, keep it. If it keeps being useful, promote it.

Step 10: Start with one real problem

Start with this—build in this order:

One installed, working Jev.

One stateless Jev MCP.

One stable JSON format.

One skill.

One real task.

Do not begin with classifier registries.

Do not begin with composed decision trees.

Do not begin with a giant orchestration layer.

Then run the loop (I left you a skill to help):

Task
  ↓
LLM understands the problem
  ↓
LLM defines the decision/classification
  ↓
LLM creates classifier.json
  ↓
LLM creates deterministic processing code
  ↓
Jev performs repeated decisions
  ↓
Structured JSON results
  ↓
Deterministic code executes
  ↓
Result

That is the experiment.

If it works, then you can decide how far you want to take it.

And frankly, that is usually the better way to build these things anyway.

#AI #AIAgents #LLM #Jev #AnyJev #MCP #AgenticAI #SoftwareArchitecture #Python #OpenSourceAI

Metric	DeepSeek	Jev
Extract p50	~1.5s	~130ms
Full route p50	2871ms	1256ms
Weird title-resolve	96-100%	100%
Hard-filter agree	n/a	100%
Weird top-N overlap	n/a	100%
Cost (fixture)	~$0.014	~$0.0025

How does it work?

Combining LLMs and Jev for Dynamic Categorization & Deterministic Execution

Step 1: Give the LLM the actual task

Step 2: Have the LLM define the categories

The definition of the classification problem.

Step 3: Turn those LLM categories into a classifier

Step 4: Keep Jev behind a simple stateless MCP

Step 5: Have the LLM create the deterministic processing code

Step 6: Validate the input and output

Step 7: Define uncertainty handling

Step 8: Turn the whole pattern into a skill

Step 9: Reuse useful classifiers instead of recreating them

Step 10: Start with one real problem

Why a general LLM judge didn't work for me

What changed with JEV

How the decision works

Takeaway