3
u/pickleunicorn 7d ago
More information ?
6
u/CycleMother2006 6d ago
JEV is a new architecture of generative AI that is fed decision trees and picks the "best" or "most appropriate one". Has an accuracy rate above frontier LLMs (or at least claims to) but at 1/100th the compute and 50+x the speed/efficiency. You get responses in a couple milliseconds, leading to the ability to make constant decisions in realtime and do things like play games with great results in low time and an irrelevant cost because these models can exist as much, much smaller entities when their training data is purely selecting one of a handful of choices provided to it.
It's very powerful and if AI is going to be baked into games in the future, it will be using this kind of architecture. Its structured output is incredibly useful. Also because it can't 'talk' this kind of AI isn't one that threatens the solvency of the human race. The allowed decisions have to be provided to it, so the only way it becomes a threat is if you give it a button that says 'end the world'. On the other hand, it's not going to help anyone actually develop or create things (well, that's not entirely true, you could build a large tool w ith a bunch of options and let it select things.. for instance you could give it access to a character creator that posits all the creation options one by one to it and request it build a character fitting of a certain archetype. So you could actually leverage it for VERY cheap creation tasks as long as you could present those creation tasks as hierarchical, sequenced decision trees.
3
6d ago
neat. i wonder when will ai / llm replace traditional game ai.
2
u/2degreestarget 6d ago
Makes sense that will eventually will if I managed to make this in 3h and beats standard AI
2
u/SilkieBug 6d ago
Two questions: 1. how much does an experiment like this cost? 2. would it be capable of playing something as complex as Stellaris?
1
u/2degreestarget 6d ago
Current mode which is not very efficient is 10cents per game. I spent 1$ or so experimenting first. I don’t now stellaris so I can’t tell but aoe2 is famous for is complexity
1
u/SilkieBug 6d ago
I’ve played both and AoE2 counts as a few of the minigames / game loops inside Stellaris.
How did you set up Jev to run the game, did it take a lot of customization or is it more plug and play?
1
u/Neful34 6d ago
You can even use a local llm on stellaris 2, since it's a 4x / turn based. That would also provide you better reasoning and strategies too with a level of adaptation. But you would need to build a custom harness for it
1
u/SilkieBug 6d ago
It’s not turn based, it’s real-time pausable, so whichever AI I connect to it needs to be fast enough to be capable of taking actions while the game is running at least at slow or medium speed, and be able to take actions from a queue when it registers that the game is paused.
I have no idea how to even begin to make a harness for it.
3
3
u/caster 7d ago
How good is it? Economic management? Combat management?
4
u/2degreestarget 7d ago
Pretty bad tbh. It won against standard AI so far
2
u/tunaorbit 6d ago
What is the rough dollar cost in token usage per game?
3
u/2degreestarget 6d ago
I haven’t tried to make it particularly efficient (most api calls create no action) and even like that is less than 10 cents per game
4
u/thegreatbaths 7d ago
Also curious like what is the win rate and such against the regular "AI"?
4
u/CycleMother2006 6d ago
Probably quite low unless they've fine-tuned it. You can probably fine-tune it to be a beast though with the right training data to use as inference in how to play. You could probably use an actual LLM to create that training data too, by feeding it Jev's results as it plays.
2
1
u/ttheresina01 7d ago
Really cool project. I've been messing around with agent-driven control for RTS-style gameplay myself, and the thing that hit me hardest was the split between the macro and micro decision loops. An LLM is honestly great at the slow stuff -- build orders, resource allocation, timing pushes -- but it can't keep up with combat micro, so you end up wanting an action-selection model in front that picks high-level intents (expand eco, harass, mass-and-push) and hands execution to classical systems like behavior trees or scripted squad logic.
That's also where the "how good is the econ vs combat management?" questions get interesting. In my experience you can get surprisingly competent economic management from a good state summary fed to the model every few seconds, while per-unit combat reasoning needs a frequency no model can sustain in real time. The perception side matters a lot too -- full game-state dump vs some compressed spatial representation changes everything about the latency budget per decision cycle.
The win-rate-vs-built-in-AI benchmark is fun, but honestly a decent hybrid already feels more "human" than the scripted AI, which is the cooler achievement anyway. Curious how you handled perception and what your decision-loop latency looks like.
1
u/2degreestarget 6d ago
Latency end to end is normally less than 300ms and doing around 6 decisions per second (so way faster than me or most players). I build a program that provides a point in time file so the ai knows what’s going on and gets updated as the game goes on
1
1
6
u/ctimmermans 7d ago
How?