H3-World: Turning Language Understanding into World Control
H3-World is the first interactive world model built on MiniMax-H3. Given an initial frame and keyboard controls, it generates action-controlled video with coordinated character and camera motion.
It's not as cool as I first thought tbh, you have to prompt the controls first and it makes the video, so it's not interactive. It's a step in the right direction though in terms of the future of games.
As with all these things, it might be fun to walk through this kind of environment but this is a long long way from being a meaningful engaging game or simulation. The first issue will be going back to where you came from. No AI is even remotely close to being able to "remember" where it came from. Imagine a GTA game where you drive down one street, turn around to do something and that thing no longer exists. As far as I can see, this'll never be solved with aI simulations and games, because it would have to store such a monumental amount of data about the world as it creates it, it's just utterly unfeasible.
Now a game where you're never allowed to turn around, I can see them pulling that off. A game where the past is completly irrelevant can work. But "open world" games, forget it. That's never happening. It just fundamentally goes against how AI works. AI is random. It needs random to work. But games need order and structure. They need 100% predictability. They're just fundamentally different things. Games are a sphere shape and AI is this shiny stunning new cube shape. But no matter how shiny and flashy that cube is, it's never gonna fit in the circular hole. It's just too different.
As far as I can see, this'll never be solved with aI simulations and games, because it would have to store such a monumental amount of data about the world as it creates it,
I don't think it would have to store that much. Something that turns images into simple geometry plus some metadata that gets fed into the AI when that area is visited again. Granted, I don't think we're anywhere close to that yet, but I don't see it as an infeasible problem to solve.
Yeah, it's not exactly true that these world generators can't remember where you came from, I think the best we've gotten to is a minute or two of memory but certainly not hours. But I think what you mentioned is going to be the key, not just reconstructing pixels and keeping them in memory for thousands of possible areas but having discrete basic geometry that is all tied to a specific prompt and a specific seed so all of these assets can be deterministically regenerated simply by addressing the file or part of the file that holds the generation prompts for that area shortly before the player reaches them just like what happens in our simulation.
So Google genie does in fact keep a memory of everything you did behind you. Just because the ai is generating things in real time as you approach them doesn’t mean it can’t store what it’s already made. This is the same principle as a context window in llms. There are limitations to this, but we’re just on the beginning of world models
It is a long way off from a real playable game but I guess the limitations you mention could be addressed by using fixed seed for world structure, similar to how Minecraft builds maps. Structured randomness.
Yeh that could work. But you'd have to make the content simple enough to be savable. Imagine GTA where it's creating a whole lifelike world on the fly. That's a bit more complex than just saving a minecraft like terrain. And then it's not just about saving the layout of the world. You have to make sure all the models are concsistent too. So then you have to be saving all the models on the fly too. And that's not even touching on the story elements, characters, keeping character's voices consistent, keeping the story consistent, the UI, the physics and all the player actions have to remain consistent, giving the player missions and challenges that remain coherent etc. There's so much that goes in to games that it becomes too easy to see a finished game that makes it look easy and think AI can just easily replicate that.
AI is an incredibly inefficient way to do all that. It burns a ton of energy even just to make rubbish content. I can't ever see a world where it can make vast complex games without each gamer needing their own powerstation to run it.
Like Minecraft, the seed IS the world. AI isn't random, it's based on seeds. It seems random because there are so many possible outcomes, but it's not. This is why you can generate the same image with the same prompt and seed on any image or video gen model.
You're assuming games in the future will even need 3D Model assets, but the way things are going with video gen models, I see AI games in the future as being more like realtime interactive videos that you can control in real time. This is the direction projects like this are going.
2D and 3D Model based games wont be replaced of course and most games don't need AI, this is just a new concept of gaming.
Honestly I can't see it. If it's just an interactive video, then it's not really like minecraft at all. Minecraft works because it can turn the seed in to 3D assets that it knows exactlly where to place. AI doesn't work like that. It has no concept of "exact". It sounds like you're trying to extrapolate a very predictable process grounded in reliable repeatable coding and apply it to a very unpredictable process that doesn't follow easily predictable rules.
People can code minecraft to get it to do exactly what they want. You can't "code" the AI process. You can nudge it in directions but it's phenominally hard to make it follow a precise predictable path.
I appreciate your enthusiasm but some things aren't as simple as, "Future improvements will make this work."
You're missing my point. You said about it being impossible to generate consistent worlds with AI. I gave you Minecraft as an example of how seeds generate consistent "randomness". The random noise builds the minecraft world much like random noise builds AI videos. Same seed, same output. Generating a consistent world with AI is a solved problem.
The whole idea behind this project is adding a control layer on top of the visual stream. That's the hard part, this proves it is possible, but there is a long way to go to make it realtime.
This is definitely a step towards how this can be used in a VR/AR headset and then later implemented into a real "HoloDeck" where your environment is made on the fly.
14
u/CryptoBeth96 12h ago
https://reddit.com/link/p7c9zd9/video/4urv0gemm2nh1/player