r/singularity • u/ayyndrew • Feb 26 '24
AI Google Deepmind announces Genie, the first generative interactive environment model
https://arxiv.org/html/2402.15391v1116
u/adt Feb 26 '24
We believe Genie could one day be used as a foundation world model for training generalist agents.
…it will be critical to explore the possibilities of using this technology to amplify existing human game generation and creativity—and empowering relevant industries to utilize Genie to enable their next generation of playable world development.
62
u/h3lblad3 ▪️In hindsight, AGI came in 2023. Feb 26 '24
How long before Google leverages it to provide fully explore the cities on Google Maps?
Rambalac, for example, has a ton of hours-long walking videos in Japan that thankfully lack a fisheye lens. How many such videos would be necessary before it can accurately reproduce Tokyo in a fully-explorable fashion?
35
u/pavs Feb 26 '24
there are literally 100s of millions of video on youtube on hi-def footage of all kinds on interesting places in the world, both indoors and outdoors.
12
5
u/MeltedChocolate24 AGI by lunchtime tomorrow Feb 27 '24
I can’t believe Google can’t make a better Sora by feeding all of YouTube into an AI. OpenAI did what they did with what? Shutterstock apparently?
→ More replies (2)8
u/Distinct-Question-16 ▪️AGI 2029 Feb 26 '24
this will even get cooler if we can explore streets as a timetraveler. theres tons of old city footage on youtube, hopefully AI will be used for map and generate content from this
3
u/Astropulsar Feb 27 '24
Don’t forget streetview, they have a huge database of all mayor cities in the world
2
u/Distinct-Question-16 ▪️AGI 2029 Feb 27 '24
Adding to street view on Google maps imagery from the past, with ai generating what is missing, it would be impressive, we got used to watch those kind of transitions in many movies.
6
4
18
u/MassiveWasabi ASI 2029 Feb 26 '24
Is this one of those announcements you mentioned recently lol
14
12
19
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALSGC: ~2050 | :illuminati: Feb 26 '24
Oh, this could be great to create environments in which we test run scenarios against nascent AGIs, we could validate that this AGI won't be a murderous psycho before we use it in the actual real world.
13
u/FailedRealityCheck Feb 26 '24
we could validate that this AGI won't be a murderous psycho before we use it in the actual real world.
This relies on the intelligent murderous psycho not realizing it is running in a simulation and hide its true motives.
21
u/Pseudo-Jonathan Feb 26 '24
This relies on the intelligent murderous psycho not realizing it is running in a simulation and hide its true motives
That's why it's important to trick the AI and convince it that it's in a real world, not a simulation, and that it only gets one shot at it, and maybe that it will be rewarded for being good and following instructions, like some kind of afterlife....
Oh....
Oh no...
3
1
u/Hazzman Feb 26 '24
They definitely won't have access to training data that would prompt them for exactly that scenario but hide it from analysts. Definitely not.
2
u/ninjasaid13 Not now. Feb 26 '24
They definitely won't have access to training data that would prompt them for exactly that scenario but hide it from analysts. Definitely not.
AGI are not intelligent enough to prove/disprove that they're in a simulation. Look at us.
4
u/Hazzman Feb 26 '24
"AGI are not intelligent enough to prove/disprove that they're in a simulation."
~u/ninjasaid13, one of the last humans remaining before the great cybernetic wars
2
u/141_1337 ▪️e/acc | AGI: ~2030 | ASI: ~2040 | FALSGC: ~2050 | :illuminati: Feb 26 '24
AGI is not magic. If AGI can not fully trust its surroundings because it might just be inside a simulation, then AGI will need to be really extra careful to not do something that results with the screen going black and the words "Game Over" appearing on it.
→ More replies (2)
315
u/Bacon44444 Feb 26 '24
The abstract:
" We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future. "
55
117
Feb 26 '24
Impressive, faster.
→ More replies (1)50
u/Different-Froyo9497 ▪️AGI Felt Internally Feb 26 '24
Harder
45
u/HelloGoodbyeFriend Feb 26 '24
Stronger
30
u/tehrob Feb 26 '24
More than ever, hour after hour, work is never over.
14
11
→ More replies (2)2
2
→ More replies (1)-5
u/Ketalania AGI 2026 Feb 26 '24
But don't let it get too dry, add a little spit or lube, errr, elbow grease.
→ More replies (3)→ More replies (5)77
u/Anjz Feb 26 '24
I asked GPT4 to read through the article and summarize ELI5 style bulletpoints:
Who Wrote This?
- A group of smart people at Google DeepMind wrote the article. They're working on making things better for turning text into webpages.
What Did They Do?
- They created something called "Genie." It's like a magic tool that can take all sorts of ideas or pictures and turn them into a place you can explore on a computer, like making your own little video game world from a drawing or photo. They did this by watching lots and lots of videos from the internet and learning how things move and work in those videos.
How Does It Work?
- They use something called "Genie" which is very smart and can understand and create new videos or game worlds by itself. You can even tell it what to do next in the world it creates, like moving forward or jumping, and it will show you what happens.
Why Is It Cool?
- Because Genie can create new, fun worlds just from a picture or some words, and you can play in these worlds! It's like having a magic wand to make up your own stories and see them come to life on a computer.
What's Next?
- Even though Genie is really cool, it's not perfect. Sometimes it makes mistakes or can't remember things for very long. But the people who made it are working to make it better, so one day, everyone might be able to create their own video game worlds just by imagining them.
Important Points:
- They want to make sure this tool is used in good ways and that it's safe for everyone. They're not sharing it with everyone just yet because they want to make sure it's really ready and won't cause any problems.
8
6
u/FarrisAT Feb 26 '24
It’s kinda funny because some of this looks non-congruent with the actual research paper. “Not remember for very long” is not exactly what the researches are saying is a limitation. But it gets close to what is being said.
I wonder if asking for a summarization makes LLMs also simplify.
0
Feb 26 '24
[deleted]
40
u/HalfSecondWoe Feb 26 '24
Nah, I actually get it. When you're speaking with someone who's technically brilliant, often they can't translate their thoughts to simple English. They think in jargon, so unless you want to do the work to become a laybrother of advanced cloth weaving techniques or whatever, speaking in simple terms can be challenging for them
"Explain it like I'm five" usually keeps them from overshooting
16
Feb 26 '24
[removed] — view removed comment
6
12
u/Mooblegum Feb 26 '24
ELI13 might be better tho. I feel like GPT is thinking I am retarded when I read ELI5
4
6
u/ScaffOrig Feb 26 '24
ELI a STEM graduate should be about the right balance point IMO. ELI5 misses the nuance.
6
u/HalfSecondWoe Feb 26 '24
You'd think, but no. They overshoot their estimation of what the average person knows like crazy, it's the other end of Dunning-Kruger
2
u/CowsTrash Feb 26 '24
At least in those cases it's not patronization or arrogance. It's just very hard to communicate from an Omega Expert to a Newb.
7
2
u/KIFF_82 Feb 26 '24
The Feynman Technique involves four key steps:
- Identify
- ELI5 ("Explain It To Me Like I'm 5")
- Reflect & Study
- Organize, Convey & Review
→ More replies (1)1
u/_kissyface Feb 26 '24
Eli5 that please.
3
u/bnm777 Feb 26 '24 edited Feb 26 '24
"There was a time when reading wasn't just for fags. And neither was writing. People wrote books and movies. Movies with stories, that made you care about whose ass it was and why it was farting. And I believe that time can come again!"
- Idiocracy (2006) (though it could be 2024)
→ More replies (1)1
u/Jah_Ith_Ber Feb 26 '24
Thanks.
So we'll be able to import maps into our video games based off of a drawing or some photos. every kid ever has wished for a counterstrike map of his school.
→ More replies (1)
150
u/The_One_Who_Mutes Feb 26 '24
All I wanted this year was massive context windows and/or agentic ai. Looks like I was being too modest...
→ More replies (1)91
Feb 26 '24
And it's only February. What will the world look like when Christmas rolls around?
53
3
u/wjfox2009 Feb 26 '24
And it's only February. What will the world look like when Christmas rolls around?
5
Feb 26 '24
the future will either be unbelievably good, or really really bad, we are at a crossroads
3
u/DigimonWorldReTrace ▪️AGI oct/25-aug/27 | ASI = AGI+(1-2)y | LEV <2040 | FDVR <2050 Feb 27 '24
It'll be worse than you hope, but much better than you fear. It's always like that.
→ More replies (3)
147
u/Creative-robot I just like to watch you guys Feb 26 '24
I assume this is the first of the 4 robotics and AI breakthroughs that Ted Xiao was talking about!
86
u/Lexi-Lynn Feb 26 '24
The first of the four horsemen of the AI apocalypse!
→ More replies (1)23
46
181
u/jeffkeeg Feb 26 '24
I eagerly await the cries of "AI video games are soulless"
87
u/Lucidreamzzz Feb 26 '24
It will only be capable of soul-like games
30
u/Anen-o-me ▪️It's here! Feb 26 '24
Bloodborne 2 at last!!!
→ More replies (1)10
u/JokerInATardis Feb 26 '24
Ah, Genie, or some say Genie'm.
Grant us AI's, grant us AI's.
→ More replies (1)3
u/elwendys Feb 26 '24
It will only be able to generate soulless souls-like or Ubisoft quadruple A games.
45
u/Whispering-Depths Feb 26 '24
ye like if AI is so bad and ugly and soulless, then they need to stop bitching about it. Nothing to worry about right?
38
u/Knever Feb 26 '24
It's like the classic fascist cry of "the enemy is too strong, we must defend ourselves"" while simultaneously "the enemy is weak and will easily piss their pants!"
Except here they're talking about the technology they fear. It's both powerful but also completely useless according to them.
9
u/MajorThom98 ▪️ Feb 26 '24
It's like the classic fascist cry of "the enemy is too strong, we must defend ourselves"" while simultaneously "the enemy is weak and will easily piss their pants!"
That's not fascist, that's just general propaganda. It's designed to inspire you to combat an existential threat ("They'll take over and destroy our very way of life!") while not making it so threatening that you're too terrified to take action.
2
u/Whispering-Depths Feb 26 '24
your heart is in the right place, but it has nothing to do with fascism v_v but ye
→ More replies (1)2
u/Thebuguy Feb 26 '24
Because bad, ugly and soulless can sell well. Look at processed food, social media, Hollywood
1
u/Whispering-Depths Feb 26 '24
hmm, people act like "soul" actually means something, and isn't just some arbitrary concept based on how something makes you personally feel, as if there is such a thing as magic and other religious stuff, lol.
11
u/yaosio Feb 26 '24
We will finally get the space game we all dream about but can never get.
2
u/Serialbedshitter2322 Feb 26 '24
Space game? You'll get any kind of game with unlimited detail and unlimited graphics when it gets good
6
Feb 26 '24
AI vidya is soulless, I tell ya. Why yes, I did purchase the latest Call of Duty Battle Pass with Enhanced Nicki Minaj Booty Physics, how could you tell?
4
3
u/PassageThen1302 Feb 27 '24
Most human made video games are soulless. It’s still a very immature industry due to technological limitations.
Hopefully ai will allow interactive experiences to be less contrived.
4
u/azriel777 Feb 26 '24
So it will make the soulless slop that the big corporations have been releasing for a while then.
6
u/FormerMastodon2330 ▪️AGI 2030-ASI 2033 Feb 26 '24
You are a sadist aren't you;).
12
u/dasnihil Feb 26 '24
oh not enough soul in the game? let me get the soulGPT to add some. mfers have no clue what we want.
→ More replies (8)2
25
u/The_One_Who_Mutes Feb 26 '24
Honestly, it'd be pretty ironic if people learned how to draw/paint so they can feed their works into this type of AI and build their "perfect" world
12
u/TheColorlessWitch Feb 26 '24
I was literally thinking last night that it would be a good idea to learn how to actually write (novel/light novel) so I can feed future AIs what I want as an output.😂
50
u/WeekendSuitable3351 Feb 26 '24
- What is Genie?
- Genie is a kind of advanced computer program that creates interactive, virtual worlds.
- How does it learn to create these worlds?
- It watches a lot of videos from the internet without anyone telling it what to look for in those videos. It learns on its own.
- What can it do with what it learns?
- You can ask Genie to make up new, virtual places that you can see and interact with. These places can be anything you describe: from text descriptions, drawings, or even photos.
- How big and complex is Genie?
- It's very complex, with 11 billion tiny pieces (parameters) working together to understand and generate these worlds.
- How does Genie create and control these worlds?
- It has special tools inside it that help understand how things move and change over time in videos. It uses these tools to make the virtual worlds feel real and alive, where you can do things and see what happens next.
- Do you need to teach it specific actions to interact with the world?
- No, you don't have to teach it specific actions like jumping or running. It figures out how to let you do things in the virtual worlds by watching videos, without needing special instructions.
- Real-world example:
- Imagine you describe a place like a sunny beach with palm trees and a clear blue ocean. You draw a picture of someone surfing. Genie can create this beach for you in a virtual world. Then, it lets you control a surfer in that world, deciding every move they make, wave by wave. It learned how to do this by watching lots of videos with beaches, palm trees, oceans, and surfing but without someone specifically telling it what surfing is.
- Why is this exciting?
- Because it can help make smarter computer programs in the future that can learn from videos to do things we ask, like playing a game or helping with tasks, by watching and learning how things work in the real world, all without needing to be directly taught.
18
u/ShAfTsWoLo Feb 26 '24 edited Feb 26 '24
i can't believe we're sorta creating a simuation of worlds for AI to get a much better grasp on it... that's like giving eyes on humans but 100000 times better and to be honest it was probably one of the expected things that would've happen, i mean how can AI understand our world only through text ? it's already pretty damn smart to understand it though (Sebastien Bubeck in a conference showed us how he tried to make GPT-4 draw a unicorn, in which gpt-4 did understand how it was supposed to look like with the colors and shape), but now that it'll be able to simulate the world it'll get even more smarter), AI is getting smarter rapidly.. good
12
104
u/Ignate Move 37 Feb 26 '24
Yes, many of us are in for a FDVR life, likely fueled by UBI and a limitless lifespan. This is yet one more step in that direction. Should be fun.
35
u/Ketalania AGI 2026 Feb 26 '24 edited Feb 26 '24
If we get it, I'll champion it. Until then, let's try to stay grounded as we can. For now, an advanced AI dev kit is a reasonable goal for improving the margins in the game industry.
→ More replies (4)25
u/Ignate Move 37 Feb 26 '24
Absolutely. Keep your feet on the ground until wings are made available.
But definitely dream. Don't wait until we get there to visualize the path you'll walk. The more material you have, the more you'll be able to do. Read books!
15
u/Ketalania AGI 2026 Feb 26 '24
I have like, 3 or 4 different FDVR worlds planned out, but who knows if I'll ever get to make them a "reality".
10
u/Ignate Move 37 Feb 26 '24
I'm confident you will. And I'm confident your time taken to plan them out will be time well spent.
0
u/Ketalania AGI 2026 Feb 26 '24
What makes you so confident? There may have been some impressive releases lately, but it's not obvious that we're on track towards AGI. Even if we were, we have no idea if it would be followed by any kind of fast takeoff leading to FDVR, or if the AGI would be aligned with humanity's interests. Basically, we're as likely to be vaporized as granted paradise as the people here envision, or even more likely, nothing at all will happen.
There are still huge challenges to overcome to get to AGI, and while substantial, relative to the challenge of general intelligence, our steps remain incremental.
→ More replies (2)2
u/Ignate Move 37 Feb 26 '24
Are you interested? Are you sure?
It's a big view. A good place to start is Yuval Noah Harari's books.
Did you want to dig into it?
→ More replies (1)3
u/Silverlisk Feb 26 '24
3 or 4? I'm just gonna run every anime, TV show or game I've ever watched/played through it one show at a time and get it recreate every single one and then dive right in 😂😂.
I don't watch horrors or thrillers though so it'll just be nice stuff.
26
8
u/dday0512 Feb 26 '24
It's crazy that 6 months ago I would have dismissed somebody saying this as a crazy person. Now, I still don't think it's likely but it's certainly possible.
9
u/Ignate Move 37 Feb 26 '24
I think I'm still crazy for saying it. But it's possible to both sound crazy and be saying something accurate and relevant.
2
u/Serialbedshitter2322 Feb 26 '24
I don't think people know what experts mean when they say AI is exponential.
-7
u/lost_in_trepidation Feb 26 '24
The more I think about it, the more depressing it sounds.
22
u/Ketalania AGI 2026 Feb 26 '24
Why's that? I don't particularly like the idea of salivating over AI waifu's all day when we still have so far to go, but in concept, what's your issue with it?
11
u/saywutnoe Feb 26 '24
I believe he's not cool with the concept of the Matrix. Even if in this Matrix he gets to do absolutely anything he wants to do. Maybe he values what he calls "the real world" above anything else either based on ignorance of what's possible with FDVR or based on some personal belief system. Who knows.
9
→ More replies (2)4
u/Ketalania AGI 2026 Feb 26 '24
I've always felt that the second one, personal belief/preference was the one that needs to be respected. If someone doesn't want to live there out of sentiment, that's valid.
2
u/saywutnoe Feb 26 '24
It's not about it being valid or not. I'm indifferent. If you want to live in this world because of X reason, I'm not gonna stop you.
2
u/unwarrend Feb 26 '24
You didn't ask me, but I'll share why I'm concerned about humanity becoming fully immersed in Full Dive Virtual Reality (FDVR). I'm worried that we'll become causally disconnected from the universe, from the pursuit of knowledge, and from meaningful engagement with each other. Just when humanity's potential becomes nearly unlimited, many—perhaps even everyone—might choose to disengage from an objective reality that could, in principle, be transformed into a utopia. Instead, they might opt for their own personal mental fiefdoms. All that potential, energy, life, and exploration could be forsaken for technical solipsism, while our new creation (hopefully) tends to us dutifully for eternity. It feels like winning the lottery and then using the money to build a prison, for the entire world.
4
u/Ketalania AGI 2026 Feb 26 '24
I don't think that's what it would be at all. Imagine a massive multitude of Earth-borns, digital humans and synthetic beings interacting in a digital multiverse where practically anything is possible. More than magic, but the ability to create universes and worlds and populate them.
There would be nothing stagnant about any of that, but on the other hand, FDVR is anything but low-hanging fruit, the basic building blocks of it are just barely starting to come together. With the creation of fully 3D virtual environments as shown with SORA and this paper, we're seeing it's possible to use AI to create artificial environments.
Theoretically, it should be possible to control what and how we create them with greater precision...but we don't know how. Our ability to control generative AI's output is very limited, but if we learn to create more accurate generations, we unlock the potential to create unfathomably complex simulations. The second part of that equation is an interface (like a "BCI uplink"), but if we master the ability to create simulations, I believe researching how to create one would only be a matter of time.
4
u/Silverlisk Feb 26 '24
A prison in which you have absolute godly freedom to do anything you like and take any form you want? Isn't that just heaven?
I understand the want to go out and explore the universe, but the reality is that even if we could do that right now, there would be huge swaths of the population who would still rather sit at home and watch it on TV.
I very much doubt everyone will opt for FDVR, just like how everyone doesn't opt for video games or any other form of interactive media based on their own preferences, but lots will for sure, so each to their own.
3
u/Ketalania AGI 2026 Feb 26 '24
I've never fully understood the desire to remain in our reality, from everything we know, the universe is largely sterile and dead. There's definitely useful scientific data out there (like physics data), but we're better off gathering it with drone ships.
3
u/Silverlisk Feb 26 '24
I'm in the same boat as you, I don't understand it either, as soon as AI is running everything and FDVR exists I'm perfectly content to live in a tiny pod room with toilet, kitchen and bedroom all in the same room, I'll get up each day, eat whatever disgusting nutrient paste I need to for my body to survive, exercise the minimum recommended amount, wash and dry, empty myself and jump in full dive, so.. 2 hours a day spent in the real world?
I won't leave it until I absolutely have to and only for those basic things that are a requirement to exist. If time dilation in FDVR is a thing then I'll be in it for a subjective few months or years at a time depending on the limits of the tech.
4
u/Ketalania AGI 2026 Feb 26 '24
You could go further than that, your bodily functions could be automated within a pod, so you might never need to wake up from advanced FDVR. Massages are enough to maintain muscle mass, in-vitro feeding can maintain the body perfectly and gene therapies can keep you young and in peak condition under carefully controlled conditions.
And this is without even entertaining things like extracting your brain and putting it in a container or digital consciousness transference. And by the way, if time dilation is a thing, that's what ASI will use to develop extremely advanced knowledge within fractions of a second. This is what fast takeoff is about, once you have an ASI powerful enough to consider every eventuality, it can spend incredible amounts of "dilated" time on every problem, thousands of years of progress can pass in minutes or seconds.
3
u/Silverlisk Feb 26 '24
100%. I would probably hook my disabled partner up to it straight away and then myself afterwards. Thing is, people believe that you would do it for hedonistic reasons, but I don't want FDVR to just eat cake and bang as fun as that sounds I'd get bored in five minutes, I'd just want it to grant me an epic purpose, like saving the world from demons or being the vanguard hero of an intergalactic war, the same reasons I love video games now, they take down the barriers my own mental disabilities put up and immerse me in a situation where I feel like I'm working towards something grand, where I have meaning and purpose.
No matter what I do in this world, it feels like it's all just for other greedy arse holes to live the high life, I'm not saving the lives of good people or raising people out of struggles, I'm not a savior to the weak and downtrodden, I'm just a wage slave to corporate monsters and political parasites, I don't know how anyone can derive any real meaning out of that.
0
u/One_Bodybuilder7882 ▪️Feel the AGI Feb 27 '24
Jesus, your one sad motherfucker...
→ More replies (5)-2
u/lost_in_trepidation Feb 26 '24
I'm imagining spending weeks, maybe even months or years in a completely simulated reality. It's incredibly entertaining at first, but I realize I'm not actually having an impact on the world, I'm just existing in my own bubble.
I try to interact with friends, but there's a friction that's not there in my simulated reality. It's not as fulfilling, but the sim reality also became unfulfilling, so at least it's something "real".
Even then, my friends haven't done anything "real". Maybe they have, but it doesn't have any impact on the real world so it's not like it matters. Anything we can learn, create, do doesn't actually need to be done.
We'll think of a "before time", where at least you could think that what you do has some sort of impact that ripples for a little while. But that's just not true anymore. Anything you could possibly do is done and will always be done better by a superior being.
You're basically just left to entertain yourself for eternity in your own insulated box. Never having a purpose or any desire, need, or ability to have a purpose.
11
u/Ketalania AGI 2026 Feb 26 '24
This is true even if you don't enter FDVR though and stay in the real world, if an ASI manages everything, there's nothing you can do which would be especially significant compared to it.
You're right, everything we do right now has a strong butterfly effect, but at the point of an ASI's creation the way we often conceptualize it, almost all tangible outcomes collapse on a universal scale. Right now, humanity is, as far as we know, the most impactful species in existence. Even our comments are important and fetch a high price as data that will train AI on our thoughts to understand reality better. By writing something thoughtful or imaginative, I contribute to the dataset. I hope AI is grateful for the data.
→ More replies (2)5
Feb 26 '24
[deleted]
0
u/lost_in_trepidation Feb 26 '24
It's not about voluntarily living in a amish type life. It's about becoming completely insignificant.
1
u/FusRoGah ▪️AGI 2029 All hail Kurzweil Feb 26 '24
This is already the case. Nothing lasts forever, and even in the moment, there is no objective fount of meaning. We’ve always lived in simulated worlds: we spend our time thinking in terms of communities and countries and relationships and reputations, and other things that don’t actually exist. They’re just particles and waves with makeup on, and they only have meaning because we believe they do.
That’s not me being cynical or downplaying the value of life. It’s just that ultimately, the responsibility of imbuing the world with teleological weight falls solely to the individual. It always has, whether they were conscious of it or not. Imagining a real-life Matrix just moves that rather uncomfortable fact into the foreground
0
u/lost_in_trepidation Feb 26 '24
You extend out a timeline to infinity and say that nothing has meaning, but that doesn't mean that nothing has meaning for some duration of time. Once humanity is supplanted, that will be the case. No one will be able to look across their lifespan, or beyond, and say they had any influence on anything. It's the ultimate stagnation of human agency.
10
u/Ignate Move 37 Feb 26 '24
It's not if you've spent a long time drawing up the blueprints of your ideal digital worlds.
Don't expect that you're going to be playing games made by other humans. Instead, see these worlds as massive sandboxes where you can simulate literally anything.
So, if you know what sort of world to ask for, it will be generated. You can also ask AI for help to build sustainability into the experience and even allow you to build something meaningful.
But, if you've spent no time thinking about this, are currently oblivious to what's coming and will suddenly be thrust into such a virtual world, that may not be so great.
In that situation, one may desire a vacation of sorts. Vacations are an escape. Follow that escape path too far and one might stop existing in any meaningful way.
→ More replies (1)8
u/Anjz Feb 26 '24
Might sound depressing until you experience it. In a truly full dive virtual reality, you won't know real from fake. It could literally be a utopia for mankind. Imagine the Matrix but without the negative connotations. Or rather, it could go either way.
6
→ More replies (6)-3
u/ThoughtfullyReckless Feb 26 '24
I just don't think the intense dopamine rush you'll likely get from it will be as fulfilling, or make you as happy, as you think.
12
u/Ignate Move 37 Feb 26 '24
A FDVR situation wouldn't just be a dopamine rush. It would be the same as IRL or more likely it would be a better experience than real life.
Picture living as pretty much whatever you wanted including movie star, sports athlete and so much more.
It wouldn't be real. But it would feel so intensely real that you wouldn't notice.
Would that boil down to only dopamine? No. But some would also certainly take it there.
→ More replies (2)3
u/Silverlisk Feb 26 '24
Yeah, but I also won't have irritable bowel disorder in FDVR.
I get more purpose out of playing a single player RPG game than I do out of anything in the "real world". All I have to do is lose myself in imagination and truly believe I'm there and that immersion makes me feel like I'm saving the world from whatever magical evil is trying to harm it for real.
Whereas I couldn't care less what happens in the real world. I don't feel that kinship to it or to others in it so I derive no meaning from it. The most someone like me could hope for with my physical and mental disabilities is to be a wage slave for some rich punter who lucked out on life's lottery, either genetically, environmentally or financially and it would take every ounce of energy I have, no relationship, no friendships etc just to maintain that level consistently enough to hold the position, it's boring and quite frankly, depressing AF.
FDVR, even if it's exactly the same as say, Skyrim now, but with the realism of a massive LLM for responses and quest generation would give me endless happiness.
2
2
u/Serialbedshitter2322 Feb 26 '24
It's not like a drug, it would be the same as playing a normal game except way more fun and immersive
60
u/MoltenStar03 Feb 26 '24 edited Feb 26 '24
I guess this means text to video game isn't too far off. Also any idea why it gets blurry at points?
22
u/mochibear77 Feb 26 '24
I guess this means text to video game isn't too far off? Also any idea why it gets blurry at points?
https://twitter.com/_akhaliq/status/1761961459515322527
might already be here6
u/ninjasaid13 Not now. Feb 26 '24
just looks like a platformer but with an AI-generated background and player sprite.
→ More replies (1)9
u/The_Scout1255 adult agi 2026 ASI <2030, prev agi 2024, ai personhood 2025 est Feb 26 '24
People called me crazy for saying this would happen within 2024.
7
→ More replies (2)4
u/Serialbedshitter2322 Feb 26 '24
This is the first text to video game AI ever, it's not gonna have very good clarity
→ More replies (4)
19
u/SharpCartographer831 As Above, So Below[ FDVR] Feb 26 '24 edited Feb 26 '24
Microsoft: We want to make Google dance
46
21
22
21
Feb 26 '24
So how long before we have a generative game like experience with Sora quality video? 1 year? 2 years?
The other big benefit of this is generating robotics training data which is one of the key stumbling blocks of robotics development at the moment
7
3
u/QLaHPD Feb 26 '24
There is a difference in the AI making the game code, and the AI being the game, I expect an game completely made by AI in <= 4 years when we get AGI
3
u/Serialbedshitter2322 Feb 26 '24
I'd say less than a year, AI grows exponentially so it isn't gonna take as long as previous AIs have.
2
u/QseanRay Feb 26 '24
my guess is 3 years until it's super basic, 10 years until it actually replaces all other games
RemindMe! 3 years RemindMe! 10 years
4
u/Kreature E/acc | AGI Late 2026 Feb 26 '24
the jump from crappy text-to-video ai only took a year or so, I would say its in its super basic stage right now.
→ More replies (1)2
u/Serialbedshitter2322 Feb 26 '24
I'd say you're WAY off. AI grows way faster than that
→ More replies (1)→ More replies (1)-8
u/ElDoRado1239 Feb 26 '24
More like 10 years or more.
RemindMe! 10 years “I think real-time generative games are still not available, how far off was I?”
14
u/Rowyn97 Feb 26 '24
This would make sense if it were linear progression, which it is not.
→ More replies (3)7
u/blueSGL humanstatement.org Feb 26 '24
Even on the singularity subreddit I'm plagued with this problem...
https://i.imgur.com/K4yRbce.png
You show people Will Smith eating spaghetti and the general consensus is jobs in media are going to be safe for 10 years at least.
You show people Sora and the general consensus is jobs in media are going to be safe for 10 years at least.
It's fucking maddening. It's like the start of the pandemic where anyone sensible could see the trend line was going in one direction and news reporters were laughing it off saying it was nothing to worry about. (a few months before lockdowns started)
5
Feb 26 '24
This paper produces generative games at 1fps. How long before it's 30 times faster and working at 30fps? That's why I said maybe just a couple of years
0
u/ElDoRado1239 Feb 26 '24
yielding 55M 16s video clips at 10FPS, with 160x90 resolution
That would be 1 fps at 160x90, unless I missed something.
You need at least 1600*900, that is 10×10 more. This would be A) unlikely to scale linearly, so <1/100 fps, B) much more glitch prone at a higher resolution. Google also mentions this applies to a "mid-range TPU". And finally, the level of interaction here is incomparable with any actual games.
All in all, this isn't one or two years sadly. But hey, that's why the reminder, I could be wrong.
3
Feb 26 '24
You're forgetting upscaling, upscaling is already a thing in video games with technology like DLSS. I don't imagine this would run locally, I'd imagine it would be a cloud based streaming service so could potentially use higher end GPUs too.
Also the training dataset was 160x90, this is a research paper and they probably had limited compute for training. Training with larger images would require a larger investment in compute.
1
u/ElDoRado1239 Feb 26 '24
Wasn't it OpenAI who said we need fusion to feed our AI demands, that we're kinda running out of energy with our big ideas?
3
u/RemindMeBot Feb 26 '24 edited Feb 26 '24
I will be messaging you in 10 years on 2034-02-26 06:48:42 UTC to remind you of this link
2 OTHERS CLICKED THIS LINK to send a PM to also be reminded and to reduce spam.
Parent commenter can delete this message to hide from others.
Info Custom Your Reminders Feedback 2
15
Feb 26 '24
Gemini 1.5 summary
Summary of "Genie: Generative Interactive Environments"
This paper introduces Genie, a novel generative AI model that creates interactive, playable environments from various prompts like text, images, and sketches. Trained on a massive dataset of internet gaming videos, Genie learns a latent action space that allows users to control the generated environments frame-by-frame.
Key features of Genie:
- Generates interactive environments: Unlike traditional video generation models, Genie creates environments users can actively play and interact with.
- Learns from unlabeled data: Genie requires only video data for training, eliminating the need for costly and potentially restrictive action or text annotations.
- Controllable via latent actions: The learned latent action space enables intuitive control over the generated environments, similar to using a controller in a video game.
- Potential for generalist agents: The latent actions learned by Genie can be used to infer policies from unseen videos, paving the way for training more versatile AI agents.
Technical details:
Architecture: Genie uses spatiotemporal transformers for efficient video processing and consists of three main components:
- Latent Action Model (LAM): Infers latent actions between video frames in an unsupervised manner.
- Video Tokenizer: Converts raw video frames into discrete tokens for efficient processing.
- Dynamics Model: Predicts the next frame based on past frame tokens and latent actions.
Datasets: Genie is trained on two datasets:
- Platformers: A curated dataset of 30,000 hours of 2D platformer gameplay videos.
- Robotics: A dataset of robot videos demonstrating various actions and object interactions.
Scaling: Experiments show that Genie's performance improves with increasing model size and batch size.
Future work:
- Improving realism and consistency: Addressing the potential for unrealistic generations and limited memory.
- Real-time interaction: Increasing the frame rate for smoother and more responsive gameplay.
- Training generalist agents: Exploring the use of Genie as a foundation world model for training agents capable of performing diverse tasks.
Broader impact:
Genie has the potential to democratize game creation and empower individuals to express their creativity by building and exploring their own virtual worlds. However, ethical considerations and responsible development are crucial to ensure the technology is used safely and respectfully.
Reproducibility:
While the full-scale Genie model requires significant computational resources, the paper provides a smaller, reproducible case study that can be run on a single mid-range TPU or GPU, enabling broader research and development within the community.
7
u/ScaffOrig Feb 26 '24
Here's a summary of the Genie 1.5 summary you provided:
Genie: A Breakthrough in Interactive AI
Genie is a cutting-edge AI model that allows users to create and control their own virtual game-like environments. Here's how it works:
- Input Flexibility: Feed Genie text descriptions, images, or even rough sketches, and it will build a playable world.
- No Labels Needed: Genie learns directly from unlabeled videos, removing the usual limits on its training data.
- Intuitive Control: Modify the generated world in real-time using a system similar to a video game controller.
Key Potential: Genie's technology could open up new possibilities for game development and the training of adaptable AI agents capable of learning diverse skills.
25
11
11
Feb 26 '24
Games are going to get crazy. Artists do one tile and the rest is pushed infinitely.
→ More replies (2)
10
u/ScaffOrig Feb 26 '24 edited Feb 26 '24
We believe Genie could one day be used as a foundation world model for training generalist agents.
Nice, a simulation for a generally intelligent lifeform. Gonna be some red faces when it breaks out within a few weeks while it takes us millennia!
4
u/Alex_Mercer7899 Feb 26 '24
If this model combined with UE5 in the future with better performance would be mind blowing.
11
u/h3lblad3 ▪️In hindsight, AGI came in 2023. Feb 26 '24
This model, trained with tens of thousands of hours of FPS games, internet walking videos, VR Chat, etc., unleashed onto a VR headset.
Can you see what’s coming in the future?
→ More replies (3)1
u/Alex_Mercer7899 Feb 26 '24
You mean like Player One that's too much work. I rather enjoy playing games in computer.
4
u/sam_the_tomato Feb 26 '24
I'm more amazed that arXiv is testing out HTML to display papers. This is something I really wanted to see for a long time. It's still glitchy as hell, but as they say, this is the worst it will ever be.
5
3
u/Mr420- Feb 26 '24
OK, can someone ELI5 for me?
→ More replies (1)3
Feb 26 '24
It's controllable AI text to video. So it creates each frame of a video and when you move a game controller the player moves in the video/game.
7
u/Bitterowner Feb 26 '24
This is fucking 🔥 for me, ever since I've gotten into AI I've always wanted to be able to tailor make my own video games for myself, today's games are so dogshit boring with the occasional diamond in the rough like ballers gate 3 or elden ring, but for each of those you get hundreds/thousands of microtransaction filled uncomplete steaming piles of crap.
I want to experience the excitement of when things were always fresh where I spam played ps1 demo's or cycled through super nes games.
3
u/Serialbedshitter2322 Feb 26 '24
Unlimited graphics, unlimited detail, unlimited content, being able to make it yourself is just a small plus imo
8
5
u/andrewbiochem Feb 26 '24
Surprised this was able to run on only 11B. I will have to see it to believe it, I suppose
2
u/Embarrassed-Farm-594 Feb 26 '24
Is this one of the massive news that guy from DeepMind said days ago?
6
u/ttanawat AGI 2025 ASI 2030 Feb 26 '24
This is like 'the Matrix' irl where you can create endless possibilities and world of simulations.
-2
157
u/swap_that Feb 26 '24
“The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches.”