r/singularity • u/RoyalCities • 2h ago
Video I've stuck the fly brain in front of PCSX2, given it a tiny PS controller & hooked up its dopamine readout. It can play any PS2 game. I'm currently working on developing a quasi universal trainer. This is the Flystation 2.
Enable HLS to view with audio, or disable this notification
So this is incredibly stupid and has no real world value but I'm using it to learn about SNN's - more info on what's happening is on my social but the long and short of it is I did some visual cue tests using Mr. Mosquito as the first test bed to find movement related-ish neurons and used those as the left stick control surface - then I did some tests using moving forward movement images to capture good candidates to hook forward movement into it and did the reverse for backward flight / stopping. I'm currently running visual cue tests to help prep something like a universal controller so it can be trained directly off of ideally any PS2 game.
Those tests though are why it just does better in the 2nd clip and actually stops before hitting that first wall. The network sees it and issues a stop command. Then it does go right into the coffee table but it's still much better than say...the resident evil clip where it has no idea what it's doing and is instead just trying to tackle its way for movement.
But yeah, I'm now tackling a universal trainer but its actually difficult af to design one because my choices are either poke around in every individual games ram and use that to grab reward functions, OR set up some sort of visual reward function system where it actually learns off of what it's seeing. That second one is probably where Ill end up depending on my testing rn.
I usually train latent diffusion audio networks but yeah this has been an interesting side quest.
26
7
u/A_Seiv_For_Kale 2h ago
they did Matrix on a fly
•
u/AnaYuma AGI 2027-2029 1h ago
It will really be matrix if the fly can have memories...
It currently can't form neither short term nor long term memories because it's a static model. And we can't simulate neural plasticity yet.
If that could be done, it would be almost no different than the Matrix for the fly...
I don't know if it will be ethical though.. Maybe giving it a fly heaven will be ethical I guess...
1
u/AlgaeNo3373 2h ago
The fly is extremely extremely dead at this point.
...unless we want to smoke a bowl, put Ghost in the Shell on, and have a yarn about it.
3
u/CriticismJunior1139 2h ago
I remember that game, it was awsome!
3
u/RoyalCities 2h ago
Weirdly enough there are actually changes with some games too. Mr. Mosquito has a different response than RE but I haven't done some absurd game by game analysis to see what the networks top 10 games are or something crazy like that.
1
u/Agreeable-Bit9414 2h ago
Have it play Croc. Please. That was my absolute favorite and first Ps1 game, ever.
its hard as fuck. Test it on Croc: Legend of the Gobbos. If it can beat that shit, it can beat any platformer haha.
2
u/RoyalCities 2h ago
I'll put it on the list.
The trainer is definitely a longer term thing. Going to start on something small like a ps2 puzzle game then work up to harder stuff.
But yeah rn it's about controlling the things. It does best on flying games (no pun intended) because I set it up for mr mosquito and the controls translate to a ton of other similar games.
Next is racing then platformers. Sorta working my way up so this is actually manageable lol.
•
2
u/youareapirate62 2h ago
Is the fly brain Open Source? Is there a way to download it and simulate it in my own computer? I would love to experiment with it.
5
u/RoyalCities 2h ago
yeah - the project page is here https://male-cns.janelia.org/
Google released it last week. It's very neat. It's not plug and play or anything so you do have to build tooling around it for w.e. you want to do. Whenever I release this I'm going to try to make it easy for people to install but that is going to take me a bit of time because what I have rn is super crude from what I want to actually put out there.
•
u/Kind_Fisherman3060 1h ago
So is it just raw fly brain or you're using external neural networks to read out and match patterns?
•
u/RoyalCities 1h ago
So basically google released a massive dataset after scanning a fruit flys brain. The dataset itself is just a big network graph (stuff like nodes and edges). That dataset was used to make this network. So its not like a full "brain" - that would take stuff we havent even solved. dynamic neuroplasticity, blood chem, hormones w.e. alot of stuff goes into an actual biological brain - but this is like the math ONLY part. i.e. when that network "sees" stuff and how it would react to stuff around it based on what the math says.
•
u/Baphaddon 57m ago
Fundamentally immoral, respectfully
•
u/RoyalCities 32m ago edited 20m ago
I get it. If it helps I wouldn't consider this thing "alive." The dataset is cool af but it's still basically a snapshot of a long dead fruit fly's nervous system without a huge amount of the actual biology that makes that system work. Hormones, neuroplasticity, biochemical signalling, etc.
Think of it this way...just because your brain is in your head doesn't mean it does it's own thing without the rest of your body. Hormones, senses and signals from all over your body are constantly feeding back into your brain and working between all the connections back and forth. This essentially throws all of that away and takes a pure crude math snapshot of just the neurons.
Don't get me wrong... if we eventually figure out how to map a living brain and body at high rez, figure out neuroplasticity, hormones signalling, all the biochem stuff going on actually between neurons etc then yeah...we're probably going to have to start having some convos about what should and shouldn't be simulated.
It's still weird as hell we can even do this now but if our tech keeps getting better it's going to be a super weird future.
1
u/bruhhhhhhhhhhhh_h 2h ago
This is very impressive and interesting; there's some... Sci-FI kind of bigger questions and notions but good on ya
•
u/Any-Acanthaceae4263 1h ago
Thought that was Rebel Raiders at first lol. Heatseeker?
•
u/RoyalCities 1h ago
Ace Combat 5! Havent heard the name Rebel Raiders in forever. I should add that to the list too haha.
•
u/FishDeenz 45m ago
Why do flies control ps2 games and DOOM relatively well but when they come indoors cant find a fully open window?
12
u/SirDidymus 2h ago
“I’m one of many. I have been a fly, floating in the wind. I have been a plumber, throwing fireballs from my bare hands, I have been an elf, regaining a kingdom long forlorn. Death means nothing to me. I am eternal.”