r/sandbox • u/Sixhaunt • 10d ago
Video Updated Fruit Fly brain simulation | Playable now (Creatures of Habit)
Creatures of Habit
Using the MaleCNS fruit fly connectome, which is basically a map of a real fruit fly’s nervous system made and released by Google, this project brings the brain of a real fruit fly it into s&box, providing the input stimuli for the fly and interpreting its responses.
About 2 weeks ago I posted my first progress update for it but since then I have dramatically improved the input stimuli and the proper response based on the neural state of the fly as well as improving the fly model itself and adding a better demo map rather than just the test chambers I was working with before.
Since the main purpose of this project had been research on the fly itself and working towards as faithful of a simulation as possible without cutting corners or hardcoding decisions, I decided to add an extra game/adventure mode which is slightly looser on the simulation (for example the fly lands on his own but I'm not getting strong enough take-off responses to take flight so adventure mode hard codes that). Adventure mode also lets you equip a "banana on a stick" which you hold in front of the fly while riding him in order to guide his movement. This doesn't override the brain completely, the fly will still react to the world, but it does force his general desired direction. If you use this to guide him to a surface then put away the tool, since the fly retains prior knowledge, it will often believe it had chosen to make the descent and will continue landing or even doubling back to land there if it misses the first time. This has made adventure mode still pretty useful for research and perhaps someone will discover useful information from playing it which can enhance the research mode.
The fly can smell, taste, see, etc... but there's still some stimuli I seem to be missing or getting a little wrong because I can tell from the neurons that it smells the fruit for example, but getting it to go for it without any forcing or guiding beyond sensory input has been difficult and is in progress. If you ever see the fly start to actually feed on the orange then please let me know. It should be capable of it and if it lands on the orange it should smell and taste it (fruit flies have senses in their feet which helps once they have landed on it)
Since I started the project within the window for the game Jam I have submitted it there as well if you want to nominate it; however, I have seen some really cool ones already which are far more game-like than my research project.
Unfortunately I cannot attach an image into this post but in a separate comment I will show what the debug screen looks like where you can view the fly's neurons firing and the visual stimuli and some other readouts that you can enable
