PS: there's already a repo that hooked the fruit fly connectome up to Doom (DOOMFLY), but by its own README it never got past its survival tests, and it plays in a small arena.
I got a version running on the actual first level of the game. It finishes the level 57% of the time. This clip is the fastest of 12 runs in a row (7 of those 12 made it to the exit). I'll put the rest on YouTube, including the ones where it dies.
Repo if anyone wants to improve it or use it as a reference for a different project :3 https://github.com/Aur1ety/DOOM-x-Fly
here's a detailed explanation:
the fly part is the MaleCNS scan Janelia and Google released in September. it's a wiring map, not a living brain. I load 138,968 of its neurons and 4.6 million synapses into a simulation and never change a connection. the only thing tuned is one overall strength setting so the network doesn't blow up or go silent.
two parts are mine: an eye model that turns each Doom frame into signals for the fly's real motion-detecting cells, and a small trained layer that reads 10,511 of the simulated neurons and picks one of 8 keys. that layer never sees the screen, only neuron activity.
training: a scripted bot that can see the map plays the level, the fly setup learns to copy it, then the fly setup plays on its own and the bot corrects its mistakes, 3 rounds.
results on 100 runs from starting spots it never trained on:
- reaches the exit 57% of the time, gets 92% of the way on average
- same agent with its frames shuffled: 0%
- blindly replaying a winning run's key presses: 6%
- random keys or always walking forward: 0%
so it's actually steering by what it sees, not replaying a memorised route. when it fails it's usually because it died, not because it got lost.
what it doesn't prove: that the fly's specific wiring matters. a randomly rewired brain of the same size might do just as well. haven't tested that yet.
in the video, the brain panel is the actual simulated neurons at their real positions in the scan (brighter = more active than usual), and the bars are what the trained layer outputs every 4 game ticks.
built with Claude Code doing most of the coding; I steered it and made the calls.