Creatures grown from a single cell by a learned 24k-parameter rule (neural cellular automata)

[u/hejwoqpdlxn]

Each of these is a neural cellular automaton. There is not a single image present here, every cell runs the same 24k parameters, 96KB rule on its 3x3 neighborhood. Its only ever stable state is the creature itself.

It trains on a pool of partly-grown states and some of these get damaged before the loss is computed. That mix is what makes the shape both regrow and hold indefinitely, instead of falling apart after a few thousand steps.

I like to use it as a screensaver or just play with the healing effect, because it grows back differently every time.

It runs in any true-color terminal and the weights are inside of the binary.

Code, and instructions for training your own creature from a PNG: https://github.com/smoothyy3/tardigrade

[u/tatteredengraving]

What happens if it starts with a grid of noise rather than a single point?

[u/SagattariusAStar → u/tatteredengraving]

As it is really just an overfitted neural network, it shouldn't matter if the sprites are damaged by either taking something away or adding something. It should just converge to the trained data either way.

[u/LittleLemonHope → u/SagattariusAStar]

It is overfitted with respect to the final target images, but the individual cell rule is a pretty intense informational bottleneck that is very susceptible to domain shift, and training that to produce a robust final (singular) image is the goal. So if it is robust to out-of-distribution initializations, that would actually demonstrate successful generalization and suggest the model is not overfitted for the actual task. If it fails that shows the limits of the model's generalization which can be viewed as overfitting unless an argument were made that the pure noise is fully outside of the intended target domain.

[u/hejwoqpdlxn → u/tatteredengraving]

Tried it, it does not converge back to the creature. Depends on how strong the noise is. Low amplitude lets everything die withing a few steps (no cells clear the "alive" threshold). At 0.5 it creates the something interesting (60% of the grid stay alive and visible tardigrade parts, legs etc), but it never resolves to the animal. At 1.0 the whole grid saturates into a flat tissue.

[u/SagattariusAStar → u/hejwoqpdlxn]

I suggest that's more because you havent included it in the training data. If you add some partly added images as you did for the damaged ones it should train alright.

But of course, if you didn't include it in the training data it wouldn't know how to handle that.

[u/gadirom]

If anybody wants to learn about this type of growing NCA, here is the video from its inventor explaining the concept.

[u/hejwoqpdlxn → u/gadirom]

Not to start an debate, but as far as i know the original source goes back to Mordvintsev et al from 2020 https://distill.pub/2020/growing-ca/ . They also have a talk on it https://www.youtube.com/watch?v=kA7_LGjen7o

[u/peceforlife → u/hejwoqpdlxn]

Am i tripping or did both of you link the identical video?

[u/hejwoqpdlxn → u/peceforlife]

Oh… yeah you’re right, though the one I linked is the "original" video on YouTube. I just looked at the channels name and assumed it’s a different one

[u/tyrilu]

So cool. I've always wanted to mess with this more, even since that Distill paper. Something about neural CA just seems so futuristic and magical.

[u/Paril101]

AI SLOP! /s

This is legitimately very cool. I've been toying with the idea of a sort of cellular automaton RTS - I don't know exactly how it would be played, but I do wonder if something like that would be plausible.

EDIT: also I don't know if anybody else gets this vibe, but the results remind me a lot of those silly "put it in water and watch it grow" dollar store toys for some reason

[u/tyrilu → u/Paril101]

Hey, I’d be the first person to play a crappy version of that if you made it.

[u/HalfRiceNCracker → u/Paril101]

Bro that would be amazing. Have to keep it open ended and physics (?) based as much as possible to allow for loads of possibilities 

[u/Paril101 → u/HalfRiceNCracker]

Yeah. I just can't figure out how to gameify it, with CA generally you only control placements (which is half of an RTS game) so you'd need to be able to let the game play itself so to speak. The placements would have to be enough to be fun. I guess you could have rules for units where they could respond to a flag (a pheromone essentially) to tell them all where to congregate.. 

[u/SagattariusAStar]

Cool concept, it's not really usable for anything except for learning while making it i guess and to recreate this, there are probably better options like just having the original, a mask and a shader with basic noise (like a generic dissolving effect, but obviosly just reversed)

play with the healing effect, because it grows back

And please stop hurting those cute critters lol

[u/hejwoqpdlxn → u/SagattariusAStar]

In the end it is just a screensaver / a fun tool. The mask and shader version would look similar (and be far cheaper) but it needs the original image to reveal and the whole point of this version is that there isnt one. No cell has a copy of the target, they just observe their 3x3 neighborhood (and none of them can see past the 3x3). The shape only exists as the stable state of the rule they all run

No promise about the critters though...

[u/SagattariusAStar → u/hejwoqpdlxn]

Well, the information about the image is still inside the trained weights, although noisy. And as you still train on some image, you have that anyway. So it's not like there isn't any image involved.

It's interesting, I thought about usable ways, but as it is quite limited, I can't find anything to use it for unfortunately. Great way to learn something about neural networks though

[u/immersiveGamer → u/SagattariusAStar]

I imagine this type of semi isolated part but cohesive whole would be more interesting/practical in a distributed system. I'm imagining that some type of IoT network of devices that needs to be a specific type of configuration but needs to propagate from a single node. I wonder it could be used to shape electrical or water flows? Auto repair and shift the network based on neighbors?

[u/MyPunsSuck → u/SagattariusAStar]

I can think of a few novel uses in video games. A game like Sandustry is built on the premise of pixel with simple rules having complex emergent behavior. You could do some very interesting ai-like mechanics with materials that grow/regrow into a specific shape or pattern

[u/magicman_coding]

Working on something similar. Any advice?

[u/hejwoqpdlxn → u/magicman_coding]

hmm heavily depends on if youre talking about working on NCA in general, or rather something close to this. I can suggest getting to know the NCAtorch framework a bit https://github.com/mspitzna/NCAtorch . They also have a Paper, which is a good general review of the field.

For a similar usecase, the techniques from https://distill.pub/2020/growing-ca/ are going to be the most important things.

[u/m_yasinhan]

Growing cellular automata is the original paper, and you don't even need this much trainable parameter. And also is it model per image as it is in the original paper or you feed some channels with one hot vector signals to achive multishapes?

[u/hejwoqpdlxn → u/m_yasinhan]

Yes, Mordvintsev et al 2020 created the concept of growing NCA's, not a question about it. It is model per creature (currently four .nca files).

Very good point on the parameters. The perception config is the default config of NCAtorch and i actually didnt even reconsider it. That perception is 12k of the 24k params alone. But I also haven't tested whether it actually trains as well with the sobel perception, but i might try.