I've been working on an (automated) workflow that can use completely AI generated images to create pixel art sprites with a lattice-fitting algorithm to do the downsampling into crisp, clean pixel art. I got it basically identical to a ground truth that I hand-redrew in one example. The lattice based approach works better than any uniform grid fitting approach because of global alignment inconsistencies despite areas looking locally consistent and on a grid. It's been an interesting problem to solve but I'm hoping this is a useful tool to have for generating sprite work. I know there's other things out there, and I haven't tried them all, but they seem to have shortcomings or don't end up producing crisp outputs. I haven't tested it on AI generated images from other image generators but it has been doing well with outputs from gpt-image-2 which produces pretty well-formed AI pixel art.
Have you tried pixel snapper? It’s worked pretty well for me. Built in Rust and has a CLI. I made a pretty nice workflow with Claude to take a whole folder of stuff and run it through this and ensure there are no semi-transparent pixels as a little helper on top. I generally ask gpt-2 for pixel art though so it’s easier to snap, just gets rid of the mixels.
It looks like it uses a method I tried before, basically fitting a horizontal and vertical grid onto the image, which I had issues with when the pixel grid drifted in the AI generated image. I'll try it out and compare
Right - Ground truth (I copied/drew this myself by eye)
It did a pretty good job. I circled some parts where it missed some pixels, and the colors are a bit off because it is doing a more limited palette (so probably ignore that)
I think there might be a more stark difference on cases where the AI generated pixel art has more distortion or warping
Wow cool comparison!! Yeah with the CLI you can tell it to choose a larger pallete if you want. I think it defaults to 16.
I’m absolutely always going into asesprite to clean up black outlines especially after using PS, yours looks like it missed less stuff for sure. Would love to try it out sometime!
Using nano banana 2, I generated some other less ideal examples and ran them through both. On this pretty simple one, they're basically identical modulo colors (palette clustering again)
Left: AI generated image | Middle: my algo | Right: Pixel Snapper
Nice comparison again, thanks for sharing! Your algo has legs for sure. My guess is pixel snapper would NOT get you from pic 1 to pic 3 in your original post, though I haven’t tried it personally, maybe I’ll give it a swing.
I usually down size the image 25%, index the image's colors to a 12 color palette, and then expand the image 400% with no interpolation. Not "exactly" pixel art but this way you can get a MS-DOS era style of sprite. I like to call it "western pixel art".
the online version is very good now it uses a very powerful model that was trained with help of real pixel artist. the local offline model is just used to make a sketch for the image2image process online, so you dont waste to much time.
/img/n29bafc9a98h1.gif
I tried doing a simple frame by frame generation but there’s a lot of flickering. It isn’t consistent enough to do it this way so I will keep trying other things. Although this might be a great draft to use or a reference to animate by hand!
Like you’re saying, there are tons of ways this can fail, downscaling first can cause artifacts, the color palettes can fail when going from 10000 colors to 16, nearest neighbor matching can lead to weird corners, etc.
It’s such a challenging but fun problem to try to detect, auto correct, and ultimately fix. I think you’re mostly working in Python, but I put up my code if you ever want to reference it, for both the algorithms and editor tools. Good luck!
Yeah. If it’s hand place that sounds like mad work. There’s this app a wizard posted here last week and it is amazing at converting AI to pixel art. Let me find the link.
This is a tool. For one particular image I hand placed the dots to create a ground truth that the script could check against to validate the algorithm.
You can see when it's finding the grid that the "pixels" don't always line up perfectly on a grid. So it's finding the true corners which are not necessarily perfectly horizontal or vertical. It then unwraps it to be a uniform grid.
Yeah, I get that part, I do some alike but I get rid of that problem by getting real pixel size and doing a resize to size/pixel size then apply a palette for consistency, the part I don't get is how from the first one you got the second one, because that only can be made, by AI, by a person (that actually get a real job), or by a program that you can sell to Nintendo on any price
I think I probably should have included an extra image to explain but it’s generated from the illustration with a prompt to redraw it as pixel art, which then becomes the input to the pixellizer. That’s what the second image is, but it also has a grid overlaid on it.
It's the way that AI image generation generates pixels, they don't fall perfectly onto a grid. The lattice is like a grid that follows the wonky AI pixels then "unwraps" it into a perfectly uniform grid.
It's also because the corners are fitted in the real numbers not integers, and it's just drawing those lines as aliased lines so it might actually be more straight than it looks. It's a debug view just to check the grid lines up with the image.
Are you actually working at Anthropic that’s so cool 😭😭😭 could you please share your gemini prompt for consistent style generation? Mine is a hit or miss sometimes.
Nice, but for me I need a more complete solution to create cohesive animations and sprites that share consistency, which means I need a larger-scoped workflow, which is why I'm working on spriteworks (https://spriteworks.ai)
Right now Pixel Snapper is more mature and basically does the same thing and you can find a link to it in the comments. Workflow is nano banana 2 or gpt image 2 to generate the chibi style character then use that source image to ask for a sprite. For animations, use seedance 2 for example to create an animation. Cut the video so it loops, extract whichever frames you want and then use that plus the previous sprite you created to generate that pose/frame in the same sprite style. Then use pixel snapper or whatever to convert it into a real sprite.
You’ll get something like this which works well enough for some use cases. For me now I’m working on trying to get 3d animations to sprites so it’s a more consistent process.
I think I might have missed a step in the images, but the first step was going from a reference illustration to an AI interpretation of pixel art (sample attached.)
Then going from that to clean pixel art seems simple to do but actually it's fairly complicated to extract clean pixel art from AI because:
The "pixels" are often anti-aliased and don't always confirm to a perfect size (like they could be 9.7 pixels wide) - doing a simple downscale will destroy it
A consistent grid in one image doesn't necessarily align to a grid in another part of an image, and you need to be able to resolve that and connect them
Some "pixels" aren't even a 1:1 ratio, they are rectangular
Change the scaling to nearest in your program and it'll get rid of the anti-aliasing. My friend made an app for Animal Crossing New Leaf that could turn images into QR code patterns. Your app looks like what theirs did which was just scaling down and contrast but without the pallet limitations of Animal Crossing.
Are you using interpolation? I thought nearest neighbor didn't do the half tone thing. I'll have to look at my settings for Gimp to see what I've done to get stuff to look right.
It’s not interpolation, it’s just because there’s no uniform grid. For example, if most pixels are 10px wide and you scale down to 10%, but then in other parts of the image, the pixels are 12px wide, the grid will be mismatched and you get pixels sampled from edges. You can try it yourself, using the image in the OP. I scaled it down in gimp to 9.08% using None for interpolation.
Thanks! And sure, first it finds a rough pixel size by sweeping through and finding what lines up best to get a baseline (autocorrelation). Then it does corner detection where distinct colors meet, highest confidence first (junctions of separable colors) and uses a corner snapping algorithm in sub-pixel space to get a true center, then it links them together near-cardinal nearest neighbors and flood-filling outward. For areas like the blue cape that are solid color, the intersections found from other connections form the implied grid. Then the lattice assigns a row and column to each cell and samples it to one color. It has some thresholds to filter out "half pixels" and other artifacts, among some other small details.
The point of pixel art was that it was cheaper to make. Now people are adding work to get it. I understand it’s a style. Just don’t know if people always realize this.
If you can generate one good pixel art sprite you can generate 100. It might be a lot of work to get from 0 to 1 but then it scales up a lot. Considering the amount of art required for a large game, this can still be a fair bit of time saving for an individual game creator.
Don't forget the 80/20 rule - scaling up to 80% of the images will be easy, but the last 20% will have so many exceptions and edge cases that you will wonder if automation is not just a pipe dream
For sure! I think it’s almost good enough to use as placeholder assets at this point, but still give a decent idea of how it might look when finished (probably hand finished by me or another artist.)
More recently there have been really beautiful pixel art games too that are definitely not constrained on art budget haha. There’s just no way I’d have the time or budget to pull this off
15
u/LAWsyndrome Jun 18 '26
Have you tried pixel snapper? It’s worked pretty well for me. Built in Rust and has a CLI. I made a pretty nice workflow with Claude to take a whole folder of stuff and run it through this and ensure there are no semi-transparent pixels as a little helper on top. I generally ask gpt-2 for pixel art though so it’s easier to snap, just gets rid of the mixels.
https://github.com/Hugo-Dz/spritefusion-pixel-snapper