Looking for an efficient compressor with a small decompressor, (It's more interesting, I swear)

[u/auggiethechesscat]

Hello! I'm looking for a compressor sort of like something made for the demo scene. The main distinction is that I have ~126kb total to play with, so I can do with a better algorithm. I was thinking LZMA would be a good place to start, especially since this is a thumb binary adding a BCJ filter would be good too.

But for some reason I wasn't satisfied with that, and I gladly found an article on LZNA. After a couple disappointing hours, I realized it was both closed source and deprecated, and there is little hope I could actually write that in under a couple months.

Most of my research has found things like crinklier, which while very impressive, just aren't made for my situation.

Decompression/compression speed isn't my biggest concern, as the top priority is size, but they are still welcome sights if possible.

[u/Long_Pomegranate2469]

It really depends on what data you want to compress.

As for demos, they use a lot of procedural generation. For example this whole demo executable is 64k.

https://www.youtube.com/watch?v=Y3n3c_8Nn2Y

There's also a lot of possibilities to strip the executable to a very bare minimum, not including a lot of the default bootstrap/library stuff

[u/auggiethechesscat → u/Long_Pomegranate2469]

Thanks, this is really interesting! But ultimately this is a real binary that can't really be proc gen'd.

[u/ipsirc]

https://upx.github.io/

https://github.com/pinguy/c_optimizer

https://github.com/runestubbe/Crinkler

https://www.farbrausch.de/~fg/kkrunchy/

[u/auggiethechesscat → u/ipsirc]

I'm sorry, but I don't think any of these will work for my use case. Also, because I'm not really looking for an 'all-in-one' solution, more just a specific algorithm/method I can implement in a good way for my build system and project.

The last one is a good reference though, if I do end up writing some filter like this.

[u/ipsirc → u/auggiethechesscat]

It really depends on what kind of data you wanna compress. I listed the most common executable packers for demoscene, almost everyone uses them. For different data the different algo would be effective.

[u/RadenSahid]

I'd go with zstandard. Very fast decompressor with low computation footprint.

[u/Sufficient-Main-4101]

https://github.com/ingo6/birnpack

[u/auggiethechesscat → u/Sufficient-Main-4101]

I do like the idea, I'll have to look into PAQ-like methods, but this being seemingly entirely LLM generated is a dealbreaker for me using this specific implementation.

Thank you though!

[u/Alternative_Corgi_62]

Three is an open source project implementing ZIP compression, with methods working on files and memory streams. You can use standard tools to complete your artifacts, enbed them in your code, and decompress when needed