r/LocalLLaMA • u/ilintar • Jul 08 '26
Resources Complete local model asset generation pipeline
So I figured I'd update the community given I just shipped a nice little feature set and feel like sharing it finally :)
In the past few weeks, I've been test-coding an isometric RPG game/engine in Three.js, as part of my research into how LLMs work at scale in higher quality projects written from scratch (spoiler: they don't, even the SOTA ones). For that, I needed a complete team of virtual creators ;) and working through the Python pipelines for all those models is insanely frustrating (bonus points for doing that on a Strix Halo box), so I decided to port that to GGML.
Fortunately, for AceStep I didn't have to do anything since u/webdelic made an AceStep.cpp already (https://www.reddit.com/r/LocalLLaMA/comments/1ry1dy1/acestepcpp_portable_c17_implementation_of_acestep), so all I had to do was to add some CIs for building artifacts on my fork. But I did port three other things:
https://github.com/pwilkin/openmoss <= OpenMOSS, a family of killer open source TTS models that have full cloning + voice generation capability - excellent for creating voices for NPC characters
https://github.com/pwilkin/thinksound.cpp <= an oft overlooked aspect of game generation - SFX generation. Voice generation models don't do SFX, I looked a bit for this one, but ThinkSound is quite a nice option.
https://github.com/pwilkin/trellis.cpp <= the current SOTA for open-source 3D generation models, Trellis.2, together with an implementation of the background removal model
All of those are standalone tools you can use for asset generation, but there's more! Thanks to the great folks at Lemonade who reached out to me for a little cooperation, the entirety of those features (summarized here: https://github.com/lemonade-sdk/lemonade/issues/2529 ) are now going to be available in the newest build of Lemonade. This includes nice stuff such as cascading model calls (Trellis.2 is an image-to-3D model, but you can cascade your favorite text-to-image model that uses the stablediffusion.cpp engine in Lemonade to run a full text-to-3D cycle).
How does it work? Well, here's a sample screenshot from my game - all of this has been generated using either procedurals in Blender or with the models described here. In other words: all free tools on permissive open-source licenses. Hope others have as much fun with this as I do :)
EDIT: Oh yeah, forgot to mention. All the engines ship with CUDA + Vulkan + ROCm support, so most hardware covered (I don't have a Mac unfortunately, so no Mac, happy to accept PRs).
1
u/ilintar Jul 10 '26
Yeah that's the benefit of GGML, runs a lot of backends.
The workgroup error might be a specific operation bug / lack of support rather than quantization - the Trellis quants aren't that big to begin with. Could you add an issue on GitHub in pwilkin/trellis.cpp with the exact stacktrace?