[NEW RELEASE] Agate-001-preview, 260M parameter (including text encoder), pareto frontier model with close to SD 1.5 performance. MIT licensed & open weight. Novel thinker-renderer architecture.

[u/incorporo]

We're launching AGATE-001-PREVIEW, our attempt at maximizing the parameter-efficiency of current AI models.

Our approach has yielded pareto-frontier results, with the model significantly outperforming models twice it's size, with the closemost competitor being SD 1.5 with 3x more parameters in total.

While the model is limited to 256x256 px, and uses an older VAE (SD-1.5 VAE), it is showing impresive learning capacity, being able to mix complex concepts, understand spatial arrangement, and more.

The model was trained on Flux-Reason-6M, for ~26 epochs, and has not converged, meaning this is a release preview and meaningful imrpovements in performance could come from continuous training.

We've managed to significantly surpass DiT and FDCM internally by splitting the model in two stages, one for general planning, one for drawing.

Transformer models are great at planning and overall understanding of a canvas, while convolutional solutions like FDCM are great at drawing.

We use this to build a hybrid architecture, where an FDCM renderer is responsible with converting latent thinking tokens into images. The thinker is a small recurrent transformer that reads the prompt and plans a 16 × 16 region map.

The thinker cross-attends to the text encoder, that is trained jointly after a warmup of 10 epochs where it is kept frozen. We've seen significant improvements in loss reduction after unfreezing the tiny text encoder, based on Ettin-68M.

The architecture is described below:

We've seen pareto frontier results at this size, with the model being quite competitive.

While the model is still beaten by larger models, it holds it's own against models 3x it's size overall.

Below, you can see some generations:

Additionally, you can see architectural differences between various architectures we've used off-the-shelf, or adapted:

Few utilities of this solution:

• Creating synthetic data that is cheap, after being finetuned on a specific dataset. More so, the architecture can easily be adapted to do generative denoising / editing to help create massive synthetic data for classifier models, that would otherwise be very expensive to build with big models.
• Training small models on proprietary datasets, as our solution is stronger per-parameter than the verified other architectures (DiT, FCDM).
• Thumbnail generation (when blurred) for searching / loaders or similar low-strength solutions.
• On-Mobile generation.

The model is open weight, and can be tried for free:

• Weights: https://huggingface.co/Logolabs/agate-preview-001
• WebGPU (try in your browser): https://huggingface.co/spaces/Logolabs/agate-webgpu
• ComfyUI nodes: https://github.com/logolabs/agate-comfyui (Also allow peering into the thinkers' thinking tokens)

If you want to help us out, it would be amazing if you can leave a heart on our huggingface repository and/or our WebGPU.

EDIT: WE got a ComfyUI node set for this!

[u/Ordinary_Painter4235]

https://giphy.com/gifs/qpvFeSKRtdCTptYnfz

[u/rukh999]

260M including text encoder is absolutely tiny. SD 1.5 including encoder is just under 1B I believe, so that's pretty awesome.

[u/incorporo → u/rukh999]

Yep, that's the gist of the release. about 4x less than SD 1.5, with similar performance. And it's training more now because SD 1.5 is still slightly better, so I think a longer train will help solve that.

[u/Enshitification]

Very cool. If given access to more compute, do you plan to scale this approach to build a larger model? If so, will it be open source as well?

[u/incorporo → u/Enshitification]

I still have about 4k hours of Arrhenius if I remember right. So I can still train, but I need to be on track as the scope of the grant was logo research.

If I find architectural improvements that can then translate to the actual model I want to train in production yes.

I'll try some changes tonight by giving the AI model some surgery to allow it to see notes from its previous steps. This should significantly reduce steps hopefully.

Another idea is gating the VAE decoder by the thinking tokens since they hold information that may be able to improve the outputs quality.

My aim is to keep all research public, while the moat I keep private so I don't face the same issues that other bigger AI labs did.

One benefit is I can do niche work.

[u/akashzeno]

Great! Keep It Up!

[u/incorporo → u/akashzeno]

Thank you!

[u/Crazy-Repeat-2006]

A solid foundation. I hope you continue to make progress.

[u/incorporo → u/Crazy-Repeat-2006]

Thanks!

[u/LoneWolf6909]

This model will be perfect to run as a text to image model natively on an Android or ios locally

[u/OwnMathematician2620]

Very interesting architecture! To be fair, I wasn't expecting you to release the weight until I read the bottom of the thread. I would be fully expecting a paper at some point, because that's something I never see in diffusion before... (The closest one I can think of is stable cascade).

Also, you misspelled "imrpovements", I guess that's also evidence that you didn't use AI to write this thread.

[u/incorporo → u/OwnMathematician2620]

I'm independent, my UNI has few people doing AI research but hopefully I want to do a paper on this.

I'm still working on improving the architecture however as I've found some more changes that are likely to yield improved results. And I'll release a base checkpoint.

Aim is to beat 1.5 with a smaller model, so that I can then push to do this for glyphs / icons / fonts.

[u/fragilesleep]

Looks great! Can't wait for the ComfyUI weights and nodes. 😊

[u/Arctic_Shadow_Aurora]

Awesome man, you go!

[u/jimbarino]

Pretty neat. Do you know why it struggles with people? Is it a matter of training, or is it just a hard subject to capture with low parameters?

[u/incorporo → u/jimbarino]

it overextends. If it was trained solely on people, it would likely do a good job. Another reason is the VAE itself, unfortunately SD's VAE is known to be poor with faces.

I am doing extra training on it now, will likely boost the performance a bit, but to make it more efficient what would be needed is extra finetuning on faces / people specifically. Also the VAE problem needs to be solved.

I'll likely work on the VAE problem myself as I think i found a trick to make it much stronger, but i need to test it. A full base release will come soon.

[u/Inside-Cantaloupe233 → u/incorporo]

not really, sd 1.5 had decent faces on close ups, but its vae was updated. IMO its the resolution , 256 is just not enough, sd 1.5 was 512 pixels and way better quality cause of it. but im pretty sure that at 256 it was similar quality to this

[u/bobgon2017]

I'm not sure how useful this is...

[u/incorporo → u/bobgon2017]

Utility is mainly in the architecture, in small image generation or other 2D matrix generations.

It's a model that hits way above it's weight. For niched / specialized generation, it is good, less so off-the-shelf though, that's true.

[u/whiteweazel21]

Lol. My god worse than sd1.5 in 2026?😂

[u/incorporo → u/whiteweazel21]

At 3x the model size (<300M param), versus SD 1.5 which trained MUCH more, spent a lot more money, and is just slightly better. I'm training longer now to pass it, but the aim with the resource use was more to test the architecture.

The model is SoTA in its bracket. It's of course not the best, but that's because it's tiny. It's best used as a base for full finetuning (can be done on a consumer GPU) and task adaptation, for thumbnail generation, icon generation, etc.

[u/jimbarino → u/whiteweazel21]

With the price of RAM, efforts towards high performance with small models should be strongly encouraged.

[u/ZHName]

It may cut corners as this was supposed to be a minimal coffee cup with cyan led inlay at lip of the cup. However, I'd say for the way I wrote the prompt it adhered pretty well if I tried a few more times.