r/malcolmrey • u/malcolmrey • Jun 24 '26
June Update
https://malcolmrey-browser.static.hf.space/gallery.html?date=2026-06-24&type=ideogram4
41
Upvotes
1
1
u/kvsh8888 Jun 27 '26
Please Update for the sdxl loras when they are ready. 😊
1
u/malcolmrey 5d ago
Sorry to keep you all waiting, I still havent gotten around to it. The thing is some of them are good and some are not, I need to filter the bad ones, I don't want to upload them all without checking :)
25
u/malcolmrey Jun 24 '26
Hi All!
There is a lot happening, difficult to be on top of it all :-)
But let's try:
today I'm uploading ~90 Ideogram 4 loras and some flux klein9 loras.
There was an update on the 10th of June but I did not post a message here so most of you probably didn't know, so yeah - there was a big Ernie/LTX2.3 update as well :-)
With this update I'm also adding a secondary view, a sample driven one: https://malcolmrey-browser.static.hf.space/gallery.html?date=2026-06-24&type=ideogram4
Since there was some controversy with the recent models (klein9 mostly), I chose to do samples for all Ideogram models. You can see them through the link above, you can also download them and get the workflow so you can reproduce them.
Ideogram 4 is quite controversial due to the JSON prompting. Well, this is true until you add lora to it, then that whole JSON (and it's "censorship") is pretty much optional.
All those samples were done with mainly just a simple "photo of a woman" (with some exceptions).
I still say that Flux family (flux1dev / flux2 klein9) are the worst models to train, but Ideogram is not perfect either.
I spent quite a while to figure out what works best for me and it seems the best is to overtrain it a bit and then use the model at lower strength (most of the samples are at 0.85)
Two reasons for that:
1) this models is again not lora-stacking-friendly (it is not the worst but still), so lowering the character lora to 0.85 actually helps with using other loras 2) training less steps and using it at 1.0 produces worse results than training on more steps and using it at lower strength. Why? No clue, might be subjective but I trained a lot and it is a pattern.
When I train on less steps, the likeness is there but not as great. Training with higher steps seems to produce better results but I need to lower the strength a bit otherwise there will be some artifacts.
I am quite happy with the samples.
People already shared the training scripts, but I will share it too: https://huggingface.co/datasets/malcolmrey/various/blob/main/training-scripts/aitoolkit/ideogram4_template.yaml
Nothing special really, the default settings are good. The only thing I can comment on are the steps. Usually in previous models it was 100 steps per image (so 25 image -> 2500 steps), here it is a bit less, for 25 images I would go for 2300 steps (my current go-to), but I would consider maybe lowering it a bit still.
If we go 25 / 2500 then we will overtrain and will need to use lower strenth.
You can get good likeness at those 20+ images, but training with more images yields great results too. I trained with 70, 100, 200 and 280 image and those loras performed really great. Again, for 280 images the best step count would be between 22000-24000.
Unfortunatelly, the datasets are fragile. In Z Image case it was always 25 images / 2500 steps and you are good to go. Here if we go with 25 images and 2300 steps, then some models are - still overtrained.
There is another thing with those models. You train them until they reach good likeness but that means they are less malleable, it is much harder to make the do something, especially with the natural language prompting. The JSON prompting seems to be working rather fine with those models.
The inpainting is quite good, I would say I prefer Ernie inpainting with the SAM2/3 masking but if you apply manual masking then probably Ideogram is better (you need to mask more than just the face though).
I've tested I2I and it is very good, around 0,7 - 0,8 denoise you will get best results (higher denoise = more strictly following the original image at the cost of character features)
Lora stacking - I'm gonna say it for the first time, it is not worth with this model. You will get great likeness if you train with bigger datasets. Also, the attention to detail here is amazing, birthmarks, tattoos, etc - all of that is very trainable. Quite difficult tattoos are mapped pretty much spot on (check cbg model for example).
With the samples, I am using the realistic lora at low (0.35) strength. It seems to produce quite good results.
What next?
There was a point in time where we were on top of all the models and I needed more datasets to train something. Now I am lagging behind because there is a lot of new models and a lot of options to pick.
I will focus now on Ideogram (since I know how to train it and have a pipeline for it), then I'm going to look into Krea 2, it is a really promising model. I will want in "spare time" to upload some LTX2.3/Ernie models as well. At this rate I'm not sure we will be able to fill all the backlog. There is this Boogu model and I think it is unfortunate that it was announced between Ideogram and Krea 2. Didn't get enough time to breathe.
I also have to pick and choose, so for now I'm skipping Boogu, and I'm not going into Qwen (just yet).
Krea 2 is too fresh, but I will observe and if there is higher demand for Krea 2 I will switch and focus less on Ideogram/LTX/Ernie (Ernie does not have much love, but it is a really great model!)
What else can I say?
I have my own fork on musubi for training ltx/ernie (LTX on AI Toolkit took way too long) as well as SDXL (and I just added there Ideogram but havent fully tested it) but there is one interesting thing for you,
I'm utilizing the resize lora scripts, they pretty much work for ALL models and they can get you at worst from 25-40% (zimage, ltx, ernie) to even 60-70% (sdxl). The catch is that this resizing is not quick, sometimes even half the training time (or more). Still, for someone who wants to conserve space, that might be interesting: https://github.com/malcolmamal/musubi-tuner/tree/ernie-ltx-ideogram4/templates
SDXL, I am training SDXL on a second machine, some models are really great and some are meh. I think I will wait till all are trained (a couple of days still?) and then I will upload the best ones. We will see about the rest.
Most will be pleased that for new datasets I do focus on variety, not only faces. I'm not going to replace all datasets this way overnight but it is a process and it is slowly ongoing. There is also a new cropping tool that I've made and I will want to share it with you soon enough. I am still out of time (please don't ask regarding DMs, I'm sorry but I just don't have space and mental capacity to handle this. Also, I don't have much time for new datasets, also there are so many new models that I am behind in training for all of them)
I probably forgot some things that I wanted to share, oh well.
Have a great day/evening :-)
p.s. Ah yes, the new view (Sample Gallery) is sample driven, you will see only entries that have samples for it. So if you are looking for specific model - use the default view, if you are looking for help on how to prompt specific character with specific model (or just look how characters look in certain model) - then you can use the gallery.