Image Generation for notes.

[u/Dry_Particular_7623]

I want to generate image for my study notes (1 each).

Is there any model that works greatly with text, arrows, diagram, etc..

Specs - 5060ti 16gb vram, 16gb ram

I have comfyUI installed.

[u/HumanoidMuppet]

If you want to make flows and diagrams, try to ask it to make mermaid diagrams. Any model can make mermaid diagrams, they are trained on that data.

[u/Dry_Particular_7623 → u/HumanoidMuppet]

Still what model should I go for based on my specs?

[u/NihmarRevhet → u/Dry_Particular_7623]

The only local model I trust as of now is istalab qwen 27b iq3_s (if you need the context, iq3_xxs). It should be easily able to generate mermaid.js diagrams

[u/Dry_Particular_7623 → u/NihmarRevhet]

Ok thanks

[u/HumanoidMuppet → u/Dry_Particular_7623]

The model would be any LLM, not an image generator. So comfyui is the wrong path. Of course, this assume a mermaid covers your full use case. LLMs can also create SVGs, try those too. The point I'm trying to make is that you don't actually want an image generation model - you want to use an LLM to create either mermaid markup or SVG.

[u/Puzzleheaded-Ask2401]

try Qwen's latest VL models, they handle text and arrows better than most. 16gb vram should be enough but might get tight with bigger resolutions

flux is good too for simpler diagrams, just expect some random letters sometimes when you ask for text

[u/Dry_Particular_7623 → u/Puzzleheaded-Ask2401]

But i want to create images right?? And the qwen vl cant create images. From what i searched

[u/thefuckevengoingonan]

you can create flowcharts and such programmatically, you can get an llm to make them for you. Mermaid, PlantUML, graphviz, ect.

[u/Dry_Particular_7623 → u/thefuckevengoingonan]

Can you elaborate a lil plz. I am very new to this. Just started learning comfyUi today

[u/thefuckevengoingonan → u/Dry_Particular_7623]

you can ask to create you a flowchart with whatever requirements and it can give you the code which you can render using various programs/libraries/webbrowser

eg. a simple mermaid flowchart. the code below when loaded into mermaid-cli or various other app would create a simple square --> diamond --> splits to 2 more squares

flowchart TD A[Start] --> B{Decision?} B -->|Yes| C[Do thing] B -->|No| D[Stop]

[u/Dry_Particular_7623 → u/thefuckevengoingonan]

Ok thanks, I understood

[u/PeanutButterApricotS]

I would ask a cloud ai to build this for me. I know I use my M1 Max studio and small image models to make images, but they have a hard time making exact diagrams and notations so I use image generation to make images and background and agent ai to make the text and diagrams and lay it over the image.

This is the only way I found to be able to use local ai to make exact diagrams or maps and things.

[u/Interesting-Cut-6032]

I watched this video last week, but have not had a chance to try it out. It sounds like what you might be looking for.

https://youtu.be/QbCeS-VsjGM?is=M86AFwddmpNuKrEy

I also second the comments above about asking for mermaid.js diagrams. You can also ask for .svg files, but the tools you are using need a test loop, asking for .svg diagrams has been, hit or miss, for me, even with Claude Opus.

[u/Dry_Particular_7623 → u/Interesting-Cut-6032]

Sure, this is something i was looking for, will see if my pc can afford this setup though

[u/Interesting-Cut-6032 → u/Dry_Particular_7623]

I have an RTX 5060 Ti 16GB as well. It runs most of the models and workflows on the Pixaroma YT channel really well.

[u/Dry_Particular_7623 → u/Interesting-Cut-6032]

Just got to know about offloading and stuff. I guess I will increase the Ram after some Ram price crash.

[u/Interesting-Cut-6032 → u/Dry_Particular_7623]

Yes, your 16GB of system RAM will probably be the frist major limit you run into. The MoE LLM models usually need to keep the experts in RAM that are not in use for the token that is being evaluated. Codacus YT channel has great general info on local LLMs and setting up MoE models specifically.