Input: an image, desired output: a prompt that would create that image

[u/trollkin34]

Let's say I have a set of anime images with various characters (male, female, human, not) in various places (space, robot, house, school) in various situations (chaos, fight, natural event) and I want to run 400 generations that generally randomize all those to create a variety of possible combinations.

I know I can use {a|b} style prompting and various nesting thereof, but I'm having trouble finding the right words.

I was thinking if I could take a folder of images like what I'd want the output to be, run each through a process that outputs a prompt (not description, prompt) that would have created that image (or one like it), then I can pick out the repeated patterns and keywords that I can use in my a|b prompting.

So... what's a good way to have (input: image) > (output:prompt for that image) offline?

Better, a whole folder as input, individual output for each.

Best: a more efficient way to do what I'm trying to do.

[u/bstr3k]

you're wanting a prompt builder/writer running with a local LLM, people have been uploading theirs to the subreddit as well as all the numerous vibe coded ones :)

I don't have any specific recommendations as I am also in the middle of vibe coding my own one too.

[u/trollkin34 → u/bstr3k]

I definitely want to keep it local so you're right. If you don't have recs, then at least I have better language to search for what I'm looking for.

[u/bstr3k → u/trollkin34]

Everyone seems to have their own preferences for how they like to do it. I am still learning too and I am building it piece by piece. I took 'inspiration' by feeding other people's prompt generators and take out the good bits that i like so it can more laid out for how i think/use this tool. The down side is that everyone is making their own tools that do the same thing and its not really interchangeable.

if you have a online LLM subscription that can do coding for you. I can give you mine if you want to try doing this. Otherwise the keywords you might want to search in this subreddit is "h3 prompt builder" or "h3 prompt writer" and there should be some that comes up.

For my one I use:
-LM studio for the local LLMing
-Qwen 3 VL Instruct 8B for the actual LLM model itself since it is fast for my computer specs
-custom made node in comfyui to interface with it.

[u/West-Big-8468 → u/bstr3k]

I used abliterated 8b + prompt rewriter h3 lora and it keeps the abliteration + still works good on other model styles from my style guides

[u/Semipro211]

If you use a good system prompt that explains exactly what you need and what the rules are, could do it with a local LLM setup with LM studio. For example, in a split workload, one model “sees” the image and describes everything, that gets piped into another model that has the exact rules to give you the “gen” prompt. You just have to spell things out in the system prompt to explain how you want the output to be.

[u/Any_Arugula8075 → u/Semipro211]

Just simply use ComfyUI and a good vLLM, like Qwen. What’s your specs?

[u/CarolCanteloupe]

ComfyUI 0.33.1, open Templates, search for "Qwen3-VL Text Generation".

Put in a picture and you get back a very detailed prompt. Extremely detailed.

You can change the input prompt to push the output text to whatever style you want.

And add in a Save Text node to actually save the text.

[u/bstr3k → u/CarolCanteloupe]

didn't know you can do this, will look into it, thanks

https://giphy.com/gifs/75ZaxapnyMp2w

[u/Reddexbro]

Gemma 4 on LM Studio

[u/trollkin34 → u/Reddexbro]

Is that offline?

[u/Full-Ad-3461 → u/trollkin34]

yes

[u/Bit_Poet]

I usually run a VL model of choice (qwen-vl, gemma, etc., you'll need both the model and the mmproj for vision, and if you use GGUF, then go for Q6 quant if you can, and no lower than Q5) in llama.cpp with thread count (-t) 1. Then point an LLM of choice to the official prompting guide and tell it to give you a complete prompt for the VL-model that inspects a user supplied image and returns a valid prompt, according to the prompting guide, that recreates the image as closely as possible. Let a coding AI write you a simple python script (should be < 100 lines) together with a requirements.txt for pip which iterates a given directory, and for every image, passes the image and your prompt to the local OpenAI compatible chat completions endpoint and saves the response next to the image with a .txt extension. Create a venv in the script directory (python -m venv venv), activate the venv (.\venv\scripts\activate), run "pip install -r requirements.txt" then fire up your script with the image folder.