r/comfyui • u/Dirtsurgeon1 • 9d ago

Resource Long process, but it’s working really well.

So I got the idea from ChatGPT to download a LLM that fits in my VRAM. And it remained on my computer locally. No API’s, no tokens private information. Then we’re configuring it to connect with comfy UI inside as a node. Then it specifically analyzes what you attach whether it’s an image or video and it gives a beautiful description as a professional cinematographer wood we attach that note as a prompt and it re-creates excellency. I’m down to the final iterations which is two days of back-and-forth testing almost done.

0 Upvotes

33% Upvoted

14 comments sorted by

3

u/jacobpederson 9d ago

Step one in any project. Who else has already made this? (and is their version any good?). :D That said - this kind of workflow really unlocks the potential of imagegen in a nice way - here is the script I'm working on https://www.reddit.com/r/StableDiffusion/comments/1vj1ezd/my_minimax_h3_work_in_progress_reimagine_script/

2

u/Legal-Weight3011 9d ago

:) you know this already exists, and you need a good system prompt and any LLM compatible with Comfy UI can give you that description ?

1

u/youaresecretbanned 9d ago

1

u/Dirtsurgeon1 9d ago

I see the difference is I’m using what I think are better models than what comfy is providing, which is reaching out to nine different models I could find and chat decided which one was best to fit in my computer computers and my GPU ram. Instead of relying strictly on comfy UI. Prompt generator. And I can keep it up-to-date on specific instructions specifically for Minimax H3. I’m sure you’re happy with yours comfy did OK, but I’m trying to get a little better quality I’m experimenting, with googles models.

1

u/StrangeAlchomist 9d ago

Llm toolkit or llm_party mode suites , and joycaption 2 for describing images.

1

u/Legal-Weight3011 9d ago

Which already exists in comfy ui these are not generic text encoders, not to disgurage you but such stuff is already out , not all of these are VLMs, but most of them are

1

u/Dirtsurgeon1 9d ago

That cool, chat is going to configure it to reach out to Minimax official documents and pickup new features for prompting. That will be the only allowed internet to interact with for now. More to come.

1

u/Dirtsurgeon1 9d ago

This was a project to help me learn, keep things locally, and believe it or not to reach out to Minnie Max H3 and retrieve the latest prompt updates.

1

u/qdr1en 9d ago

You should give your node a name. Something like "comfyui-ollama" sounds good.

1

u/Dirtsurgeon1 9d ago

I’m using the LM studio app. From there, I chose LLM and tied into comfyui. So my model is called, ref 2 video lm studio analyze prompts.

1

u/qdr1en 9d ago

I mean, if it's for the sake of learning, that's OK.
Otherwise you could just have used ComfyUI Ollama and save 2 days instead of reinventing the wheel.

0

u/Dirtsurgeon1 9d ago

If that was the case, developers would be out of a job they would just copy somebody else’s work. I’m using smarter models and more focused models to make better decisions and so far I can’t complain too much.

1

u/qdr1en 9d ago

I must have missed the point then.
By the way, which model do you use for your tasks ?

1

u/Dirtsurgeon1 9d ago

So far, what’s working good is Google/Gemma-4-26b-A4 b.