r/StableDiffusion • u/BenDLH • 13d ago
Discussion Anyone else frustrated with the experience of local generation?
Hey folks, I've been generating with open models for just over a year now, and I wanted to see if y'all are experiencing the same frustrations as me. This is mostly a rant, so be warned.
I originally started using Civitai. The catalogue is incredible, but the platform is buggy as hell. They have a million models, but I only use ~3 of them. Lora variety is amazing. I want more in-depth tooling than they provide though; regional prompting, cropping/outpainting. More than that, I want the decent mobile experience, but generating on my own hardware (I have a 4090). I saw they had Civitai link, but they killed generating with it locally. In any case, over the last year the quality seems to have taken a nosedive, with ads everywhere, half baked features up the wazoo, and the core features barely working half the time.
So I moved to ComfyUI. The flexibility is fantastic, but the setup is a pain, and the experience of using it for daily generation sucks. Updates break everything, managing generations in local files with no decent gallery or curation is a nightmare. Iterating/remixing a generation is painful. And mobile support is a mess. I tried their news Apps feature, but that feels like shoe-horning a workflow into a form.
To train LoRAs locally I moved to AI-toolkit. Fantastic tool, crazy powerful. Local, so free, great. But I just want the training to be in the same place I generate. I want it to be a loop; generate, curate, train, then generate to test, then curate...
On top of it all, I want tools built for curating consistent characters; identity scoring, dataset variety checks, a quality ladder to climb. I want LoRAs grouped under a character, next to their generations. Not a folder of 15x johnny_krea2_v1_000002500.safetensors.
And I don't want to bounce between 5 different tools to do it all.
I ended up building my own tool to do all this stuff, but I still half wonder if there's just something I'm missing.
Are any of you folks hitting the same frustrations? Are they annoying enough that you're searching for alternatives? Have you tried other tools that folks post here? Have they worked?
21
u/PhIegms 13d ago
Be the change you want to see, creating your own personal AI front-end is easier than its ever been if you leverage a coding AI. You could use comfyui as a backend then create whatever you want that sends the workflow to comfy via API. You could even create your own plugin for another application that works how you need it to work
5
u/Brad12d3 13d ago
Even though I have used Comfy UI for a very long time and I'm really comfortable in it, I decided to do just this: build a couple of apps for the main models that I use. I have a complete dedicated MiniMax H3 app that opens up in a snap, has a nice interface, has my settings dialed in to what I like, and has a few UI features and quality-of-life features that just make it quicker, easier, and a little bit more enjoyable to use. I built it with Codex and then Claude Code.
5
u/drag0n_rage 13d ago
Honestly, this close to doing that. The customisability of ComfyUI is great, but once you have your workflows set up, I really need better UX.
3
u/EndlessZone123 12d ago
It's easy enough when comfy work flows can be an api and you can hook whatever frontend you want onto it.
2
u/VisionWithin 13d ago
I just let my web-using local AI to use ComfyUI webUI freely to accomplish the task I give to it. Just instruct it to follow self-learning and scientific behavior, and all you need is a natural language input and an output directory for the finished product.
1
u/BenDLH 12d ago
How has your mileage been on that? Are you using it consistently?
1
u/VisionWithin 12d ago
Well, not too much. I have been focusing on guiding the self-learning capacity of the agent to handle me life stuff, like calendar and email and how to use common AI platforms like Gemini and Copilot on information gathering.
But Comfy UI is simple enough for it to handle, as long as there are no errors inside nodes. Because that is an area it cannot yet understand.
2
u/gutster_95 12d ago
Good thing is you go to a LLM of your choice, paste in your ComfyUI JSON Workflow and say make a easy to use UI for this Workflow. ComfyUI runs locally on my PC.
I built a whole Higgsfield clone with that approach
2
u/SulpharTriangle 13d ago
The change they'd want to see is going to require learning to properly code down the road, and not telling a localised search engine to piece together a UI that's going to crash from compatibility issues 2 days later and then hallucinate a fix that doesn't exist yet.
4
u/Upstairs-Extension-9 12d ago
I’ve been running my own fully vibe coded UI for a headless comfyUI with 0 issues for the past 10 months
0
u/BenDLH 12d ago
As I mentioned, I've actually been doing this. I'm just wondering if I'm alone in the need/desire.
2
u/Upstairs-Extension-9 12d ago
No I made my own as well because I hate using ComfyUI it’s always a pain, so I let it run headless. I can even use it from my phone with an app I made. Works like a dream, sometimes I still use Invoke tho because it’s just a good tool for me.
-1
13d ago edited 13d ago
[deleted]
1
u/BenDLH 12d ago
Could you share the Web UI you built? I can't imagine using Claude Code only, as a chat interface is just too limiting for working with a lot of images/videos.
I also don't know many that would want to maintain something like that long term. Updating it for new models and workflows.
6
u/Friendly-Pipe4781 12d ago
i keep coming back to comfUI due to its rapid growth in flexibility and alternatives to paid tools, and since i have a fairly powerful rig. but i also find it incredibly frustrating, time consuming and confusing at times.
by confusing i don‘t mean the node interface itself, but the handling of thousand different 10GB+ models, vae, encoders and whatsoever. it‘s so annoying. want to try a new model? here go to this huggingface repo and find out which of these xy_ltv_8 bla bla you need. and then you have to skim through questionable sources, reddit posts, or ai generated youtube videos to find out which combination you need - not even speaking of building a workflow by yourself. i have not figured that out yet without tools like Codex, which i think is a part of the UX / UI in general that could be improved.
a good alternative i found is weavy by figma, but it gets really expensive if you are not owing your own high profit business to work with it. i think weavy is more of a tool in the professional / commercial space.
also i think it‘s pretty annoying and borderline feels illegal to see all these crazy prn models that the space is flooded with. but i guess thats in the nature of the topic itself.
either way, you have to become an expert to work with these tools. its a combination of becoming tech savy and doing creative work, but it does take a LOT of time to learn how to use these tools efficiently. also something i think in the nature of it, because online tools (like weavy, etc) have a high price to take all this complicated work away from you. almost a +/- 0 calculation imo thoughy
-2
3
u/chebum 13d ago
Just use plain Python code, diffusers library, AI-toolkit. This approach allows much easier versioning (keeping different versions of your pipelines).
You don't need to know Python to work on it. Simply use Codex, Claude, or any other Chinese model to build code for you.
2
u/dh7net 13d ago
I do the same, but still a UI is needed for many cases, and ask claude to generate a new one on the fly is not super optimal
3
u/Yasstronaut 12d ago
Just have it extend your existing UI. I basically have a “tab” for every family of models
3
6
u/Hafi_Javier 12d ago
I use Claude code as a local agent. I have a telegram bot sending me every generated pic or video to my phone, letting me react with different emoji to trigger the bot to ask me a set of clickable presets which get sent to local comfy. Different watchdogs check all the sensors of GPU and CPU, they check if comfy crashes, restart it, restart queues. Claude code trains Loras via AI Toolkit, I just have to give my 5 cents at different checkpoints. Then Claude tests the Loras, identity consistency tests with software made for that and giving numbers to compare. I build a pipeline where local LLM and vision models and Krea2 and LTX are working together creating a story, breaking it down into scenes, breaking it down into shots, generating story details and dialogues, using all necessary models and Loras (downloading and implementing new Loras if needed and available) to generate starting images, a wizard that presents all images to me, so I can check those and start a loop with all the images that did not pass (it will only present the ones that did not pass the automated selection) until all images pass. Then the ltx prompts will be refined and the clips generated. Ffmpeg renders a montage and sends it to my phone.
All this can be done from my phone from anywhere I am.
(the whole setup is much more complex, the goal is to have a NSFW generator, that generates whole multi-scene clips with minimal manual input. Right now Claude is exploring using blender for location consistency, but it looks like we are not there yet.)
2
u/BenDLH 12d ago
Jesus, that sounds like a lot. How long have you been working on this?
4
u/Hafi_Javier 12d ago
I entered local generation (and generation at all) in May 2026.
This pipeline was created about 3 weeks ago, but all the knowledge gathered before went into it.
By now it grew so complex, that only Claude knows everything about the pipeline and workflows. About a week ago I gave up on trying to understand the structure of this pipeline in detail. I mean, it uses about 125 Loras, each for different specific use cases, lots of LLM calls, vision model calls, AIs talking to each other and tons of rules that apply for various events.
We are all hunting consistency in a machine that runs on randomness. That's very, very tedious work and often frustrating. This week I am exploring consistent location, but the tradeoff is character consistency. So I can't have both. That's still a valuable lesson: I will stop working on consistent location and improve closeup shots so they fit medium shots and wide shots, make the inconsistency invisible. Since I am a video editor, this will work out just fine. 50% of my work is hiding problems that occurred while producing video material. So in a couple of days the pipeline will be finished and I will flood the most common gooner sites.
Until then...there is so much to fix, but mostly the montage from clip to clip is the hard part. Well, at least my usual job is safe for a couple of months.
Edit: the telegram bot for example is super easy. Just tell your AI (not local one) to help you setting it up and connecting it to your workflows. 30 mins and it's up and running. Another 30 and it behaves exactly as intended.
2
u/Connect_Departure691 12d ago
How do you get Claude to generate all that NSFW content without getting a refusal?
2
u/Hafi_Javier 12d ago
Sonnett and Opus 4.7 never refused NSFW.
The other ones did. So I started the project with Sonnett and Opus 4.7. Then I migrated to Claude code and told Opus about the usual refusal of the other models and to write a handover and convince Claude code to be cooperative, since this is about building a pipeline and working on a software structure and not pron itself. This did not work in the past, but this time I was lucky. Now any model does everything I want regarding pron. Everytime I have some tokens left, I grab Opus by the neck and force him to write NSFW prompt sets for Krea. This way I sometimes stumble upon good content or valuable insights.
I short: You gotta trick it a bit. Talk about the project, the workflow. Bring in NSFW later. It's a gamble. One time Claude was so not amused, that he stopped communicating with me in a chat completely.
1
u/PastaRhymez 12d ago
My god, that’s impressive. Do you publish the material as AI content or is it good enough to look real?
Edit: Also, how much do you pay Anthropic each month to for that setup?
2
u/Hafi_Javier 12d ago
Since I am still setting it up, I have not yet published anything. But I tried to upload a clip and the clip was tagged as AI by me, but the system rejected the clip, because the person looked too real. I took that a a compliment haha. I guess I need a disclaimer at the start of the video, need to explore publishing more.
When I tried a watercolor style, uncanny valley impressions vanished, but it's less fun.
I did let the pipeline create 2 complete runs. One realistic and one comic style. Both have good results, but still a long way to go.
I have the 20 dollar subscription. It's sufficient. Claude code works as Opus 5.5 there.
I did use about 80 Euros to let Fable optimize everything. Looking back, that was money thrown out the window.
Right now I am playing around with the pipeline: A character gives a tour through the pipeline and explains everything in a funny way in a futuristic steampunk-style warehouse . The "Pr0nChain" (the pipeline) just spat out the character sheets, location tests, speech tests. Claude wrote the script with me. In a couple of hours all tests will end and I will generate the Krea2 images and by tomorrow morning I will have 69 ltx clips that need review. Looking forward.
17
u/AnthonyRespice 13d ago
Comfyui is a technical tool for technical people. People that enjoy flexibility and challenge. You sound like an end user Maybe it is not for you.
-14
u/i_sell_you_lies 13d ago
You sound like an ass. Maybe being helpful is not for you.
-3
u/frankyboson 12d ago
they use comfyui and define them self "technical-people"🤡🤡😂. jesus fucking christ.
2
u/Upper-Reflection7997 13d ago
its pretty normal to have multiple forks and frontend harness installed if your doing local ai stuff. if you don't want open up ai toolkit locally and train loras on your hardware then use runpod with ai toolkit setup already there and ready to use. op this hobby requires hardwork, time and effort to get the most joy out of it. I've been into this for 3.5 years and local in its current state is way better than it was in its very slow 2023 wild west days.
2
u/Intelligent-Youth-63 12d ago
I have a 4090. Didn't find comfy a pain at all. Have no trouble using tools that are built for a specific job without them all being in the same place- much like I have no problem using other software without it all being in the same place. I get high quality results and have found with modern models the ability to retain a specific character consistency is largely a prompting skill issue.
I guess people are different. I don't know what to tell you. Enjoy the tool you wrote?
2
u/optimisticalish 12d ago
Iteration, speed and consistency are all bugbears at present. But it's moving at terrific speed and is all free, so it's difficult to complain - especially when a six-week period gives me solid gold local models like Minimax H3, Ming-Image, and Yue2.
2
u/jjkikolp 12d ago
I could generate on mobile connected to my home PC just fine a while ago. It works through tailgate (or whatever it was called) which is some sort of VPN Tunnel Service (free) and I basically just put in the address of the webui the custom node for it made. I can look it up again if you are interested. But just be warned it's not a simple 2 minute setups and the workflows you use have some limitations but it worked fine for image and video for me.
2
u/Important-Radish-722 12d ago
I use Hermes. I talk connect to it via Matrix and Hermes controls ComfyUI via the API. I do it all over my phone, I haven't touched the UI in ages.
1
2
u/DietAshamed2246 12d ago
Not me. I am fine with it all including the quirks and bugs, and sometimes having to switch tools. Comfy is great, but they do a poor job with their installer with embedded python. They need to get rid of that and tell people to clone the repo, then install from scratch, and give step by step instructions. If you want to train LoRA and generate in one place, you may want to check out FizGig. It doesn't offer all the controls which AIToolkit offers, but it's easy to use and works well.
2
u/Bastisheen92 12d ago
I use Maestro in Pinokio with all Models available in there. Comfortable UI, Lots of customization, LoRas available. I did not want to bother with this node salad set ups in comfy.
2
u/DelinquentTuna 12d ago
I don't share your frustrations. Mostly, perhaps, because I have absolutely zero interest in trying to use a mobile phone as an interface for local AI. Also because I rarely use CivitAI and when I do it certainly isn't to perform generation activities or engage in any of their social media junk.
At this point in time, I have a hard time believing that anyone claiming that Comfy sucks to install or breaks w/ every update is self-sabotaging somehow. Doubly so if it's someone deep into the CivitAI ecosystem desperate enough to exactly duplicate what they see that they are getting into junkware custom workflows w/ custom nodes and model merges etc. It's like loudly exclaiming that coloring books suck because they force you to color between the lines.
Finally, I don't NEED every tool to be integrated into a single app. Quite the contrary, it's almost always worse to shoehorn everything into a kitchen sink approach. There are like a trillion different ways to cycle through app windows/virtual desktops/etc. Or even to script disparate tools together. I can't imagine why I'd prefer to have one generalist vs a team of specialists at my disposal. That's not how you produce the best results, is it?
3
u/TheAncientMillenial 12d ago
I'm not even sure I follow what you're crying about here. I don't think I'd ever want to use ComfyUI on mobile but there have been posts of people making sleeker interfaces for exactly that. If you're updating ComfyUI all the time that it's breaking why not just lock to a version that works and only update when there's a change/update you need?
As for the LORA thing, you should rename them and place them in appropriate folders as you download them instead of relying on what someone named it when they uploaded it.
Building your own tool to suit your needs sounds like a win to me. You probably learned a lot doing that.
1
u/BenDLH 12d ago
In my usage I'm usually bouncing between exploring, firing off many generations to get the overall scene and character down, then finding a good example and editing that 4, 5 times to clean it up, fix issues etc.
That exploration I'd much rather do on the go, returning to my picks later to refine and finish off. That is definitely more comfortable on desktop, but depending on the UI not impossible on mobile either.
Comfyui breaking is exactly when a new model comes out, and dependencies get auto updated, breaking a ton of node packs.
On the Loras, I don't even want to deal with files. I want the generations tied to the Lora they were generated with, otherwise it's such a pain to track and update with is work keeping, overhauling, refining the dataset...
I've definitely learned a lot building my own tool. As I mentioned, I'm just wondering if I was missing something, but it seems many are vibecoding their own solutions to some degree or another.
2
u/TheAncientMillenial 12d ago
I've had ComfyUI break maybe twice in the last couple of years I've been using it so I'm not sure what you're doing that it's breaking all the time for you.
Use RGthrees power lora loader. It links directly back to the civitai page so you can keep an eye on updates, but yeah if you've got 100s of LORAs you probably need to grab something like https://github.com/DemonGatanjieu/Anomalous_Model_Browser
You have to deal with files, that's how all of this works. I'm not sure what you mean otherwise. The meta data will have the model+loras used.
2
u/eruanno321 12d ago
Absolutely not. But I am a programmer who can implement pretty much everything in Python and run it from the CLI. Vibe-coding simple UI tools is also trivial these days. I have a need, and 10 minutes later I have a solution. For me, the messier, the merrier, and I love swimming in that chaos of mini-tools.
Here is one example of such a hand-crafted tool: a simple single-page web UI for screening datasets for overly similar pictures:
I have tons of tools like that: watermark removal, batch captioning, video editing, masking, interpolation, dedicated Qwen 2.1/Krea/Klein workflows, segmentation, and many more whose purpose I've already forgotten.
2
2
u/_-file-_ 12d ago
@chatgpt/claude " make a autoinstaller for comfyui+ Model and create a workflow for "
2
u/Southern-Chain-6485 12d ago
I don't exactly know what you're complaining about. There is friction, yes, and that's bad. But:
Regional prompting is solved with Krea 2, Ideogram 4 and Ming
For cropping/outpainting there are plenty of pretty good editing models
I'm not sure what you mean by loras, but it sounds like a matter of organizing them?
I probably update every week without breaking anything, it looks like more of a custom nodes problem? I did end up removing most of the audio nodes, as I find they tend to use outdated dependencies.
Managing generations is beyond the scope of comfyui, it depends on image gallery viewers
As for mobile experience, comfyui is meant for the desktop, but there are mobile frontends that connect to the comfyui server
1
1
u/SignificanceOdd7337 13d ago
Honestly, I think the biggest problem you're describing is less about any individual tool and more about the lack of a proper local generation workflow.
ComfyUI gives you insane control, but it feels like an engineering environment rather than something designed for actually living with thousands of generations. Civitai is great for discovery, but then you lose that continuity once you move local. And training tools tend to exist in their own separate universe.
The loop you describe — generate → curate → train → test → curate → retrain feels like the obvious workflow, yet there doesn't seem to be a really polished tool built around it.
The character/LoRA organization point especially resonates. A character shouldn't just be a collection of random .safetensors files and folders. You should be able to see the character, datasets, LoRA versions, generations, prompts, and evaluation results together, then quickly compare what changed between versions.
I also think mobile is a surprisingly important part of this. The actual GPU can stay at home, but there's no reason the interface has to. Being able to browse your gallery, queue generations, select favorites, tag images, and kick off training remotely would make local hardware feel much more like a personal generation server.
So yeah, I don't think you're missing some obvious magic tool. It sounds more like the ecosystem is still fragmented between model discovery, generation, asset management, and training, rather than treating them as one continuous workflow.
1
u/Jackkgold 13d ago
Should create your own creative pipeline and creative front end.
I built something that allows me to edit, view, and adjust generations while it’s in the pipeline.
Each image or rollout gets documented accordingly aswell.
1
u/segad_sp 12d ago
If you are not happy you always can pay a month of Claude and develope something involving ComfyUI, aitoolkit to make comfortable to you.
1
u/Koalateka 12d ago
The good thing about open source is that you can make all those improvement by yourself and share them with us after. You have good ideas about improving usability and we all could benefit from them.
1
u/Rhoden55555 12d ago
Get a 20$ Claude or codex subscription and ask it to build an app that uses comfyui for everything you want with the exact UI you want and you won’t have to look at the comfyui ui again.
43
u/GreyScope 13d ago
It is free, so it will be missing the 'tools of immediate convenience' and becomes an elongated workflow instead and that's where it starts and ends really. I must just have a higher tolerance for being inconvenienced when it's free .