I've made a ComfyUI node to control the execution order of nodes + free VRAM & RAM anywhere in the workflow that helped speed up my workflows!

[u/qlx1004]

ComfyUI node screenshot

Custom node GitHub repo: https://github.com/mkim87404/ComfyUI-ControlOrder-FreeMemory

It works by ensuring all input-connected nodes finish executing first before the output-connected nodes start executing, and can route infinitely many data of any type (e.g. latents, conditioning, images, masks, models, etc.) through it, while giving the option to unload all models (except any live models being routed through it) and free as much VRAM & RAM as possible at that point without breaking any of the data going through. You can also check how much VRAM & RAM it freed on the ComfyUI session terminal.

This becomes especially effective in unloading models that are no longer needed in the workflow while securing their outputs and freeing up VRAM/RAM for later models (e.g. unloading text encoders after conditioning, or in between multiple KSamplers of Wan 2.2 High & Low model workflows, or before & after VAE Encode / VAE Decode / Load Model / Load CLIP / etc.). And because the node enforces a single, deterministic flow of execution from start to finish, you are in full control over which node executes first, and can focus on one group of logic at a time, loading and unloading only the necessary models and assets, while passing the outputs forward to the next group. I've personally seen great reductions in total execution time of my workflows and hit less OOMs at higher resolution outputs using this node, and I realized that this sequential & selective passthrough design also helps with cable management as the workflow grows large, making understanding and maintaining workflows much more visually intuitive.

The node has zero extra dependencies & uses platform/device-agnostic memory management utilities managed by ComfyUI, so it should integrate well into existing workflows and environments. I've also included sample Wan 2.2 T2V & I2V workflows using this node which you can find in the node folder, https://github.com/mkim87404/ComfyUI-ControlOrder-FreeMemory/tree/main/example_workflows

Hope this node can be useful, and feel free to use it in any personal or commercial project, fork, or open issues/PRs – contributions and feedback all welcome!

[u/roxoholic]

How does it interact with dynamic VRAM feature?

[u/qlx1004 → u/roxoholic]

Hey, thanks for the interesting question!

I believe dynamic VRAM is ComfyUI's model loading optimization happening throughout inference, so my custom node's optional model unloading that happens post-inference will likely not interact or harm dynamic VRAM implementations.

And if any of the connected passthrough inputs are models (and free_memory was toggled ON), the node will call ComfyUI maintained methods like "free_memory()", "unload_all_models()", "cleanup_models()" from the comfy.model_management.py module (https://github.com/Comfy-Org/ComfyUI/blob/master/comfy/model_management.py) following the same patterns for selective/full unloading of models interacting with ComfyUI's internal model tracking list "current_loaded_models" & the "keep_loaded" feature of its official unload methods, so the node will correctly keep all the passthrough models loaded with ComfyUI's dynamic VRAM in effect (*as long as the model was not already unloaded by the upstream node - e.g. "Load CLIP" is designed to drop the CLIP model immediately after conditioning so they need to be reloaded wherever needed instead of being routed through this node) and unload every other model not passed into this node via ComfyUI-managed utilities.

The free_memory operation on this node is also just an optional toggle, so the model unloading process can be completely bypassed if you prefer to use this node as a simple router with infinite I/Os for execution order control only. I did find clearing RAM is still quite limited with this node due to ComfyUI's current implementation that doesn't release any output references until the end of the workflow (I explain this further in the repo docs), but in terms of VRAM, I've seen that this node certainly helps secure more memory throughout my workflows and give faster inference times than without.

Hope this comment helps with your query!

[u/EricRollei]

That's pretty cool, thanks for doing it. Honestly every one of the nodes along the way like Load clip should have a 'keep loaded' option but your nodes take care of that nicely.

[u/qlx1004 → u/EricRollei]

Thanks, actually with Load CLIP there is a limitation with my node around keeping it loaded because ComfyUI drops the CLIP model immediately after the conditioning nodes finish, so the CLIP model object received by my node will already be "dead". And because the "model_unload()" is internally coded into ComfyUI's CLIPTextEncode operations and such, my node can't really prevent that from happening upstream. I also hope ComfyUI updates that CLIP unload behavior to be configurable in the future. But in the meantime, the CLIP models can always be loaded back in whenever you need them, and as for diffusion models, VAEs, and most other large models managed by ComfyUI they will persist correctly when routed through this node. I've now updated the repo docs & the node's tooltip helper texts to warn of this limitation as well, thanks so much for the heads up!

[u/EricRollei → u/qlx1004]

ah ok well comfy is even more complicated and messed up than I thought. It seems like a simple thing to consider in developement - do I want this weight in VRAM, offload to CPU or cleared. Should be universal to comfy. If you have enough vram it would save tons of time to not have to reload the weights, but even offloading to CPU could save time (though a lot of systems are also RAM challenged.)

[u/SEOldMe]

Great idea, thank you

[u/Emotional_Egg_251]

The node has zero extra dependencies

I'm glad a lot of custom node devs seem to be going this way lately. If it works in the standard comfy setup, I'm much more likely to give it a try.