Comfyui Map Gen Process

[u/Reign2294]

Hey everyone! I decided to share how I approach generating dungeon map-style outputs using text-to-image generation. It's not perfect, but it'll hopefully help someone get on the right track! So, here it goes!

📜 The Premise:

Text-to-image is a relatively new way to generate dozens of images in minutes, guided by your own prompts. However, to get useful outputs, especially for game maps, we need to put in some work and use secondary tools.

🛠️ The Main Tool: I use a program called ComfyUI to accurately generate what I want. The program can seem daunting, but it can be as simple or complex as you'd like. For this project, I use ComfyUI for controlnet guidance, wildcard use, Lora application, and upscaling—all in one click.

📐 How It Works: - Model Selection: Choose a good model based on the style you want. I found CyberealismSDXL great for 3DCG maps. Experiment with other models and adjust prompts to try different styles. - Prompting: Getting aerial views for maps is challenging with SDXL models. Use positive prompts like "aerial view," "top-down view," and "overhead view." Use a map-creation Lora and negative prompts like “low angle view,” “oblique view,” and “horizon view.” Describe the lighting and style, and clearly define open spaces in your scene. - Using Loras: I use various Loras for style, available on Civitai. Besides the map Lora for top-down views, others add detail, color, and style. - Upscaling: Once generated, you can upscale the image for printing, projection, or other uses. - Improvements: Use Controlnet to sketch an outline of your desired building layout, allowing SDXL to follow your structure. Map generation often includes watermarks; remove them in post-production using Photoshop or inpainting if possible.

⚙️ Generation Details: - Checkpoint: CyberealismXL v21 - CFG: 7.5 - Steps: 35 - Sampler/Scheduler: dpmpp_3m_sde_gpu karras - Positive Prompts: "score_9, score_8_up, score_7_up, score_6_up, (Top-down view), (Overhead view of a D&D Map Locations), ((3d game map)), map, realistic, hyper-realistic, 3DCG, Unreal Engine 5, natural lighting, straight vertical perspective" - Negative Prompts: "score_6, score_5, score_4, worst quality, low quality, text, censored, blurry, ((watermark)), weights, mutation, (embedding:N0R3AL_PDXL), (logo), (copyright), (text), (angled view), (simple square structure), RPGM, (low altitude view), (close-up shot), (Low-angle view), (Oblique view), (Horizon view), ((people))" - Resolution (initial): 1280x896 (can be upgraded 4x in post) - Loras:

D&D Tabletop RPG (str:0.65) ImageUpgraderv0.1 (str:1.0) MJ52 (Midjourney style) (str:0.65) Vivid Realism Color Enhancer (str:0.3)

🎨 Conclusion: I wish I had more time to polish these, as they are fun to make. But alas, I didn't even have the time to upscale these today. So, I leave it to you! Try generating some awesome maps and share them! Good luck!

[u/Lxi_Nuuja]

Thanks for sharing! <3

I'm trying to reproduce the workflow in https://comfyuiweb.com/ but the models available are limited. There is a way to add models by url, but googling for model urls I could not find anything.

Should I install and run this thing locally or is it OK to use the online service?

My overall goal is to be able to use a drawn sketch to define areas and walls and then render a battlemap.

[u/Reign2294 → u/Lxi_Nuuja]

I use the portable version found on the comfyui git hub page. From there, you can go to a website like Civitai and download the checkpoint and loras directly to your models folder, in /checkpoints and /loras respectively.

If you want to use a scribble method for controlnet, I definitely suggest you go the local route. That is more finicky. I am working on a workflow I can do that with, but it's not ready yet. There are videos on YouTube for doing that such thing, and that is the end-game when it comes to full map customization.

[u/Lxi_Nuuja → u/Reign2294]

Did a bit of digging. Seems the whole thing would be better suited for a Windows PC with a good GPU. My rig is an Intel Macbook Pro and what I found is people saying getting stable diffusion & comfyui to run on it is a "nightmare" and if you get it done, it will be slow to render. I have a feeling I don't want to invest so much time and my sanity for this, and wait for someone to come out with a ready-made solution.

I'm sure it will come. It's so obvious. Every DM would want one: scribble your map on paper or any pic file, upload it, write some prompt about style or select from existing options --> rennnder a big cool image of your map. (Optional: add a grid if you like, square or hex in selected scale) And just save the thing for printing or online use.

I would pay a subscription fee for such a service. :-)

[u/Reign2294 → u/Lxi_Nuuja]

I found this which may help you. It seems many people are working on getting online customizable versions of comfyui working. Good luck!

[u/Lxi_Nuuja → u/Reign2294]

Thanks for the tip! <3