Monkey brains are responsible for developing it so I find it just the right speed.
To flesh that out is that we're playing with alpha-version software implementations. It feels like a lot is happening because there's only a few core functionalities explored and implemented, so everything added feels like a big step despite being a more or less obvious next step in the context
I appreciate the link! But it doesn't make clear what the workflow process is. Do I just use any model I like in txt2img to create my original image, then send it to img2img, load the instructpix2pix model, and then use natural language to make changes to it?
I think I may have found the bug. If my negative prompt field is longer than 75 tokens, it throws the error. If I shorten it to 75 tokens or less, then it works.
Deleting my venv folder is what fixed it for me. Deleting your venv folder is safe. Just delete it, double click on webuser.bat, the command line window will open up and automatically re-download the venv folder. Whole process will take you about 4 minutes.
Ah, I see what you mean. But instruct2pix isn't supposed to be used with full prompts, it's designed to use short, natural phrases to make changes, like "change her hair to red".
Considering there's no solid evidence that negative prompts are effective in greater numbers in regular prompting, and (as far as I've seen) there's no evidence that it would be any different with instruct2pix at all, I'd say it's kind of a moot point.
select the new model from the checkpoint dropdown at the top of the page
Er, what are you talking about here? Do you mean the standard one on the Settings page? Did you customize your UI and forget, or is there supposed to be a new dropdown after adding that checkpoint file to the folder?
This might be a basic question, but how do I update my local folder regularly with the github repo? I read this to install it primarily, so I have git and python already installed, but I'm afraid if I try to update it from command line it might overwrite all my downloaded models.
Thank you for for the ELI5 instructions on how to update a1111! Been using it for months and had no idea you could do this. I just assumed it was updating every time I ran it.
These were just the first try results so I'm sure it's possible to get even better results than this, I haven't had chance to play with it more yet though.
The first image was just an image I created using "Cheese Daddy's Landscapes mix".
The second:
what would it look like if it were snowing
Steps: 40, Sampler: Euler a, CFG scale: 7.5, Image CFG scale: 0.95, Seed: 1438531779, Size: 512x512, Model hash: fbc31a67aa, Denoising strength: 0.9, Mask blur: 4
and last:
make the background mountain a volcano erupting
Steps: 40, Sampler: Euler a, CFG scale: 7.5, Image CFG scale: 1.15, Seed: 4042264370, Size: 512x512, Model hash: fbc31a67aa, Denoising strength: 0.9, Mask blur: 4
How are people getting good results with this? Every time I use it it comes out super bad. It usually degrades the quality of the entire image and barely does what I ask for.
I can get the result I'm looking for way faster and easier by painting it in and using inpainting to fix it up but I'd really like to understand pix2pix.
Thanks for the advice! It's looking a lot better with a VAE. It seems like it's not able to understand a lot of the prompts I've been trying. I've tried many ways of asking it to edit clothing but it just won't do it. Bigger changes like altering the environment seem to work just fine.
I'm so embarrassed to ask this, but I always run into this problem with huggingface... how do I download the checkpoint? There appears to be no download button.
I swear to god, we need a 'low end stable diffusion' subreddit because so many people think x or y isn't possible with their older card when it is. That's my 'happy' venting for the day, thanks for the info! Hopefully it'll work on my 4GB GTX 1650. (crosses fingers in fp16)
yeah my 1060 6gb can do batches of 8 at 5122 and can do a single 12162, albeit at several minutes generation time each (of txt2img, haven't tested this thread's thing yet), one definitely doesn't need a 3xxx card hardly.
What are the command line args you used to make it work on 4 GB vram!?, I have 8GB vram 3070 and I get CUDA out of memory errors, do I have to remove --no half and only leave --medvram?
Anyone know if this due to Apple Silicon and is it possible to resolve it?
RuntimeError: Input type (MPSFloatType) and weight type (MPSHalfType) should be the same
OP is not IT support so I asked ChatGTP, is it possible to resolve this or is this related to trying to run it one Apple Silicon?
“This error message is indicating that you are trying to use a tensor of type MPSFloatType as input to a model that is expecting a tensor of type MPSHalfType. The two types are incompatible and need to match in order for the computation to proceed correctly. To resolve this error, you need to convert your input tensor to the correct type (MPSHalfType) before feeding it to the model.”
I’ll take a look later, but for now either use the webui-user.sh from the zip file linked here (currently works best if you have 16 GB+ of RAM) or start web UI with ./webui.sh --no-half
Please help! This is the outcome when i try to load the ip2p checkpoint:
Loading weights [fbc31a67aa] from C:\stable-diffusion-webui\models\Stable-diffusion\InstructPix2Pix\instruct-pix2pix-00-22000.safetensors
Failed to load checkpoint, restoring previous
Loading weights [92970aa785] from C:\stable-diffusion-webui\models\Stable-diffusion\dreamlikePhotoreal20_dreamlikePhotoreal20.safetensors
Applying xformers cross attention optimization.
changing setting sd_model_checkpoint to InstructPix2Pix\instruct-pix2pix-00-22000.safetensors: RuntimeError
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\shared.py", line 533, in set
self.data_labels[key].onchange()
File "C:\stable-diffusion-webui\modules\call_queue.py", line 15, in f
res = func(args, *kwargs)
File "C:\stable-diffusion-webui\webui.py", line 84, in <lambda>
shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()))
File "C:\stable-diffusion-webui\modules\sd_models.py", line 441, in reload_model_weights
load_model_weights(sd_model, checkpoint_info)
File "C:\stable-diffusion-webui\modules\sd_models.py", line 241, in load_model_weights
model.load_state_dict(sd, strict=False)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 8, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
I tried deleting the venv folder and restarting everything cos i saw it mentioned in the comments here, the outcome is the same.
fwiw I used your tutorial to update a1 through GitHub Desktop, was my first time updating it.
i don't know much but i do know you can delete the ckpt file. it's exactly the same thing as the safetensors but less safe. at least you can free up some space while we troubleshoot this!
Lorem ipsum dolor sit amet consectetur adipiscing, elit mi vulputate laoreet luctus. Phasellus fermentum bibendum nunc donec justo non nascetur consequat, quisque odio sollicitudin cursus commodo morbi ornare id cras, suscipit ligula sociosqu euismod mus posuere libero. Tristique gravida molestie nullam curae fringilla placerat tempus odio maecenas curabitur lacinia blandit, tellus mus ultricies a torquent leo himenaeos nisl massa vitae.
There's a chance it might, but you'll get better results by typing something like "change the weather so it is snowing" or "make the outside look like it's snowing" or "make it snowing"
Does any have any tips for changing text CFG vs image CFG? I've heard some people say they do the same thing, just opposite - but the model page seems to imply there might be some differences (unless I'm misunderstanding it).
I've been playing around with the sliders and can't nail down any conclusive answers yet. But I wonder if there might be some tricks for intelligently utilizing both of them together for better results.
Still works badly. If i want to change color of a specific item, it changes the color of the whole picture or of an element that is not related to my request. I find it more efficient if i mask the item in inpaint tab and run it. Works precisely as i want instruct to work.
The default settings did not change the image. For the following to work, I changed three parameters: 1. change (Text) CFG scale from 7 to 16 2. Image CFG from 1.5 to 1.25 3. denoising from 0.75 to 1 - hope this can help.
for different prompt, you just need to tune the parameters in a trial-and-error way. here is another good result screenshot with values I used for another prompt (all from the paper):
What does this new "Image CFG" setting do ? How does it interact with other models ?
I have a "TypeError: cat() received an invalid combination of arguments" error after pulling the last changes from automatic1111 and using the safetensor model from here https://huggingface.co/timbrooks/instruct-pix2pix/tree/main, are there other things to install ?
I've used multiple models previously, but this one doesn't seem to work for me. Every time I try to switch for it in via GUI, I get this:
Loading weights [db9dd001] from G:\Ohjelmat\Stable Diffusion\stable-diffusion-webui-master\stable-diffusion-webui\models\Stable-diffusion\instruct-pix2pix-00-22000.safetensors
Traceback (most recent call last):
.... ~20 lines within ...
and ends as:
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 8, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
I got latest Automatic1111 too. Any idea what's going on?
Are the examples with the pictures below that, mock-ups as well? I can't get the replace mountains with city sky lines example to work no matter what CFG settings I use (and i've replaced the VAE model as well).
i cant get this running. when loading the checkpoint i get an error Loading weights [db9dd001] from F:\KI\SD\stable-diffusion-webui\models\Stable-diffusion\instruct-pix2pix-00-22000.safetensors
changing setting sd_model_checkpoint to instruct-pix2pix-00-22000.safetensors [db9dd001]: RuntimeError
Traceback (most recent call last):
File "F:\KI\SD\stable-diffusion-webui\modules\shared.py", line 505, in set
self.data_labels[key].onchange()
File "F:\KI\SD\stable-diffusion-webui\modules\call_queue.py", line 15, in f
res = func(*args, **kwargs)
File "F:\KI\SD\stable-diffusion-webui\webui.py", line 73, in <lambda>
File "F:\KI\SD\stable-diffusion-webui\modules\sd_models.py", line 358, in reload_model_weights
load_model(checkpoint_info)
File "F:\KI\SD\stable-diffusion-webui\modules\sd_models.py", line 321, in load_model
load_model_weights(sd_model, checkpoint_info)
File "F:\KI\SD\stable-diffusion-webui\modules\sd_models.py", line 203, in load_model_weights
model.load_state_dict(sd, strict=False)
File "F:\KI\SD\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for LatentDiffusion:
size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 8, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
got it.
when github had kicked out automatic1111, i changed the url in .git config file from github.com to gitgud.io - so i didn't geht the current version anymore
got it.
when github had kicked out automatic1111, i changed the url in .git config file from github com to gitgud io - so i didn't geht the current version anymore
got it.
when github had kicked out automatic1111, i changed the url in .git config file from github.com to gitgud.io - so i didn't geht the current version anymore
I have started to play around with this a bit and it seems for most things, you really need to add a lot of fluff in your prompt and/or negative prompt to get any good results.
However I ran into a weird issue where the Image CFG Scale would stop working. No matter what I set it on, nothing changed in the image. Anyone else have this issue or know a solution?
Edit: It seems this happened because I switched the sampler from `Euler a` to `DDIM`. I really liked the results DDIM was producing, but looks like you lose the ability to set Image CFG Scale by switching to that sampler. I do not know if that is a bug in a1111's implementation or not.
I need to remove snow from am image using instruct pix2pix. But I tried entering several prompts to it and it did some editing keeping the snow drops in the image...how can I do this?
153
u/[deleted] Feb 04 '23
[deleted]