Fix for Img2Img blurry, adding too much noise, changing too much and low step count #16734
Luke2642
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
A scheduler and noise tweak to improve img2img.
If you find at the low end img2img seems to either add noise, makes stuff blurry, or not remove noise already present in an image, this is for you! At the high end img2img changes images too much.
If you're going to try it, also add quicksettings for "initial_noise_multiplier" and "img2img_extra_noise" which will give instant feedback when playing around. I also recommend live previews to see how it's adding the noise during the generation.
This was motivated by using hyper 4 or 8 step Loras, and the UniPC sampler at low steps like 8 or 12, which means low denoising would error as it automatically reduces the step count behind the scenes. So, first switch this setting on:
"With img2img, do exactly the amount of steps the slider specifies. (normally you'd do less with less denoising)"
However, I didn't like how it behaves. I feel img2img should remove noise at the low end, always producing a crispy not blurry image. Then with more steps it should more than denoise, also change the image, without adding visible noise, nudging the image many times to a new composition, like loopback, but in one go. It's a fine balance.
In other words, with these changes you can put a noisy image into img2img, choose low denoising, and it comes out clean. Then as the slider goes up, it doesn't just clean it, it changes it. This seems better to me.
The first thing I tried in sd_samplers_kdiffusion.py replace line 143:
xi = x + noise * sigma_sched[0]
with
xi = x
This helped a bit, but it comes out a bit blurry. So, also change the scheduler timesteps that remove the noise in sd_samplers_timesteps.py, line 92, change the t_encs to t_enc-1:
Let me know if this makes img2img behave more as you like too, or if you have a better idea!
Beta Was this translation helpful? Give feedback.
All reactions