Vanilla image noise generator option#12812
Vanilla image noise generator option#12812lllyasviel wants to merge 3 commits intoAUTOMATIC1111:devfrom lllyasviel:dev
Conversation
|
and we will need something like this to check if some extensions' implementation is correct, like control-lora, revision, ip-adapter, etc. |
|
|
||
| x = x * sigmas[0] | ||
| if opts.use_vanilla_random_number_generator: | ||
| # https://github.com/comfyanonymous/ComfyUI/blob/a57b0c797b0670e6fd48ff3cf1f37dbf0d0a67b5/comfy/samplers.py#L716 |
There was a problem hiding this comment.
Makes more sense to link to the original sgm repo where this originates.
| # https://github.com/comfyanonymous/ComfyUI/blob/a57b0c797b0670e6fd48ff3cf1f37dbf0d0a67b5/comfy/samplers.py#L716 | |
| # https://github.com/Stability-AI/generative-models/blob/477d8b9a7730d9b2e92b326a770c0420d00308c9/sgm/modules/diffusionmodules/sampling.py#L50 |
| })) | ||
|
|
||
| options_templates.update(options_section(('compatibility', "Compatibility"), { | ||
| "use_vanilla_random_number_generator": OptionInfo(False, "Use vanilla random number generator. Can reproduce other UI's results (Fooocus/ComfyUI). Require full restart."), |
There was a problem hiding this comment.
This feels too wordy for me so linking back to this PR makes more sense imo. Also a minor fix to use a chained function indiciating it requires a restart. (I will test this shortly, but -- does it actually?)
| "use_vanilla_random_number_generator": OptionInfo(False, "Use vanilla random number generator. Can reproduce other UI's results (Fooocus/ComfyUI). Require full restart."), | |
| "use_vanilla_random_number_generator": OptionInfo(False, "Use vanilla random number generator").link("PR", "https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12812").info("can reproduce other UI's results").needs_restart(), |
Something also tells me this is the only actual change needed. I will test that in a moment.
|
Okay, this PR is definitely making it more complicated than it should -- I get identical results just by making your sigma change from sgm's repo, and setting the existing x * sigmas[0]
x * torch.sqrt(1.0 + sigmas[0] ** 2.0)
I think more notably this changes it for all k-diffusion samplers across the board. It doesn't look like this was in the original ldm repo either. While I'm personally ok with including it disabled by default, do you know the mathematical rationale behind the change? |
|
I've made a PR with the only change needed. #12818 |
|
I do not know the reason for this sigmas, probably related to tsnr. |
|
There is an existing option to disable noise overriding of the SDE seeds. That's probably what you're thinking of. Look for If there needs to be yet another implementation for making that consistent across different software that would be better for a separate PR, just as I made the sigma change separate. "Vanilla" in this sense here doesn't make sense when it's replicating ComfyUI's implementation. |
|
“Vanilla” means it is consistent to This is also used by others like diffusers and native scripts written by people without taking special cares about noise. |
|
wait a moment - this seems to break some other things related to img2img - Let me take a look first and close it for now |
Description
Right now webui’s noise generating system is very powerful yet convoluted, making it difficult to produce consistent results with other software.
This pull request add an option “use_vanilla_random_number_generator” to temporarily disable some webui hackings to the noise generators.
The result is that now we can perfectly reproduce results from other software like comfyui, fooocus, etc. This pull request also include a minor fix in the dpm++’s first sigmas for the noise processing so that the results are even more consistent.
My recommendation is that the option “use_vanilla_random_number_generator” should only be targeted to code development, since introducing this to users will cause some confusing about reproducing results from old seeds.
This option is very useful when we want to implement extensions and check those extensions’ behaviors across different software.
Example 1
cinematic film still, a car in the city street . shallow depth of field, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy
Negative prompt: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 8, Seed: 123456, Size: 1024x1024, Model hash: e6bb9ea85b, Model: sd_xl_base_1.0_0.9vae, Version: v1.6.0-RC-32-gde0db85f
Webui
Comfy
Fooocus (all tricks disabled)
Webui (with "use_vanilla_random_number_generator")
Example 2
professional 3d model of a cute cat. octane render, highly detailed, volumetric, dramatic lighting
Negative prompt: ugly, deformed, noisy, low poly, blurry, painting
Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 8, Seed: 123456, Size: 1024x1024, Model hash: e6bb9ea85b, Model: sd_xl_base_1.0_0.9vae, Version: v1.6.0-RC-33-gc8f0b780
Webui
Comfy
Fooocus (all tricks disabled)
Webui (with "use_vanilla_random_number_generator")
Minor
If you may still see some minor difference between these results, my guess is that this is caused by other software using cpu to process clip text encoding. Nevertheless, we do not think it is necessary to perfectly reproduce that.
Checklist: