Unprompted has stopped being deterministic

[entusiastaIA]

Due diligence

• I checked for similar issues and couldn't find any.
• My WebUI and Unprompted are both up-to-date.
• I disabled my other extensions but the problem persists.

Describe the bug

Half of the time the same seeds generate the same images and prompts, however randomly, clicking generate a controlled batch again results in some of the prompts mid processing change, generating different images. Some times it is just some random images within the batch, some times, from a certain point it changes drastically.

I have checked both ticking synchronize seed, and unticking it and setting the initial seed to be the same as the original seed. The problem persits. Example grids after Hires Fix with NSFW removed. They should be the same but they aren't (batch count is 16):

If I keep pressing generate I can keep getting variations. Most of the time the first images are alright, and it tends to happen towards the middle/end of the generation.

Prompt

No response

Log output

No response

Unprompted version

10.2.3

WebUI version

1.6.0

Other comments

P.S: I know images are uncanny, I'm making some very specific tests right now, and actually having them undeterministic is making things much more complex.

[entusiastaIA]

Further observations show that the more variables you change, from checkpoint to using hires fix the more likely it is to happen, or so it seems. When you click generate without HRF active on the same configuration it is much more rare than if using HRF or across different models.

[entusiastaIA]

Update: my template used the shortcode [chance] only in two places over 107 files. Removing them make it much more stable. Yet changing the model or altering other non-related variables makes it tend to change the rng hence still being non-deterministic. Even without changing anything else it still is not deterministic (just hitting generate without changing anything can still output different images, although it is rarer after removing "chance" shortcode). I tried changing Automatic RNG to CPU and GPU, none of them seem to matter at all.

[ThereforeGames]

Hi @entusiastaIA,

Thank you for bringing this to my attention.

So far, I haven't been able to reproduce the problem in my tests - I've tried running a few of my own templates at a batch_count of 10 and the images always come back the same.

I will experiment with [chance] and some other WebUI settings to see if I can figure out what the underlying issue is.

[entusiastaIA]

So far, without using Hires Fix and without changing any model it barely ever happens. I use 16 batch count and I'd say that only one of every 8-10 times one or two images end up having different prompts.
However if I activate Hires fix it starts haoppening more often, and if I also change to use SDXL from using my 1.5 based model (Requiem) it happens almost always: some of the pictures end up with different prompts, while some retain them.

It's a very weird behaivour... I have noticed that most of the times (but not always either) is just one specific part of the prompt (the first "half") the one that stops being deterministic, while the end is still the same that it was before. However for the life of me I don't see anything special or different on my code in those parts... And sometimes it is the whole prompt anyway.

The same about the images themselves... sometimes from a 16 grid only one or two images in the middle are changed but after them the new generated images are again back the same they should be. However some other times once one image is different, all the images after that one are changed as well...

So is a really weird behaivour... these days I'm actually working trying to narrow the issue at the same time that I clean my code (as I admit it was full of workarounds and dirty patches), but so far the only thing that my code has left depending on RNG are "[choose]" and "[choose _weighted]" so I doubt there's much more I can do on my end, but if you think any specific test could be worth it, tell me so.

I think, but this is really hard to proof, that the problem is that at some point some of the [choose] (or other RNG elements) lose, change or have a problem with the seed they should be using, but after them, all the next RNG retains their numbers, that's why only certain parts of the prompts and images get affected while the rest retain their expected prompts. However, since in my template, some "choose" CAN lead to other "choose" but it is not something that always happens; if the seed changes in a way that causes one of this branching elements to be altered (be it cutting previously existing branch or adding a new branch), then all the following rng get screwed because they now happen at different places...

If as I suspect is actually a low chance event for every independent rng decission, then depending on how many of those your templates have you may need more or less images to make it happen. I could try later to build a simple code filled with rng and calls to try force it happening.

Thank you very much for your time, this is a big issue for me, hindering a lot the workflow, because I need deterministic image grids to compare the specific prompt and model changes I'm making, and definetly it wasn't happening at older versions of Unprompted and Automatic.

[entusiastaIA]

I have updated, it is still happening for the latest release yet. I have been experimenting a bit with SDXL Turbo, and changing parameters such as Steps, model, CFG and almost anything breaks determinism. Sometimes it breaks on it's own, but anytime I change parameters it is likely to change prompts from some images.

If you need the template reach me out by the same email, but it shouldn't have anything very special... just many files (181)

[entusiastaIA]

Ok, I've managed to get an undeterministic behaivour on the first branch, this was important because that way I can now it happens pretty soon in the code, so it's unlikely to be because of any quirk of my code. Anyway reaching the first branch will mean sharing quite some lines. Will share them should they help you identifying the problem.

First there's the initializing file for the wizard, I'll skip all the commentary and paste the code part:

[template]...
[/template]

[set stability _new _label="General Parameter" _ui="dropdown" _choices="all|stable"]all[/set]

[set reality _new _label="General Parameter" _ui="dropdown" _choices="all|real|ultra"]all[/set]

[set camera _new _label="General Parameter" _ui="dropdown" _choices="all|direct|front|side|back|above"]all[/set]

[set distance _new _label="General Parameter" _ui="dropdown" _choices="all|close|medium|far"]all[/set]

[set race _new _label="General Parameter" _ui="dropdown" _choices="all|white|asian|black|ethnic|fantasy"]all[/set]

[set age _new _label="General Parameter" _ui="dropdown" _choices="all|safe|beautiful|adult"]all[/set]

[set subject _new _label="General Parameter" _ui="dropdown" _choices="all|female|male|femaleandfame|maleandfame|famefemale|famemale|fakefemale|fakemale"]all[/set]

[set androgyny _new _label="General Parameter" _ui="dropdown" _choices="false|true"]false[/set]

[set haircolor _new _label="General Parameter" _ui="dropdown" _choices="all|ginger|blonde|dark"]all[/set]

[set haircut _new _label="General Parameter" _ui="dropdown" _choices="all|long|short"]all[/set]

[set nsfw _new _label="General Parameter" _ui="dropdown" _choices="all|pure|dress|nude|porn"]all[/set]

[set qualityboolean _new _label="Advanced Variables: Override picture quality logic?" _ui="checkbox"]0[/set]
[set qualitytext _new _label="New quality"]One word or short sentence to describe a first look about a photo. E.g. "expert" or "underexposed".[/set]
[if qualityboolean="1"][sets qualityusedoverride="{get qualitytext}"][/if]

[set cameraboolean _new _label="Advanced Variables: Override camera logic?" _ui="checkbox"]0[/set]
[set cameratext _new _label="New Camera"]Complete camera angle, E.g. "from the back" (Remember to match it with the General parameter)[/set]
[if cameraboolean="1"][sets camerausedoverride="{get cameratext}"][/if]

[set workboolean _new _label="Advanced Variables: Override phototype logic?" _ui="checkbox"]0[/set]
[set worktext _new _label="New work"]One word type recommended (photo, picture, portrait, selfie...).[/set]
[if workboolean="1"][sets workusedoverride="{get worktext}"][/if]

[set lightingboolean _new _label="Advanced Variables: Override lighting logic?" _ui="checkbox"]0[/set]
[set lightingtext _new _label="New lighting"]Short sentence starting by "with" and finished by "lighting" recommended. E.g. "with cold lighting".[/set]
[if lightingboolean="1"][sets lightingusedoverride="{get lightingtext}"][/if]

[set photographersboolean _new _label="Advanced Variables: Override photographers logic?" _ui="checkbox"]0[/set]
[set photographerstext _new _label="New photographers"]Short sentence starting with "captured by" recommended. E.g. "captured by helmut newton".[/set]
[if photographersboolean="1"][sets photographerusedoverride="{get photographerstext}"][/if]

[set raceboolean _new _label="Advanced Variables: Override race logic?" _ui="checkbox"]0[/set]
[set racetext _new _label="New race"]Short nationality recommended. E.g. "south african".[/set]
[if raceboolean="1"][sets raceusedoverride="{get racetext}"][/if]

[set personageboolean _new _label="Advanced Variables: Override age logic?" _ui="checkbox"]0[/set]
[set personagetext _new _label="New age"]One single word recommended (young, old, adult...).[/set]
[if personageboolean="1"][sets ageusedoverride="{get personagetext}"][/if]

[set complexionboolean _new _label="Advanced Variables: Override complexion logic?" _ui="checkbox"]0[/set]
[set complexiontext _new _label="New complexion"]One single word recommended (thick, athletic, slim...).[/set]
[if complexionboolean="1"][sets complexionusedoverride="{get complexiontext}"][/if]

[set subjectboolean _new _label="Advanced Variables: Override subject logic?" _ui="checkbox"]0[/set]
[set subjecttext _new _label="New subject"]One single word recommended (woman, boy, mother...), character names could work too (emma watson).[/set]
[if subjectboolean="1"][sets subjectusedoverride="{get subjecttext}"][/if]

[set haircolorboolean _new _label="Advanced Variables: Override haircolor logic?" _ui="checkbox"]0[/set]
[set haircolortext _new _label="New haircolor"]One word finished by "-haired" recommended. E.g. "ginger-haired"[/set]
[if haircolorboolean="1"][sets haircolorusedoverride="{get haircolortext}"][/if]

[set haircutboolean _new _label="Advanced Variables: Override haircut logic?" _ui="checkbox"]0[/set]
[set haircuttext _new _label="New haircut"]Short sentence and finished by "haircut" or "hairstyle" recommended. E.g. "curly haircut"[/set]
[if haircutboolean="1"][sets haircutusedoverride="{get haircuttext}"][/if]

[set faceboolean _new _label="Advanced Variables: Override face logic?" _ui="checkbox"]0[/set]
[set facetext _new _label="New face"]Short sentence fitting after the word "having" about the face recommended. E.g. "a happy smile" or "a chiseled visage"[/set]
[if faceboolean="1"][sets faceusedoverride="{get facetext}"][/if]

[set appearanceboolean _new _label="Advanced Variables: Override body or clothes logic?" _ui="checkbox"]0[/set]
[set appearancetext _new _label="New body"]Describe naturally but short. E.g. "naked with a muscled body" or "wearing a casual chiffon blouse with detailed fabric".[/set]
[if appearanceboolean="1"][sets appearanceusedoverride="{get appearancetext}"][/if]

[set attractiveboolean _new _label="Advanced Variables: Override attractive logic?" _ui="checkbox"]0[/set]
[set attractivetext _new _label="New attractive"]Overrieden by appearance - One single word recommended E.g. "desirable" or  "dirty".[/set]
[if attractiveboolean="1"][sets attractiveusedoverride="{get attractivetext}"][/if]

[set bodyshapeboolean _new _label="Advanced Variables: Override bodyshape logic?" _ui="checkbox"]0[/set]
[set bodyshapetext _new _label="New bodyshape"]Overrieden by appearance - Few words fitting after word "with". E.g. "wide hips" or "big breasts".[/set]
[if bodyshapeboolean="1"][sets bodyshapeusedoverride="{get bodyshapetext}"][/if]

[set fleshboolean _new _label="Advanced Variables: Override flesh logic?" _ui="checkbox"]0[/set]
[set fleshtext _new _label="New flesh"]Overrieden by appearance - Few words finished in "skin" or "flesh". E.g. "tattoed flesh" or "wet skin".[/set]
[if fleshboolean="1"][sets fleshusedoverride="{get fleshtext}"][/if]

[set extraboolean _new _label="Advanced Variables: Override extra logic?" _ui="checkbox"]0[/set]
[set extratext _new _label="New extra"]Overrieden by appearance - Short sentence to add final body details. E.g. "having a natural hairy pubes".[/set]
[if extraboolean="1"][sets extrausedoverride="{get extratext}"][/if]

[set poseboolean _new _label="Advanced Variables: Override pose logic?" _ui="checkbox"]0[/set]
[set posetext _new _label="New pose"]Describe action and emotion naturally but short. E.g: "standing with a relaxed pose" or "playing with butterflies".[/set]
[if poseboolean="1"][sets poseusedoverride="{get posetext}"][/if]

[set backgroundsboolean _new _label="Advanced Variables: Override backgrounds logic?" _ui="checkbox"]0[/set]
[set backgroundstext _new _label="New backgrounds"]Describe natural and effective but not verbose. E.g. "amidst a cyberpunk street food market with neon signs at night".[/set]
[if backgroundsboolean="1"][sets backgroundusedoverride="{get backgroundstext}"][/if]

[set colorboolean _new _label="Advanced Variables: Override color logic?" _ui="checkbox"]0[/set]
[set colortext _new _label="New color"]The color you enter here will override random material color selection, if any (not very common)[/set]
[if colorboolean="1"][sets colorusedoverride="{get colortext}"][/if]

[set materialboolean _new _label="Advanced Variables: Override material logic?" _ui="checkbox"]0[/set]
[set materialtext _new _label="New material"]The material you enter here will override random material substance selection, if any (rare)[/set]
[if materialboolean="1"][sets materialusedoverride="{get materialtext}"][/if]

[set negative_promptboolean _new _label="Advanced Variables: Override negative_prompt logic?" _ui="checkbox"]0[/set]
[set negative_prompttext _new _label="New negative_prompt"]Only use very few words, well calculated negatives, looking like a complete sentence. E.g. "a grotesque render.".[/set]
[if negative_promptboolean="1"][sets negative_promptusedoverride="{get negative_prompttext}"][/if]

[call masterprompt-sdxl_turbo/zmainlogic]

Now let's go to that "zmainlogic" file:

[if nsfw="porn"][set porn]true[/set][set nsfw]nude[/set][/if][else][set porn]false[/set][/else]
[if nsfw="pure"][set pure]true[/set][set nsfw]dress[/set][/if][else][set pure]false[/set][/else]
[if age="safe"][set safe]true[/set][set age]adult[/set][/if][else][set safe]false[/set][/else]

[set subjectused][call masterprompt-sdxl_turbo/subject/zsubjectlogic][/set]
[if subjectboolean="1"][set subjectused][get subjectusedoverride][/set][/if]
...

The code keeps going but I got it to diverge on zsubjectlogic for a same seed, 16 batch count, it diverged there on the 9th image. The only non-default variable I used was subject -> "female". So what's inside zsubjectlogic:

[set xsubjectfemalenormal]true[/set]
[set xsubjectfemaleyoung]true[/set]
[set xsubjectfemaleold]true[/set]
[set xsubjectmalenormal]true[/set]
[set xsubjectmaleyoung]true[/set]
[set xsubjectmaleold]true[/set]
[set xsubjectfamefemale]true[/set]
[set xsubjectfamemale]true[/set]
[set xsubjectfakefemale]true[/set]
[set xsubjectfakemale]true[/set]

[if age="adult"][sets xsubjectfemaleyoung=false xsubjectmaleyoung=false][/if]
[if age="bear"][sets xsubjectfemalenormal=false xsubjectfemaleold=false xsubjectmalenormal=false xsubjectmaleold=false][/if]
[if age="young"][sets xsubjectfemaleold=false xsubjectmaleold=false][/if]
[if age="beautiful"][sets xsubjectfemaleold=false xsubjectmaleold=false][/if]
[if subject="female"][sets xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamefemale=false xsubjectfamemale=false xsubjectfakefemale=false xsubjectfakemale=false][/if]
[if subject="femaleandfame"][sets xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamemale=false xsubjectfakemale=false][/if]
[if subject="male"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectfamefemale=false xsubjectfamemale=false xsubjectfakefemale=false][/if]
[if subject="maleandfame"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectfamefemale=false xsubjectfakefemale=false][/if]
[if subject="famefemale"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamemale=false xsubjectfakemale=false xsubjectfakefemale=false][/if]
[if subject="famemale"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamefemale=false xsubjectfakefemale=false xsubjectfakemale=false][/if]
[if subject="fakefemale"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamefemale=false xsubjectfamemale=false xsubjectfakemale=false][/if]
[if subject="fakemale"][sets xsubjectfemaleold=false xsubjectfemalenormal=false xsubjectfemaleyoung=false xsubjectmaleyoung=false xsubjectmalenormal=false xsubjectmaleold=false xsubjectfamefemale=false xsubjectfamemale=false xsubjectfakefemale=false][/if]

[if xsubjectfemalenormal="true"][array arrsubject _append="16|subjectfemalenormal"][/if]
[if xsubjectfemaleyoung="true"][array arrsubject _append="8|subjectfemaleyoung"][/if]
[if xsubjectfemaleold="true"][array arrsubject _append="2|subjectfemaleold"][/if]
[if xsubjectfamefemale="true"][array arrsubject _append="1|subjectfamefemale"][/if]
[if xsubjectfakefemale="true"][array arrsubject _append="4|subjectfakefemale"][/if]

[if xsubjectmalenormal="true"][array arrsubject _append="8|subjectmalenormal"][/if]
[if xsubjectmaleyoung="true"][array arrsubject _append="4|subjectmaleyoung"][/if]
[if xsubjectmaleold="true"][array arrsubject _append="1|subjectmaleold"][/if]
[if xsubjectfamemale="true"][array arrsubject _append="1|subjectfamemale"][/if]
[if xsubjectfakemale="true"][array arrsubject _append="2|subjectfakemale"][/if]

[set subjectlocal][choose _weighted][get _sep="|" arrsubject][/choose][/set]

[if subjectlocal="subjectfemalenormal"][set elderused]adult[/set][set genderused]female[/set][call masterprompt-sdxl_turbo/subject/xsubjectfemalenormal][/if]
[if subjectlocal="subjectfemaleyoung"][set elderused]young[/set][set genderused]female[/set][call masterprompt-sdxl_turbo/subject/xsubjectfemaleyoung][/if]
[if subjectlocal="subjectfemaleold"][set elderused]old[/set][set genderused]female[/set][call masterprompt-sdxl_turbo/subject/xsubjectfemaleold][/if]
[if subjectlocal="subjectfamefemale"][set elderused]adult[/set][set genderused]female[/set][call masterprompt-sdxl_turbo/subject/xsubjectfamefemale][/if]
[if subjectlocal="subjectfakefemale"][set elderused]adult[/set][set genderused]female[/set][call masterprompt-sdxl_turbo/subject/xsubjectfakefemale][/if]

[if subjectlocal="subjectmalenormal"][set elderused]adult[/set][set genderused]male[/set][call masterprompt-sdxl_turbo/subject/xsubjectmalenormal][/if]
[if subjectlocal="subjectmaleyoung"][set elderused]young[/set][set genderused]male[/set][call masterprompt-sdxl_turbo/subject/xsubjectmaleyoung][/if]
[if subjectlocal="subjectmaleold"][set elderused]old[/set][set genderused]male[/set][call masterprompt-sdxl_turbo/subject/xsubjectmaleold][/if]
[if subjectlocal="subjectfamemale"][set elderused]adult[/set][set genderused]male[/set][call masterprompt-sdxl_turbo/subject/xsubjectfamemale][/if]
[if subjectlocal="subjectfakemale"][set elderused]adult[/set][set genderused]male[/set][call masterprompt-sdxl_turbo/subject/xsubjectfakemale][/if]

All the final calls there are just files with [choose]listofwords[/choose] inside. And when making the image with 3 steps it generated "lady" which belongs to the "xsubjectfemalenormal" but when using 4 steps it generated "girl" which belongs to the "xsubjectfemaleyoung". From there on, of course, many other different choices happened since the RNG had been changed and it beared consecuences across the whole template, for example due to the modified "elderused" variable. This means that this very first branching line happened different from one run to the other:

[set subjectlocal][choose _weighted][get _sep="|" arrsubject][/choose][/set]

Please, let me know if I can help any way better to tailor this issue, is becoming a killer problem as I cannot make solid tests if the prompts are changing constantly.

[entusiastaIA]

Ok, I don't know much of what I'm doing but if within the choose.py file I add this:

All the images get now always the same prompts... which makes me think that nor choose or chance are the problem, but something breaks from time to time synchronizing the rng seeds of unprompted...

Now I'm trying to figure how to import unprompted_seed or p.seed variables from the unprompted.py to the choose.py file to see what happens if I read them directly on that file... or importing seed directly from web-ui file if I knew where and how to grab it... Also I'm reading with the help of ChatGPT the code from unprompted.py trying to understand what's happening with the seeds... But I'm no programmer... so I'll enjoy a helping hand, I'm almost just but a monkey pressing buttons right now...

[entusiastaIA]

I "did" it. Kind of. Kindish of... Adding this in the choose.py makes the prompts random but deterministic again:

I don't know well what I'm doing. I've been chasing many many many theories so far. I don't know if it's unprompted or A1111 fault. Countless ChatGPT computing help has been spent to be able to do that little change.

It introduces 2 new problems:

1st) For a given seed there is always a same prompt if the template isn't changed. That mean that it doesn't matter if you arrive at seed 23 because you started at seed 19 with a batch count of 5, or because you started at seed 10 with a batch count of 14. Seed 23 will always have the same prompt for both given cases. This is half a problem half a new feature, I cannot decide yet. But definetively is different from the usual behaivour.

2nd) Randomness seem to be a bit... simplistic? It means that some images from close seeds end up with pretty similar prompts where almost everything is the same except a few things, but not always, depending on seeds there are high or low variation.

Anyway both problems are orders of magnitude lesser than the one we had. They belong to different universes at all... now I can work with unprompted again, so until you or somebody more experienced makes a proper fix, that floats my boat...

EDIT: I've done some tests and nothing I change from the random.seed in the unprompted.py or other mentions to random in that file seem to modify the determinism leak... so I'd say whatever is going on is not on that file? I really cannot understand anymore what's happening...

Ok, back to working on prompting again. This thing has melted my brain ...

[entusiastaIA]

Ok... both problems fixed... now this seems pretty reliable... It's dirty. But reliable:

For whatever reason it doesn't seem to work if attempted at the unprompted.py level. So probably it will have to be implemented in other random based files such as chance... so far, I'm happy with this.