How to use single-file custom UNET with Flux? #3487

SAC020 · 2024-10-13T14:32:21Z

SAC020
Oct 13, 2024

Putting this up as a question and not an issue because maybe I don't understand the operating instructions.

I am trying to use a single-file custom UNET with Flux, and it does not appear to "kick in" (do something).

I have tried all unets from here: https://civitai.com/models/843551?modelVersionId=943939 (4 "flavors"), put them in the \models\unet folder, tried them with Flux qint4 and nf4, and I am getting identical images every time

For example

PS C:\ai\automatic> .\webui.bat --debug
Using VENV: C:\ai\automatic\venv
17:22:47-442851 INFO     Starting SD.Next
17:22:47-446809 INFO     Logger: file="C:\ai\automatic\sdnext.log" level=DEBUG size=65 mode=create
17:22:47-448580 INFO     Python: version=3.11.9 platform=Windows bin="C:\ai\automatic\venv\Scripts\python.exe"
                         venv="C:\ai\automatic\venv"
17:22:47-715755 INFO     Version: app=sd.next updated=2024-10-13 hash=aeddd6ce branch=dev
                         url=https://github.com/vladmandic/automatic/tree/dev ui=dev
17:22:48-667184 INFO     Repository latest available bb6a40da131269554f026379ff97c5480b40e64e 2024-10-13T01:26:43Z
17:22:48-681559 INFO     Platform: arch=AMD64 cpu=Intel64 Family 6 Model 165 Stepping 5, GenuineIntel system=Windows
                         release=Windows-10-10.0.22631-SP0 python=3.11.9
17:22:48-683581 DEBUG    Setting environment tuning
17:22:48-684552 DEBUG    Torch allocator: "garbage_collection_threshold:0.80,max_split_size_mb:512"
17:22:48-696519 DEBUG    Torch overrides: cuda=False rocm=False ipex=False diml=False openvino=False zluda=False
17:22:48-709922 INFO     CUDA: nVidia toolkit detected
17:22:48-926484 WARNING  Modified files: ['models/Reference/playgroundai--playground-v2-1024px-aesthetic.jpg']
17:22:49-026217 INFO     Verifying requirements
17:22:49-030207 INFO     Verifying packages
17:22:49-078079 DEBUG    Timestamp repository update time: Sun Oct 13 16:52:27 2024
17:22:49-079077 INFO     Startup: standard
17:22:49-080075 INFO     Verifying submodules
17:22:52-614838 DEBUG    Git detached head detected: folder="extensions-builtin/sd-extension-chainner" reattach=main
17:22:52-616833 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner / main
17:22:52-747583 DEBUG    Git detached head detected: folder="extensions-builtin/sd-extension-system-info" reattach=main
17:22:52-749578 DEBUG    Git submodule: extensions-builtin/sd-extension-system-info / main
17:22:52-910879 DEBUG    Git detached head detected: folder="extensions-builtin/sd-webui-agent-scheduler" reattach=main
17:22:52-911876 DEBUG    Git submodule: extensions-builtin/sd-webui-agent-scheduler / main
17:22:53-091418 DEBUG    Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=dev
17:22:53-092416 DEBUG    Git submodule: extensions-builtin/sdnext-modernui / dev
17:22:53-261723 DEBUG    Git detached head detected: folder="extensions-builtin/stable-diffusion-webui-rembg"
                         reattach=master
17:22:53-263718 DEBUG    Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master
17:22:53-398832 DEBUG    Git detached head detected: folder="modules/k-diffusion" reattach=master
17:22:53-399829 DEBUG    Git submodule: modules/k-diffusion / master
17:22:53-526466 DEBUG    Git detached head detected: folder="wiki" reattach=master
17:22:53-527491 DEBUG    Git submodule: wiki / master
17:22:53-699374 DEBUG    Register paths
17:22:53-794600 DEBUG    Installed packages: 222
17:22:53-796716 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sdnext-modernui',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg']
17:22:53-984214 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-extension-system-info\install.py
17:22:54-182913 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-webui-agent-scheduler\install.py
17:22:57-102163 DEBUG    Extension force: name="sd-webui-controlnet" commit=ecd33eb
17:22:57-218850 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\sd-webui-controlnet\install.py
17:23:11-523640 DEBUG    Extension installer:
                         C:\ai\automatic\extensions-builtin\stable-diffusion-webui-images-browser\install.py
17:23:14-206924 DEBUG    Extension installer: C:\ai\automatic\extensions-builtin\stable-diffusion-webui-rembg\install.py
17:23:23-467614 DEBUG    Extensions all: []
17:23:23-468599 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',
                         'sd-webui-agent-scheduler', 'sd-webui-controlnet', 'sdnext-modernui',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg']
17:23:23-469582 INFO     Verifying requirements
17:23:23-470579 DEBUG    Setup complete without errors: 1728829403
17:23:23-496603 DEBUG    Extension preload: {'extensions-builtin': 0.01, 'extensions': 0.0}
17:23:23-497920 DEBUG    Starting module: <module 'webui' from 'C:\\ai\\automatic\\webui.py'>
17:23:23-498950 INFO     Command line args: ['--debug'] debug=True
17:23:23-499945 DEBUG    Env flags: []
17:23:35-225722 INFO     System packages: {'torch': '2.4.1+cu124', 'diffusers': '0.31.0.dev0', 'gradio': '3.43.2',
                         'transformers': '4.45.2', 'accelerate': '1.0.0'}
17:23:35-969169 DEBUG    Huggingface cache: folder="C:\Users\sebas\.cache\huggingface\hub"
17:23:36-084650 INFO     Device detect: memory=24.0 optimization=none
17:23:36-086646 DEBUG    Read: file="config.json" json=35 bytes=1610 time=0.000
17:23:36-087643 DEBUG    Unknown settings: ['cross_attention_options', 'face_restoration_unload']
17:23:36-089638 INFO     Engine: backend=Backend.DIFFUSERS compute=cuda device=cuda attention="Scaled-Dot-Product"
                         mode=no_grad
17:23:36-091636 DEBUG    Read: file="html\reference.json" json=52 bytes=29118 time=0.001
17:23:36-260667 INFO     Torch parameters: backend=cuda device=cuda config=BF16 dtype=torch.bfloat16 vae=torch.bfloat16
                         unet=torch.bfloat16 context=no_grad nohalf=False nohalfvae=False upscast=False
                         deterministic=False test-fp16=True test-bf16=True optimization="Scaled-Dot-Product"
17:23:36-976674 DEBUG    ONNX: version=1.19.2 provider=CUDAExecutionProvider, available=['TensorrtExecutionProvider',
                         'CUDAExecutionProvider', 'CPUExecutionProvider']
17:23:37-212055 INFO     Device: device=NVIDIA GeForce RTX 4090 n=1 arch=sm_90 capability=(8, 9) cuda=12.4 cudnn=90100
                         driver=561.09
17:23:37-321022 DEBUG    Importing LDM
17:23:37-343227 DEBUG    Entering start sequence
17:23:37-348708 DEBUG    Initializing
17:23:37-402123 INFO     Available VAEs: path="models\VAE" items=0
17:23:37-404120 INFO     Available UNets: path="models\UNET" items=3
17:23:37-405117 INFO     Available TEs: path="models\Text-encoder" items=4
17:23:37-407111 INFO     Disabled extensions: ['sd-webui-controlnet', 'sdnext-modernui']
17:23:37-415946 DEBUG    Read: file="cache.json" json=2 bytes=11134 time=0.001
17:23:37-428855 DEBUG    Read: file="metadata.json" json=717 bytes=2542270 time=0.012
17:23:37-481899 DEBUG    Scanning diffusers cache: folder="models\Diffusers" items=3 time=0.01
17:23:37-482896 INFO     Available Models: path="models\Stable-diffusion" items=15 time=0.07
17:23:37-587148 INFO     Available Yolo: path="models\yolo items=6 downloaded=3
17:23:37-589365 DEBUG    Load extensions
17:23:37-722888 INFO     Extension: script='extensions-builtin\Lora\scripts\lora_script.py'
                         [2;36m17:23:37-717587[0m[2;36m [0m[34mINFO    [0m Available LoRAs: [33mitems[0m=[1;36m152[0m
                         [33mfolders[0m=[1;36m6[0m
17:23:38-308749 INFO     Extension: script='extensions-builtin\sd-webui-agent-scheduler\scripts\task_scheduler.py' Using
                         sqlite file: extensions-builtin\sd-webui-agent-scheduler\task_scheduler.sqlite3
17:23:38-565454 DEBUG    Extensions init time: 0.98 Lora=0.06 sd-extension-chainner=0.06 sd-webui-agent-scheduler=0.51
                         stable-diffusion-webui-images-browser=0.25
17:23:38-586993 DEBUG    Read: file="html/upscalers.json" json=4 bytes=2672 time=0.001
17:23:38-589070 DEBUG    Read: file="extensions-builtin\sd-extension-chainner\models.json" json=24 bytes=2719 time=0.001
17:23:38-594615 DEBUG    chaiNNer models: path="models\chaiNNer" defined=24 discovered=0 downloaded=8
17:23:38-596872 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="1x-ITF-SkinDiffDetail-Lite-v1"
                         path="models\ESRGAN\1x-ITF-SkinDiffDetail-Lite-v1.pth"
17:23:38-598048 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="4xNMKDSuperscale_4xNMKDSuperscale"
                         path="models\ESRGAN\4xNMKDSuperscale_4xNMKDSuperscale.pth"
17:23:38-599349 DEBUG    Upscaler type=ESRGAN folder="models\ESRGAN" model="4x_NMKD-Siax_200k"
                         path="models\ESRGAN\4x_NMKD-Siax_200k.pth"
17:23:38-602342 INFO     Available Upscalers: items=56 downloaded=11 user=3 time=0.03 types=['None', 'Lanczos',
                         'Nearest', 'ChaiNNer', 'AuraSR', 'ESRGAN', 'LDSR', 'RealESRGAN', 'SCUNet', 'SD', 'SwinIR']
17:23:38-621714 INFO     Available Styles: folder="models\styles" items=288 time=0.02
17:23:38-628348 DEBUG    Creating UI
17:23:38-629348 DEBUG    UI themes available: type=Standard themes=12
17:23:38-630344 INFO     UI theme: type=Standard name="black-teal"
17:23:38-641296 DEBUG    UI theme: css="C:\ai\automatic\javascript\black-teal.css" base="sdnext.css" user="None"
17:23:38-644288 DEBUG    UI initialize: txt2img
17:23:38-725195 DEBUG    Networks: page='model' items=66 subfolders=2 tab=txt2img folders=['models\\Stable-diffusion',
                         'models\\Diffusers', 'models\\Reference'] list=0.06 thumb=0.01 desc=0.01 info=0.00 workers=8
                         sort=Default
17:23:38-730182 DEBUG    Networks: page='lora' items=152 subfolders=0 tab=txt2img folders=['models\\Lora',
                         'models\\LyCORIS'] list=0.06 thumb=0.02 desc=0.06 info=0.03 workers=8 sort=Default
17:23:38-739157 DEBUG    Networks: page='style' items=288 subfolders=1 tab=txt2img folders=['models\\styles', 'html']
                         list=0.06 thumb=0.00 desc=0.00 info=0.00 workers=8 sort=Default
17:23:38-743147 DEBUG    Networks: page='embedding' items=13 subfolders=0 tab=txt2img folders=['models\\embeddings']
                         list=0.05 thumb=0.02 desc=0.01 info=0.00 workers=8 sort=Default
17:23:38-745189 DEBUG    Networks: page='vae' items=0 subfolders=0 tab=txt2img folders=['models\\VAE'] list=0.00
                         thumb=0.00 desc=0.00 info=0.00 workers=8 sort=Default
17:23:38-747184 DEBUG    Networks: page='history' items=0 subfolders=0 tab=txt2img folders=[] list=0.00 thumb=0.00
                         desc=0.00 info=0.00 workers=8 sort=Default
17:23:38-852060 DEBUG    UI initialize: img2img
17:23:39-120973 DEBUG    UI initialize: control models=models\control
17:23:39-460741 DEBUG    Read: file="ui-config.json" json=0 bytes=2 time=0.001
17:23:39-574438 DEBUG    UI themes available: type=Standard themes=12
17:23:40-196146 DEBUG    Reading failed: C:\ai\automatic\html\extensions.json [Errno 2] No such file or directory:
                         'C:\\ai\\automatic\\html\\extensions.json'
17:23:40-199124 INFO     Extension list is empty: refresh required
17:23:40-859528 DEBUG    Extension list: processed=8 installed=8 enabled=6 disabled=2 visible=8 hidden=0
17:23:41-217224 DEBUG    Root paths: ['C:\\ai\\automatic']
17:23:41-354539 INFO     Local URL: http://127.0.0.1:7860/
17:23:41-356534 DEBUG    Gradio functions: registered=2500
17:23:41-358529 DEBUG    FastAPI middleware: ['Middleware', 'Middleware']
17:23:41-361520 DEBUG    Creating API
17:23:41-549021 INFO     [AgentScheduler] Task queue is empty
17:23:41-550467 INFO     [AgentScheduler] Registering APIs
17:23:41-871610 DEBUG    Scripts setup: ['IP Adapters:0.171', 'XYZ Grid:0.026', 'Face:0.013', 'AnimateDiff:0.006',
                         'CogVideoX:0.009', 'Ctrl-X:0.006', 'LUT Color grading:0.006', 'Image-to-Video:0.006', 'Stable
                         Video Diffusion:0.006']
17:23:41-927489 DEBUG    Save: file="metadata.json" json=721 bytes=2490978 time=0.055
17:23:41-929401 INFO     Model metadata saved: file="metadata.json" items=4 time=0.03
17:23:41-931722 DEBUG    Model requested: fn=run:<lambda>
17:23:41-932918 INFO     Load model: select="Diffusers\Disty0/FLUX.1-dev-qint4 [82811df42b]"
17:23:41-933918 DEBUG    Load model:
                         target="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375
                         d5170c306a6eb" existing=False info=None
17:23:41-935911 DEBUG    Load model:
                         path="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb"
17:23:41-936909 INFO     Autodetect model: detect="FLUX" class=FluxPipeline
                         file="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb" size=0MB
17:23:41-948843 DEBUG    Load model: type=FLUX model="Diffusers\Disty0/FLUX.1-dev-qint4" repo="Disty0/FLUX.1-dev-qint4"
                         unet="fluxDEVDEDISTILLED_q4KM.gguf" t5="None" vae="None" quant=qint4 offload=none
                         dtype=torch.bfloat16
17:23:41-949843 DEBUG    HF login: no token provided
17:23:41-951838 INFO     Load module: type=UNet/Transformer
                         file="C:\ai\automatic\models\UNET\fluxDEVDEDISTILLED_q4KM.gguf" offload=none quant=gguf
                         dtype=torch.bfloat16
17:24:07-952381 DEBUG    Load model: type=Unet/Transformer applied=1156 skipped=0 stats={'F32': 468, 'Q4_K': 266,
                         'Q5_K': 38, 'F16': 4}
17:24:07-953378 DEBUG    Quantization: module=quanto fn=load_flux:load_flux_quanto
17:24:35-534621 DEBUG    Model requested: fn=f:txt2img
17:24:35-536615 WARNING  Model not loaded
17:25:19-901164 DEBUG    Load model: type=FLUX preloaded=['transformer', 'text_encoder_2']
Diffusers 10.20it/s ████████ 100% 7/7 00:00 00:00 Loading pipeline components...
17:25:20-931911 INFO     Load network: type=embeddings loaded=0 skipped=13 time=0.03
17:25:20-932908 DEBUG    Setting model: component=VAE slicing=True
17:25:20-933906 DEBUG    Setting model: attention="Scaled-Dot-Product"
17:25:20-950860 DEBUG    Setting model: offload=none
17:25:27-498661 DEBUG    GC: utilization={'gpu': 58, 'ram': 3, 'threshold': 80} gc={'collected': 430, 'saved': 3.21}
                         before={'gpu': 13.84, 'ram': 2.14} after={'gpu': 10.63, 'ram': 2.14, 'retries': 0, 'oom': 0}
                         device=cuda fn=reload_model_weights:load_diffuser time=0.29
17:25:27-505643 INFO     Load model: time=105.28 load=98.97 move=6.26 native=1024 memory={'ram': {'used': 2.14, 'total':
                         63.92}, 'gpu': {'used': 10.63, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:25:27-508635 DEBUG    Script callback init time: image_browser.py:ui_tabs=0.44 system-info.py:app_started=0.08
                         task_scheduler.py:app_started=0.34
17:25:27-509633 INFO     Startup time: 124.00 torch=8.50 gradio=2.01 diffusers=0.92 libraries=2.39 samplers=0.06
                         extensions=0.98 models=0.08 detailer=0.11 ui-networks=0.24 ui-txt2img=0.08 ui-img2img=0.23
                         ui-control=0.14 ui-extras=0.05 ui-settings=0.27 ui-extensions=1.16 ui-defaults=0.29 launch=0.20
                         api=0.09 app-started=0.42 checkpoint=105.64
17:25:27-511627 DEBUG    Save: file="config.json" json=35 bytes=1558 time=0.004
17:25:27-514619 DEBUG    Unused settings: ['cross_attention_options', 'face_restoration_unload']
17:25:34-032338 INFO     XYZ grid start: images=2 grid=1 shape=2x1 cells=1 steps=120
17:25:34-040316 INFO     Base: class=FluxPipeline
17:25:34-042311 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:25:34-076221 DEBUG    Torch generator: device=cuda seeds=[1453647502]
17:25:34-077218 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.05s/it █▏                                   3% 2/60 00:02 01:01 Base17:25:37-818255 DEBUG    VAE load: type=taesd model=models\TAESD\taef1_decoder.pth
Progress  1.06s/it ████████████                       37% 22/60 00:24 00:40 Base17:25:59-537339 DEBUG    Server: alive=True jobs=1 requests=72 uptime=144 memory=1.96/63.92 backend=Backend.DIFFUSERS
                         state=idle
Progress  1.08s/it █████████████████████████████████ 100% 60/60 01:05 00:00 Base
17:26:42-187040 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.859
17:26:42-241893 INFO     Processed: images=1 its=0.88 time=68.19 timers={'args': 0.03, 'callback': 7.15, 'pipeline':
                         65.93, 'decode': 2.19, 'post': 0.04} memory={'ram': {'used': 2.03, 'total': 63.92}, 'gpu':
                         {'used': 11.19, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:26:42-245882 INFO     Base: class=FluxPipeline
17:26:42-246881 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:26:42-249873 DEBUG    Torch generator: device=cuda seeds=[1453647502]
17:26:42-250870 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.03s/it █████████████████████████████████ 100% 60/60 01:01 00:00 Base
17:27:44-210912 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=0.019
17:27:44-398858 INFO     Processed: images=1 its=0.97 time=62.15 timers={'callback': 8.0, 'pipeline': 61.88, 'decode':
                         0.23, 'post': 0.04} memory={'ram': {'used': 2.04, 'total': 63.92}, 'gpu': {'used': 15.68,
                         'total': 23.99}, 'retries': 0, 'oom': 0}
17:27:44-450839 INFO     XYZ grid complete: images=2 size=(2048, 1298) time=130.37 save=0.05
17:27:44-471929 INFO     Processed: images=3 its=17171.39 time=0.01 timers={'init': 0.05, 'callback': 8.0, 'pipeline':
                         61.88, 'decode': 0.23, 'post': 0.06} memory={'ram': {'used': 2.05, 'total': 63.92}, 'gpu':
                         {'used': 15.68, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:27:44-968020 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmpbimz_e6p.png" width=2048
                         height=1298 size=1267662
17:27:45-385432 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmpdchs2kko.png" width=1024
                         height=1024 size=1182982
17:27:45-805148 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmpmjsyobzb.png" width=1024
                         height=1024 size=1182982

vladmandic · 2024-10-13T14:39:41Z

vladmandic
Oct 13, 2024
Maintainer

reverse the order - NONE cannot be after you've loaded something that already overwrote it.
same applies to any model component - vae, unet, text-encoder, etc.

1 reply

SAC020 Oct 13, 2024
Author

Ok, makes sense, however it still doesn't work.

Restarted the server with "none" in the unet quicksetting, started xyz, identical images with none + 3 different unets

I don't understand why it complains about no HF token, everything is downloaded locally.


18:01:25-198978 INFO     XYZ grid start: images=4 grid=1 shape=4x1 cells=1 steps=240
18:01:25-201971 INFO     Base: class=FluxPipeline
18:01:25-203966 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
18:01:25-205960 DEBUG    Torch generator: device=cuda seeds=[3827383365]
18:01:25-206958 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.02s/it ██████████████████▏                55% 33/60 00:33 00:27 Base18:01:59-676304 DEBUG    Server: alive=True jobs=1 requests=101 uptime=176 memory=1.98/63.92 backend=Backend.DIFFUSERS
                         state=idle
Progress  1.03s/it █████████████████████████████████ 100% 60/60 01:01 00:00 Base
18:02:29-032859 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.83
18:02:29-075745 INFO     Processed: images=1 its=0.94 time=63.87 timers={'callback': 7.8, 'pipeline': 61.93, 'decode':
                         1.9, 'post': 0.03} memory={'ram': {'used': 2.05, 'total': 63.92}, 'gpu': {'used': 11.43,
                         'total': 23.99}, 'retries': 0, 'oom': 0}
18:02:29-078737 DEBUG    XYZ grid apply unet: "consolidated_s6700_fp8"
18:02:29-079734 DEBUG    Override: {'sd_unet': 'consolidated_s6700_fp8'}
18:02:29-082725 INFO     Base: class=FluxPipeline
18:02:29-083723 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
18:02:29-085717 DEBUG    Torch generator: device=cuda seeds=[3827383365]
18:02:29-086715 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.02s/it █████████████████████████████████ 100% 60/60 01:01 00:00 Base
18:03:30-802637 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=0.02
18:03:30-984151 INFO     Processed: images=1 its=0.97 time=61.89 timers={'callback': 7.85, 'pipeline': 61.63, 'decode':
                         0.23, 'post': 0.04} memory={'ram': {'used': 2.05, 'total': 63.92}, 'gpu': {'used': 15.67,
                         'total': 23.99}, 'retries': 0, 'oom': 0}
18:03:30-987144 INFO     Load model: select="Diffusers\Disty0/FLUX.1-dev-qint4 [82811df42b]"
18:03:30-988141 DEBUG    Load model:
                         path="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb"
18:03:30-989138 INFO     Autodetect model: detect="FLUX" class=FluxPipeline
                         file="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb" size=0MB
18:03:30-993129 DEBUG    Load model: type=FLUX model="Diffusers\Disty0/FLUX.1-dev-qint4" repo="Disty0/FLUX.1-dev-qint4"
                         unet="consolidated_s6700_fp8" t5="ViT-L-14-BEST-smooth-GmP-TE-only-HF-format" vae="None"
                         quant=qint4 offload=none dtype=torch.bfloat16
18:03:30-994126 DEBUG    HF login: no token provided
18:03:30-996120 INFO     Load module: type=UNet/Transformer
                         file="C:\ai\automatic\models\UNET\consolidated_s6700_fp8.safetensors" offload=none quant=fp8
                         dtype=torch.bfloat16
18:03:30-997117 DEBUG    Quantization: module=bitsandbytes fn=load_transformer:load_flux_bnb
18:03:59-773690 DEBUG    Server: alive=True jobs=1 requests=340 uptime=296 memory=27.02/63.92 backend=Backend.DIFFUSERS
                         state=idle
18:04:12-663481 DEBUG    HF login: no token provided
18:05:59-829548 DEBUG    Server: alive=True jobs=1 requests=589 uptime=416 memory=11.9/63.92 backend=Backend.DIFFUSERS
                         state=idle
18:06:05-475018 DEBUG    Load model: type=FLUX preloaded=['transformer', 'text_encoder_2']
Diffusers  2.24it/s ████████ 100% 7/7 00:03 00:00 Loading pipeline components...
18:06:09-012075 INFO     Load network: type=embeddings loaded=0 skipped=13 time=0.06
18:06:09-013073 DEBUG    Setting model: component=VAE slicing=True
18:06:09-014070 DEBUG    Setting model: attention="Scaled-Dot-Product"
18:06:09-032022 DEBUG    Setting model: offload=none
18:06:16-882497 DEBUG    GC: utilization={'gpu': 86, 'ram': 4, 'threshold': 80} gc={'collected': 783, 'saved': 0.0}
                         before={'gpu': 20.56, 'ram': 2.31} after={'gpu': 20.56, 'ram': 2.31, 'retries': 0, 'oom': 0}
                         device=cuda fn=load_diffuser:move_model time=0.27
18:06:17-159271 DEBUG    GC: utilization={'gpu': 86, 'ram': 4, 'threshold': 80} gc={'collected': 127, 'saved': 0.01}
                         before={'gpu': 20.57, 'ram': 2.31} after={'gpu': 20.56, 'ram': 2.31, 'retries': 0, 'oom': 0}
                         device=cuda fn=load_unet:load_diffuser time=0.27
18:06:17-166253 INFO     Load model: time=165.90 load=157.97 embeddings=0.06 move=7.85 native=1024 memory={'ram':
                         {'used': 2.31, 'total': 63.92}, 'gpu': {'used': 20.57, 'total': 23.99}, 'retries': 0, 'oom': 0}
18:06:17-451491 DEBUG    GC: utilization={'gpu': 86, 'ram': 4, 'threshold': 80} gc={'collected': 287, 'saved': 0.01}
                         before={'gpu': 20.57, 'ram': 2.31} after={'gpu': 20.56, 'ram': 2.31, 'retries': 0, 'oom': 0}
                         device=cuda fn=apply_unet:load_unet time=0.28
18:06:17-454482 DEBUG    XYZ grid apply unet: "fluxDEVDEDISTILLED_fp16"
18:06:17-455479 DEBUG    Override: {'sd_unet': 'fluxDEVDEDISTILLED_fp16'}
18:06:17-729746 DEBUG    GC: utilization={'gpu': 86, 'ram': 4, 'threshold': 80} gc={'collected': 127, 'saved': 0.01}
                         before={'gpu': 20.57, 'ram': 2.31} after={'gpu': 20.56, 'ram': 2.31, 'retries': 0, 'oom': 0}
                         device=cuda fn=activate:load_networks time=0.27
18:06:17-801555 INFO     Base: class=FluxPipeline
18:06:17-803549 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
18:06:17-809533 DEBUG    Torch generator: device=cuda seeds=[3827383365]
18:06:17-810530 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.04s/it █████████████████████████████████ 100% 60/60 01:02 00:00 Base
18:07:20-749205 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=0.09
18:07:20-881850 INFO     Processed: images=1 its=0.95 time=63.41 timers={'gc': 0.27, 'init': 0.28, 'prepare': 0.07,
                         'callback': 7.69, 'pipeline': 62.79, 'decode': 0.25, 'post': 0.03} memory={'ram': {'used': 0.6,
                         'total': 63.92}, 'gpu': {'used': 17.04, 'total': 23.99}, 'retries': 0, 'oom': 0}
18:07:20-887836 INFO     Load model: select="Diffusers\Disty0/FLUX.1-dev-qint4 [82811df42b]"
18:07:20-889830 DEBUG    Load model:
                         path="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb"
18:07:20-891825 INFO     Autodetect model: detect="FLUX" class=FluxPipeline
                         file="models\Diffusers\models--Disty0--FLUX.1-dev-qint4\snapshots\82811df42b556a1153b971d8375d5
                         170c306a6eb" size=0MB
18:07:20-894817 DEBUG    Load model: type=FLUX model="Diffusers\Disty0/FLUX.1-dev-qint4" repo="Disty0/FLUX.1-dev-qint4"
                         unet="fluxDEVDEDISTILLED_fp16" t5="ViT-L-14-BEST-smooth-GmP-TE-only-HF-format" vae="None"
                         quant=qint4 offload=none dtype=torch.bfloat16
18:07:20-896811 DEBUG    HF login: no token provided
18:07:20-900800 INFO     Load module: type=UNet/Transformer
                         file="C:\ai\automatic\models\UNET\fluxDEVDEDISTILLED_fp16.safetensors" offload=none quant=none
                         dtype=torch.bfloat16
18:07:37-673571 DEBUG    HF login: no token provided
18:08:00-209611 DEBUG    Server: alive=True jobs=1 requests=822 uptime=537 memory=29.39/63.92 backend=Backend.DIFFUSERS
                         state=idle
18:09:06-361745 DEBUG    Load model: type=FLUX preloaded=['transformer', 'text_encoder_2']
Diffusers  7.40it/s ████████ 100% 7/7 00:00 00:00 Loading pipeline components...
18:09:07-630152 INFO     Load network: type=embeddings loaded=0 skipped=13 time=0.02
18:09:07-632146 DEBUG    Setting model: component=VAE slicing=True
18:09:07-633144 DEBUG    Setting model: attention="Scaled-Dot-Product"
18:09:07-651096 DEBUG    Setting model: offload=none
18:09:12-346754 DEBUG    GC: utilization={'gpu': 86, 'ram': 15, 'threshold': 80} gc={'collected': 613, 'saved': 0.0}
                         before={'gpu': 20.55, 'ram': 9.35} after={'gpu': 20.55, 'ram': 9.35, 'retries': 0, 'oom': 0}
                         device=cuda fn=load_diffuser:move_model time=0.26
18:09:12-605064 DEBUG    GC: utilization={'gpu': 86, 'ram': 15, 'threshold': 80} gc={'collected': 127, 'saved': 0.01}
                         before={'gpu': 20.56, 'ram': 9.35} after={'gpu': 20.55, 'ram': 9.35, 'retries': 0, 'oom': 0}
                         device=cuda fn=load_unet:load_diffuser time=0.26
18:09:12-616034 INFO     Load model: time=111.46 load=106.72 move=4.70 native=1024 memory={'ram': {'used': 9.35,
                         'total': 63.92}, 'gpu': {'used': 20.56, 'total': 23.99}, 'retries': 0, 'oom': 0}
18:09:12-874343 DEBUG    GC: utilization={'gpu': 86, 'ram': 15, 'threshold': 80} gc={'collected': 287, 'saved': 0.01}
                         before={'gpu': 20.56, 'ram': 9.35} after={'gpu': 20.55, 'ram': 9.35, 'retries': 0, 'oom': 0}
                         device=cuda fn=apply_unet:load_unet time=0.26
18:09:12-876339 DEBUG    XYZ grid apply unet: "fluxDEVDEDISTILLED_q4KM.gguf"
18:09:12-877335 DEBUG    Override: {'sd_unet': 'fluxDEVDEDISTILLED_q4KM.gguf'}
18:09:13-136642 DEBUG    GC: utilization={'gpu': 86, 'ram': 15, 'threshold': 80} gc={'collected': 127, 'saved': 0.01}
                         before={'gpu': 20.56, 'ram': 9.35} after={'gpu': 20.55, 'ram': 9.35, 'retries': 0, 'oom': 0}
                         device=cuda fn=activate:load_networks time=0.26
18:09:13-171549 INFO     Base: class=FluxPipeline
18:09:13-173543 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
18:09:13-176535 DEBUG    Torch generator: device=cuda seeds=[3827383365]
18:09:13-177533 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 6, 'num_inference_steps': 60, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.02s/it ████████████████████████▊          75% 45/60 00:46 00:15 Base18:10:00-467663 DEBUG    Server: alive=True jobs=1 requests=1063 uptime=657 memory=0.6/63.92 backend=Backend.DIFFUSERS
                         state=idle
Progress  1.03s/it █████████████████████████████████ 100% 60/60 01:01 00:00 Base
18:10:16-240730 DEBUG    GC: utilization={'gpu': 90, 'ram': 1, 'threshold': 80} gc={'collected': 407, 'saved': 9.51}
                         before={'gpu': 21.53, 'ram': 0.6} after={'gpu': 12.02, 'ram': 0.6, 'retries': 0, 'oom': 0}
                         device=cuda fn=process_base:nextjob time=0.31
18:10:16-277631 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=False latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=0.034
18:10:16-456155 INFO     Processed: images=1 its=0.94 time=63.57 timers={'gc': 0.57, 'init': 0.26, 'prepare': 0.04,
                         'callback': 7.86, 'pipeline': 62.69, 'decode': 0.55, 'post': 0.04} memory={'ram': {'used':
                         0.63, 'total': 63.92}, 'gpu': {'used': 16.83, 'total': 23.99}, 'retries': 0, 'oom': 0}
18:10:16-553409 INFO     XYZ grid complete: images=4 size=(4096, 1298) time=531.26 save=0.10
18:10:16-584326 INFO     Processed: images=5 its=15831.74 time=0.02 timers={'gc': 0.57, 'init': 0.36, 'prepare': 0.04,
                         'callback': 7.86, 'pipeline': 62.69, 'decode': 0.55, 'post': 0.07} memory={'ram': {'used':
                         0.65, 'total': 63.92}, 'gpu': {'used': 16.83, 'total': 23.99}, 'retries': 0, 'oom': 0}
18:10:17-201676 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmp7yv3yl3q.png" width=4096
                         height=1298 size=1329223
18:10:17-703072 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmpdlrvxi7w.png" width=1024
                         height=1024 size=1155910
18:10:18-144271 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmphue0aa38.png" width=1024
                         height=1024 size=1155940
18:10:18-575044 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmpcny0bcbd.png" width=1024
                         height=1024 size=1155941
18:10:19-101484 DEBUG    Save temp: image="C:\Users\sebas\AppData\Local\Temp\gradio\tmp8gdp0aj4.png" width=1024
                         height=1024 size=1155946

vladmandic · 2024-10-13T18:14:37Z

vladmandic
Oct 13, 2024
Maintainer

i don't see unet actually loading, i'll check.
update: fixed in latest dev.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to use single-file custom UNET with Flux? #3487

{{title}}

Replies: 2 comments 1 reply

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

How to use single-file custom UNET with Flux? #3487

SAC020 Oct 13, 2024

Replies: 2 comments · 1 reply

vladmandic Oct 13, 2024 Maintainer

SAC020 Oct 13, 2024 Author

vladmandic Oct 13, 2024 Maintainer

SAC020
Oct 13, 2024

Replies: 2 comments 1 reply

vladmandic
Oct 13, 2024
Maintainer

SAC020 Oct 13, 2024
Author

vladmandic
Oct 13, 2024
Maintainer