ValueError: torch.cuda.is_available() should be True but is False

[Jennal]

I always got this error when I start to train Lora.

10:09:34-562798 INFO     Kohya_ss GUI version: v23.1.4
10:09:34-890842 INFO     Submodule initialized and updated.
10:09:34-890842 INFO     nVidia toolkit detected
10:09:35-750562 INFO     Torch 2.1.2+cu118
10:09:35-766065 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8905
10:09:35-766065 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 24564 Arch (8, 9) Cores 128
10:09:35-766065 INFO     Python version is 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit
                         (AMD64)]
10:09:35-766065 INFO     Verifying modules installation status from requirements_pytorch_windows.txt...
10:09:35-766065 INFO     Verifying modules installation status from requirements_windows.txt...
10:09:35-766065 INFO     Verifying modules installation status from requirements.txt...

...

Error in sys.excepthook:
Traceback (most recent call last):
  File "D:\StableDiffusion\kohya_ss\sd-scripts\sdxl_train_network.py", line 185, in <module>
    trainer.train(args)
  File "D:\StableDiffusion\kohya_ss\sd-scripts\train_network.py", line 242, in train
    vae.set_use_memory_efficient_attention_xformers(args.xformers)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 262, in set_use_memory_efficient_attention_xformers
    fn_recursive_set_mem_eff(module)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 255, in fn_recursive_set_mem_eff
    module.set_use_memory_efficient_attention_xformers(valid, attention_op)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\attention_processor.py", line 260, in set_use_memory_efficient_attention_xformers
    raise ValueError(
ValueError: torch.cuda.is_available() should be True but is False. xformers' memory efficient attention is only available for GPU
Traceback (most recent call last):
  File "C:\Users\Administrator\.pyenv\pyenv-win\versions\3.10.11\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Administrator\.pyenv\pyenv-win\versions\3.10.11\lib\runpy.py", line 86, in _run_code
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
    simple_launcher(args)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)

My host PC has python 3.10.11 from pyenv global setting. And venv python is the same version. And I have installed cuda 11.8 and 12.4, I have configured 11.8 to PATH, and make sure it is the default one. I try both host python and venv python, start python and enter these code:

import torch
torch.cuda.is_available()

The output is True. Then I try to add these code into gui.bat after activate the virtual environment

@echo off

set PYTHON_VER=3.10.9

:: Deactivate the virtual environment
call .\venv\Scripts\deactivate.bat

:: Activate the virtual environment
call .\venv\Scripts\activate.bat
set PATH=%PATH%;%~dp0venv\Lib\site-packages\torch\lib

:: Validate requirements
python.exe .\setup\validate_requirements.py
if %errorlevel% neq 0 exit /b %errorlevel%

↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
:: Debug: Check if GPU is available
echo 'Checking CUDA:'
python.exe -c "import torch; print(torch.cuda.is_available())"
↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

:: If the exit code is 0, run the kohya_gui.py script with the command-line arguments
if %errorlevel% equ 0 (
    REM Check if the batch was started via double-click
    IF /i "%comspec% /c %~0 " equ "%cmdcmdline:"=%" (
        REM echo This script was started by double clicking.
        cmd /k python.exe kohya_gui.py %*
    ) ELSE (
        REM echo This script was started from a command prompt.
        python.exe kohya_gui.py %*
    )
)

The output is

10:52:53-206445 INFO     Kohya_ss GUI version: v23.1.4
10:52:53-503239 INFO     Submodule initialized and updated.
10:52:53-503239 INFO     nVidia toolkit detected
10:52:54-436926 INFO     Torch 2.1.2+cu118
10:52:54-452496 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8905
10:52:54-452496 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 24564 Arch (8, 9) Cores 128
10:52:54-452496 INFO     Python version is 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit
                         (AMD64)]
10:52:54-452496 INFO     Verifying modules installation status from requirements_pytorch_windows.txt...
10:52:54-468120 INFO     Verifying modules installation status from requirements_windows.txt...
10:52:54-468120 INFO     Verifying modules installation status from requirements.txt...
'Checking CUDA:'
True
10:52:58-251406 INFO     headless: False
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

But when I start train, the script stoped at the same error.
By the way, I have 2 GPU, one is CPU embeded GPU from Intel 14900K, the other is 4090.
In Manually configure Accelerate from setup.bat, I answered alll to this question

What GPU(s) (by id) should be used for training on this machine as a comma-seperated list? [all]:

Or how can I specify the script to use 4090 only?

I appreciate if someone can help.

[Jennal]

I try to reinstall all venv environment, and still got same issue. But I try the last command directly in cmd, it is working good.

18:52:22-074081 INFO     Kohya_ss GUI version: v24.0.3
18:52:22-370878 INFO     Submodule initialized and updated.
18:52:22-370878 INFO     nVidia toolkit detected
18:52:23-285349 INFO     Torch 2.1.2+cu118
18:52:23-300976 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8905
18:52:23-302071 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 24564 Arch (8, 9) Cores 128
18:52:23-302071 INFO     Python version is 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit
                         (AMD64)]
18:52:23-302071 INFO     Verifying modules installation status from requirements_pytorch_windows.txt...
18:52:23-302071 INFO     Verifying modules installation status from requirements_windows.txt...
18:52:23-302071 INFO     Verifying modules installation status from requirements.txt...
18:52:26-000706 INFO     headless: False
18:52:26-031957 INFO     Using shell=True when running external commands...
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
18:52:30-474339 INFO     Start training LoRA Standard ...
18:52:30-475340 INFO     Validating model file or folder path
                         D:/StableDiffusion/sd-webui-aki-v4.7/models/Stable-diffusion/sd_xl_base_1.0.safetensors
                         existence...
18:52:30-475340 INFO     ...valid
18:52:30-476340 INFO     Validating output_dir path C:/Work/Fish/temp/kohya_train/model existence...
18:52:30-476835 INFO     ...valid
18:52:30-476835 INFO     Validating train_data_dir path C:/Work/Fish/temp/kohya_train/img existence...
18:52:30-477840 INFO     ...valid
18:52:30-478254 INFO     reg_data_dir not specified, skipping validation
18:52:30-478715 INFO     Validating logging_dir path C:/Work/Fish/temp/kohya_train/log existence...
18:52:30-478715 INFO     ...valid
18:52:30-479718 INFO     log_tracker_config not specified, skipping validation
18:52:30-479718 INFO     resume not specified, skipping validation
18:52:30-480718 INFO     Validating vae path D:/StableDiffusion/sd-webui-aki-v4.7/models/VAE/sdxl_vae.safetensors
                         existence...
18:52:30-480718 INFO     ...valid
18:52:30-481718 INFO     lora_network_weights not specified, skipping validation
18:52:30-481718 INFO     dataset_config not specified, skipping validation
18:52:30-482718 INFO     Folder 6_chillquarium fish: 6 repeats found
18:52:30-482718 INFO     Folder 6_chillquarium fish: 76 images found
18:52:30-483719 INFO     Folder 6_chillquarium fish: 76 * 6 = 456 steps
18:52:30-483719 INFO     Regulatization factor: 1
18:52:30-484718 INFO     Total steps: 456
18:52:30-484718 INFO     Train batch size: 1
18:52:30-485718 INFO     Gradient accumulation steps: 1
18:52:30-485718 INFO     Epoch: 10
18:52:30-485718 INFO     max_train_steps (456 / 1 / 1 * 10 * 1) = 4560
18:52:30-486718 INFO     stop_text_encoder_training = 0
18:52:30-486718 INFO     lr_warmup_steps = 456
18:52:30-488805 INFO     Saving training config to
                         C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16_20240418-185230.json...
18:52:30-489809 INFO     Executing command: "D:\StableDiffusion\kohya_ss\venv\Scripts\accelerate.EXE" launch
                         --dynamo_backend no --dynamo_mode default --gpu_ids 1 --mixed_precision bf16 --num_processes 1
                         --num_machines 1 --num_cpu_threads_per_process 2
                         "D:/StableDiffusion/kohya_ss/sd-scripts/sdxl_train_network.py" --config_file
                         "./outputs/tmpfilelora.toml" with shell=True
18:52:30-493829 INFO     Command executed.
2024-04-18 18:52:35 INFO     Loading settings from ./outputs/tmpfilelora.toml...                      train_util.py:3744
                    INFO     ./outputs/tmpfilelora                                                    train_util.py:3763
2024-04-18 18:52:35 INFO     prepare tokenizers                                                   sdxl_train_util.py:134
2024-04-18 18:52:36 INFO     update token length: 75                                              sdxl_train_util.py:159
                    INFO     Using DreamBooth method.                                               train_network.py:172
                    INFO     prepare images.                                                          train_util.py:1572
                    INFO     found directory C:\Work\Fish\temp\kohya_train\img\6_chillquarium fish    train_util.py:1519
                             contains 76 image files
                    INFO     456 train images with repeating.                                         train_util.py:1613
                    INFO     0 reg images.                                                            train_util.py:1616
                    WARNING  no regularization images / 正則化画像が見つかりませんでした              train_util.py:1621
                    INFO     [Dataset 0]                                                              config_util.py:565
                               batch_size: 1
                               resolution: (1024, 1024)
                               enable_bucket: True
                               network_multiplier: 1.0
                               min_bucket_reso: 256
                               max_bucket_reso: 2048
                               bucket_reso_steps: 64
                               bucket_no_upscale: True

                               [Subset 0 of Dataset 0]
                                 image_dir: "C:\Work\Fish\temp\kohya_train\img\6_chillquarium fish"
                                 image_count: 76
                                 num_repeats: 6
                                 shuffle_caption: False
                                 keep_tokens: 0
                                 keep_tokens_separator:
                                 secondary_separator: None
                                 enable_wildcard: False
                                 caption_dropout_rate: 0
                                 caption_dropout_every_n_epoches: 0
                                 caption_tag_dropout_rate: 0.0
                                 caption_prefix: None
                                 caption_suffix: None
                                 color_aug: True
                                 flip_aug: True
                                 face_crop_aug_range: None
                                 random_crop: False
                                 token_warmup_min: 1,
                                 token_warmup_step: 0,
                                 is_reg: False
                                 class_tokens: chillquarium fish
                                 caption_extension: .txt


                    INFO     [Dataset 0]                                                              config_util.py:571
                    INFO     loading image sizes.                                                      train_util.py:853
100%|██████████████████████████████████████████████████████████████████████████████████████████| 76/76 [00:00<?, ?it/s]
                    INFO     make buckets                                                              train_util.py:859
                    WARNING  min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is   train_util.py:876
                             set, because bucket reso is defined by image size automatically /
                             bucket_no_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計
                             算されるため、min_bucket_resoとmax_bucket_resoは無視されます
                    INFO     number of images (including repeats) /                                    train_util.py:905
                             各bucketの画像枚数（繰り返し回数を含む）
                    INFO     bucket 0: resolution (1024, 1024), count: 456                             train_util.py:910
                    INFO     mean ar error (without repeats): 0.0                                      train_util.py:915
                    WARNING  clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません   sdxl_train_util.py:343
                    INFO     preparing accelerator                                                  train_network.py:225
accelerator device: cpu
                    INFO     loading model for process 0/1                                         sdxl_train_util.py:30
                    INFO     load StableDiffusion checkpoint:                                      sdxl_train_util.py:70
                             D:/StableDiffusion/sd-webui-aki-v4.7/models/Stable-diffusion/sd_xl_ba
                             se_1.0.safetensors
                    INFO     building U-Net                                                       sdxl_model_util.py:192
                    INFO     loading U-Net from checkpoint                                        sdxl_model_util.py:196
2024-04-18 18:52:38 INFO     U-Net: <All keys matched successfully>                               sdxl_model_util.py:202
                    INFO     building text encoders                                               sdxl_model_util.py:205
                    INFO     loading text encoders from checkpoint                                sdxl_model_util.py:258
                    INFO     text encoder 1: <All keys matched successfully>                      sdxl_model_util.py:272
2024-04-18 18:52:39 INFO     text encoder 2: <All keys matched successfully>                      sdxl_model_util.py:276
                    INFO     building VAE                                                         sdxl_model_util.py:279
                    INFO     loading VAE from checkpoint                                          sdxl_model_util.py:284
                    INFO     VAE: <All keys matched successfully>                                 sdxl_model_util.py:287
                    INFO     load VAE:                                                                model_util.py:1268
                             D:/StableDiffusion/sd-webui-aki-v4.7/models/VAE/sdxl_vae.safetensors
                    INFO     additional VAE loaded                                                sdxl_train_util.py:128
                    INFO     Enable xformers for U-Net                                                train_util.py:2660
Traceback (most recent call last):
  File "D:\StableDiffusion\kohya_ss\sd-scripts\sdxl_train_network.py", line 185, in <module>
    trainer.train(args)
  File "D:\StableDiffusion\kohya_ss\sd-scripts\train_network.py", line 242, in train
    vae.set_use_memory_efficient_attention_xformers(args.xformers)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 262, in set_use_memory_efficient_attention_xformers
    fn_recursive_set_mem_eff(module)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 258, in fn_recursive_set_mem_eff
    fn_recursive_set_mem_eff(child)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\modeling_utils.py", line 255, in fn_recursive_set_mem_eff
    module.set_use_memory_efficient_attention_xformers(valid, attention_op)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\diffusers\models\attention_processor.py", line 260, in set_use_memory_efficient_attention_xformers
    raise ValueError(
ValueError: torch.cuda.is_available() should be True but is False. xformers' memory efficient attention is only available for GPU
Traceback (most recent call last):
  File "C:\Users\Administrator\.pyenv\pyenv-win\versions\3.10.11\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Administrator\.pyenv\pyenv-win\versions\3.10.11\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\StableDiffusion\kohya_ss\venv\Scripts\accelerate.exe\__main__.py", line 7, in <module>
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
    args.func(args)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
    simple_launcher(args)
  File "D:\StableDiffusion\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['D:\\StableDiffusion\\kohya_ss\\venv\\Scripts\\python.exe', 'D:/StableDiffusion/kohya_ss/sd-scripts/sdxl_train_network.py', '--config_file', './outputs/tmpfilelora.toml']' returned non-zero exit status 1.
18:52:41-735028 INFO     Training has ended.

And if I put this command directly in cmd.

D:\\StableDiffusion\\kohya_ss\\venv\\Scripts\\python.exe D:/StableDiffusion/kohya_ss/sd-scripts/sdxl_train_network.py --config_file ./outputs/tmpfilelora.toml

It is working.

d:\StableDiffusion\kohya_ss>D:\\StableDiffusion\\kohya_ss\\venv\\Scripts\\python.exe D:/StableDiffusion/kohya_ss/sd-scripts/sdxl_train_network.py --config_file ./outputs/tmpfilelora.toml
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
2024-04-18 18:56:01.462099: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-18 18:56:01.815867: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-04-18 18:56:02 INFO     Loading settings from ./outputs/tmpfilelora.toml...                      train_util.py:3744
                    INFO     ./outputs/tmpfilelora                                                    train_util.py:3763
2024-04-18 18:56:02 INFO     prepare tokenizers                                                   sdxl_train_util.py:134
2024-04-18 18:56:03 INFO     update token length: 75                                              sdxl_train_util.py:159
                    INFO     Using DreamBooth method.                                               train_network.py:172
                    INFO     prepare images.                                                          train_util.py:1572
                    INFO     found directory C:\Work\Fish\temp\kohya_train\img\6_chillquarium fish    train_util.py:1519
                             contains 76 image files
                    INFO     456 train images with repeating.                                         train_util.py:1613
                    INFO     0 reg images.                                                            train_util.py:1616
                    WARNING  no regularization images / 正則化画像が見つかりませんでした              train_util.py:1621
                    INFO     [Dataset 0]                                                              config_util.py:565
                               batch_size: 1
                               resolution: (1024, 1024)
                               enable_bucket: True
                               network_multiplier: 1.0
                               min_bucket_reso: 256
                               max_bucket_reso: 2048
                               bucket_reso_steps: 64
                               bucket_no_upscale: True

                               [Subset 0 of Dataset 0]
                                 image_dir: "C:\Work\Fish\temp\kohya_train\img\6_chillquarium fish"
                                 image_count: 76
                                 num_repeats: 6
                                 shuffle_caption: False
                                 keep_tokens: 0
                                 keep_tokens_separator:
                                 secondary_separator: None
                                 enable_wildcard: False
                                 caption_dropout_rate: 0
                                 caption_dropout_every_n_epoches: 0
                                 caption_tag_dropout_rate: 0.0
                                 caption_prefix: None
                                 caption_suffix: None
                                 color_aug: True
                                 flip_aug: True
                                 face_crop_aug_range: None
                                 random_crop: False
                                 token_warmup_min: 1,
                                 token_warmup_step: 0,
                                 is_reg: False
                                 class_tokens: chillquarium fish
                                 caption_extension: .txt


                    INFO     [Dataset 0]                                                              config_util.py:571
                    INFO     loading image sizes.                                                      train_util.py:853
100%|████████████████████████████████████████████████████████████████████████████████| 76/76 [00:00<00:00, 4876.65it/s]
                    INFO     make buckets                                                              train_util.py:859
                    WARNING  min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is   train_util.py:876
                             set, because bucket reso is defined by image size automatically /
                             bucket_no_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計
                             算されるため、min_bucket_resoとmax_bucket_resoは無視されます
                    INFO     number of images (including repeats) /                                    train_util.py:905
                             各bucketの画像枚数（繰り返し回数を含む）
                    INFO     bucket 0: resolution (1024, 1024), count: 456                             train_util.py:910
                    INFO     mean ar error (without repeats): 0.0                                      train_util.py:915
                    WARNING  clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません   sdxl_train_util.py:343
                    INFO     preparing accelerator                                                  train_network.py:225
accelerator device: cuda
                    INFO     loading model for process 0/1                                         sdxl_train_util.py:30
                    INFO     load StableDiffusion checkpoint:                                      sdxl_train_util.py:70
                             D:/StableDiffusion/sd-webui-aki-v4.7/models/Stable-diffusion/sd_xl_ba
                             se_1.0.safetensors
                    INFO     building U-Net                                                       sdxl_model_util.py:192
                    INFO     loading U-Net from checkpoint                                        sdxl_model_util.py:196
2024-04-18 18:56:05 INFO     U-Net: <All keys matched successfully>                               sdxl_model_util.py:202
                    INFO     building text encoders                                               sdxl_model_util.py:205
                    INFO     loading text encoders from checkpoint                                sdxl_model_util.py:258
                    INFO     text encoder 1: <All keys matched successfully>                      sdxl_model_util.py:272
                    INFO     text encoder 2: <All keys matched successfully>                      sdxl_model_util.py:276
                    INFO     building VAE                                                         sdxl_model_util.py:279
                    INFO     loading VAE from checkpoint                                          sdxl_model_util.py:284
                    INFO     VAE: <All keys matched successfully>                                 sdxl_model_util.py:287
                    INFO     load VAE:                                                                model_util.py:1268
                             D:/StableDiffusion/sd-webui-aki-v4.7/models/VAE/sdxl_vae.safetensors
2024-04-18 18:56:06 INFO     additional VAE loaded                                                sdxl_train_util.py:128
                    INFO     Enable xformers for U-Net                                                train_util.py:2660
import network module: networks.lora
                    INFO     create LoRA network. base dim (rank): 8, alpha: 1                               lora.py:810
                    INFO     neuron dropout: p=0, rank dropout: p=None, module dropout: p=None               lora.py:811
                    INFO     create LoRA for Text Encoder 1:                                                 lora.py:902
                    INFO     create LoRA for Text Encoder 2:                                                 lora.py:902
                    INFO     create LoRA for Text Encoder: 264 modules.                                      lora.py:910
                    INFO     create LoRA for U-Net: 722 modules.                                             lora.py:918
                    INFO     enable LoRA for text encoder                                                    lora.py:961
                    INFO     enable LoRA for U-Net                                                           lora.py:966
prepare optimizer, data loader etc.
                    INFO     use 8-bit AdamW optimizer | {}                                           train_util.py:3889
override steps. steps for 10 epochs is / 指定エポックまでのステップ数: 4560
running training / 学習開始
  num train images * repeats / 学習画像の数×繰り返し回数: 456
  num reg images / 正則化画像の数: 0
  num batches per epoch / 1epochのバッチ数: 456
  num epochs / epoch数: 10
  batch size per device / バッチサイズ: 1
  gradient accumulation steps / 勾配を合計するステップ数 = 1
  total optimization steps / 学習ステップ数: 4560
steps:   0%|                                                                                  | 0/4560 [00:00<?, ?it/s]
epoch 1/10
steps:  10%|█████▌                                                 | 456/4560 [05:54<53:12,  1.29it/s, avr_loss=0.0586]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000001.safetensors

epoch 2/10
steps:  20%|███████████                                            | 912/4560 [11:34<46:19,  1.31it/s, avr_loss=0.0513]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000002.safetensors

epoch 3/10
steps:  30%|████████████████▏                                     | 1368/4560 [17:22<40:33,  1.31it/s, avr_loss=0.0513]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000003.safetensors

epoch 4/10
steps:  40%|█████████████████████▌                                | 1824/4560 [23:13<34:49,  1.31it/s, avr_loss=0.0559]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000004.safetensors

epoch 5/10
steps:  50%|███████████████████████████                           | 2280/4560 [28:57<28:57,  1.31it/s, avr_loss=0.0511]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000005.safetensors

epoch 6/10
steps:  60%|█████████████████████████████████                      | 2736/4560 [34:50<23:13,  1.31it/s, avr_loss=0.052]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000006.safetensors

epoch 7/10
steps:  70%|█████████████████████████████████████▊                | 3192/4560 [40:38<17:25,  1.31it/s, avr_loss=0.0535]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000007.safetensors

epoch 8/10
steps:  80%|███████████████████████████████████████████▏          | 3648/4560 [46:36<11:39,  1.30it/s, avr_loss=0.0507]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000008.safetensors

epoch 9/10
steps:  90%|████████████████████████████████████████████████▌     | 4104/4560 [52:24<05:49,  1.31it/s, avr_loss=0.0506]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16-000009.safetensors

epoch 10/10
steps: 100%|███████████████████████████████████████████████████████| 4560/4560 [58:28<00:00,  1.30it/s, avr_loss=0.053]
saving checkpoint: C:/Work/Fish/temp/kohya_train/model\fish_sdxl_bp16.safetensors
2024-04-18 19:54:41 INFO     model saved.                                                           train_network.py:999
steps: 100%|███████████████████████████████████████████████████████| 4560/4560 [58:28<00:00,  1.30it/s, avr_loss=0.053]

But still not working when I click Start training on web page.

[5KilosOfCheese]

Or how can I specify the script to use 4090 only?

Check your taskmaster and see which GPU number your 4090 has

You see the ID in "Physical location:" in my case it is 0 since I only have one GPU enabled. Specify this number in the acceleration setup.

Also in the GUI you can force override the accelerate setup to choose a specific GPU to use.

Don't worry about screwing up the accelerate config, you can redo it if need be.

[Jennal]

Thank you so much, I thought it was device 1, but it is device 0.