RTX 3050Ti 16GB Running Insanely Slow (Current Build)

[ArmyOfPun1776]

Howdy,

Before I get into it let me say; I really appreciate the work you do and the tool you've provided. I am a user of your COLAB branch and really enjoy it's ease of use.

I recently got a new laptop with a RTX 3050Ti card and decided it was time I tried my hand at local LoRa Training. Wish I could say it's been a pleasant experience.

I am by now means even partially adept with this type of program, and am sure that my experience is the product of something I've done wrong.

I followed your install instructions for Windows step by step and, after some time got the GUI running.
*running on Local Machine
*running on only the GPU
*set GPU ID as (ALL), (0), and (1) in multiple config attempts
*Installed CUDDA (read this can help with speeds)
*Installed bytesands for Windows (tried all versions)
*Installed CUDA Toolkit and Drivers (latest version)
*Installed Visual Studio ReDist
*I'm not sure what version of Torch I am on (Newb)
*installed Python by way(proxy) of Automatic1111 install (So what ever version came with that I guess? Newb)

Also important to note that I deleted 'logs' and 'venv' folders before performing each "fresh" install.

Something else I've noticed: My GPU has split MEM. 4GB dedicated and 12GB Shared. I'm not sure if this is a symptom of the issue, but it wouldn't surprise me in the slightest.

Install 1:

setup all the folders, parameters, etc. and began training.

• GPU ID set to ALL
• 143 Images (70 Training and 73 Regularization)
• 2 Batches
• 2 Epochs
• Saves at every 700 steps for testing
• Adam8bit on constant
• Gradient Checkpoints and memory efficiency enabled
• Kohya shows the process will take ~5hrs 30min (Let this be even though I thought it was odd it was going to run so long)

At ~step 700 I got Nan=Loss

Install 2:

• GPU ID set to 0
  All the same parameters except I dropped the Regularization Images. So 70 Images.
• ~26hrs estimated completion time. (Couldn't be right)
• Terminated the training.

---

Insatll 3:

• GPU ID set to 1
• Same parameters as Install 2

same results as Install 2

Install 4:

• GPU ID set to ALL
• Same parameters as Install 2

Caching of images is now taking ~50min (Terminated Training)

Again, I'm a newb to all of this (including coding), but I'm not too afraid to try things so long as I understand what I'm doing to a comfortable degree. However, a lot of what I read on Git and other forums assume a level of understanding that I just don't have at this point. I'm trying... lol

I'm an IT guy and hate when I have to ask for someone to "hold my hand" but... yeah... I'm lost.

Thanks for your attention. I'm sure I'm leaving pertinent information out here, should you decide to help me out here. Please let me know if I need to provide any additional info, logs, etc.

[pheobus63]

Hi the RTX3050Ti has just 4GB GDDR6 (VRAM) I believe. Forget shared as that just slows everything down considerably.