[Feature Request] - Apply Memory optimization from basujindal

[ReinerBforartists]

This guy here made some memory optimizations that allows to create 576x1280 images with 6 gb vram. Maybe worth a look at his script. And maybe you can get in touch with him to implement it into this repo. Maybe even worth to commit it into the main stable diffusion repository.

I would not suggest to copy it simply over though. Seems that there is some whitespace in the file, and some unnecessary code too.

basujindal/stable-diffusion#103

Kind regards
Reiner

[Filarius]

I had test for this repo. --optimized-turbo now use same or a bit less VRAM than --optimized
with 8 VRAM i can do about 702x1024 or 832x832 on --optimized-turbo

[Muskelmagier]

What is optimized-turbo?

[EliEron]

A version of the optimized mode that uses slightly more VRAM but is significantly faster, it is equivalent to using the --turbo option with basujindal's original script. It's generally recommended for cards with 6GB or so of VRAM.

[Muskelmagier]

Here is the fork repo https://github.com/neonsecret/stable-diffusion

[RandyRandom]

Even more now. With the latest update on his repo he managed to squeeze 768x2048 and 1216x1216 out of a 3070 with 8GB VRAM. Pretty Impressive.

[ReinerBforartists]

Okay, seems that i followed the wrong repo :D
Impressing :)

[RandyRandom]

I'm currently unable to replicate his claimed results. I tried to generate a image with 1216x1216 resolution on a dedicated 3060 Ti with 8GB and completely empty VRAM, but it was still not enough. It's missing around ~0,5-0,7GB. On Linux CUDA runs out of memory to allocate, but on Windows it uses additional 0,5GB of the virtual GPU memory or so called shared GPU memory. Still pretty impressive his improvements. I wonder why he gets different results.

[thedyze]

I've managed 1280x960 max using his code

[uncoolcat]

I've been able to achieve 1280x1024 with 50 steps in under 2 minutes on a 3090 with this fork.

[ReinerBforartists]

This would be two different things then i guess. One is fewer ram, the other improved speed.

[A2va]

There is also another optimization that was not in the neonsecret PR. But another PR could be released in a few days.
basujindal/stable-diffusion#117 (comment)

Ah, it seems to be this PR: basujindal/stable-diffusion#122

[mchaker]

The Doggettx optimization is cleaner and better, in my opinion, without sacrificing noticeable speed:

(item 1 in this list: https://gist.github.com/mchaker/e0cf180db6849fa03696293b3fc8f485 )