v0.15.0 - VideoToVideo Diffusion + ControlNet Support

[saddam213]

VideoToVideo Diffusion + ControlNet Support

Video2Video - Pass Videos or GIF images though your favorite Stable-Diffusion pipeline and get Video output
ControlNet - Highly requested feature, initial support includes ControlNet, ImageToImage and VideoToVideo

What's Changed

• Video Support by @saddam213 in Video Support #85
• DirectX Device Detection by @saddam213 in DX Device Detection #87
• ControlNet Support by @saddam213 in Add ControlNet Support #88

Nuget Packages Updated

https://www.nuget.org/packages/OnnxStack.Core/0.15.0
https://www.nuget.org/packages/OnnxStack.ImageUpscaler/0.15.0
https://www.nuget.org/packages/OnnxStack.StableDiffusion/0.15.0

Full Changelog: v0.14.3...v0.15.0

Example UI Builds

Name	ONNX EP	Device	Requirements	Status
OnnxStack_UI.zip	DirectML	CPU, AMD GPU, Nvidia GPU	At least Windows10	Stable
OnnxStack_UI_Cuda.zip	CUDA	CPU, Nvidia GPU	CUDA 11 and cuDNN toolkit	Experimental
OnnxStack_UI_TensorRT.zip	TensorRT	CPU, Nvidia GPU	CUDA 11 , cuDNN and TensorRT libraries	Experimental

---

This discussion was created from the release v0.15.0 - VideoToVideo Diffusion + ControlNet Support.

[saddam213]

OnnxStack UI Setup Steps for ControlNet

Onnx ControlNet
Download or clone all files
Onnx ControlNet

Onnx ControlNet enabled StableDiffusion models
Download or clone one of these models
Stable Diffusion 1.5 with ControlNet
Realistic Vision with ControlNet

Install the ControlNet enabled StableDiffusion model using the SD-ControlNet template

Then install one or more of the desired ControlNet models

Some models support Annotation models they help pre-process the image for control net, these are optional and can be left blank

When you select a ControlNet enabled StableDiffusion model a dropdown with the ControlNet models available is shown

Once you have loaded both models you can now use control net.