Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable offline single file model loading #6740

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Enable offline single file model loading #6740

wants to merge 19 commits into from

Conversation

lstein
Copy link
Collaborator

@lstein lstein commented Aug 14, 2024 •
edited
Loading

Summary

This PR bumps diffusers to v0.30.0 and reenables the ability to load main checkpoint models without a connection to the internet.

Related Issues / Discussions

Closes #6623.

QA Instructions

  1. Test generation with sd-1, sd-2 and sdxl models.
  2. Test controlnet guidance
  3. Test ti-adapter guidance
  4. Test ip-adapter guidance
  5. Test model loading when internet not connected.

Merge Plan

Given that this introduces the two regressions noted above, we should discuss whether to merge now or wait until the issue is fixed in diffusers.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)

@lstein lstein requested a review from psychedelicious August 14, 2024 02:53
@github-actions github-actions bot added python PRs that change python files Root backend PRs that change backend files python-deps PRs that change python dependencies labels Aug 14, 2024
@RyanJDick
Copy link
Collaborator

Link to the diffusers issue for reference: huggingface/diffusers#9171

@lstein lstein changed the title Use Diffusers 0.30.0 Use Diffusers 0.30.0 / Enable offline single file model loading Aug 17, 2024
@lstein lstein marked this pull request as draft August 17, 2024 18:59
@lstein
Copy link
Collaborator Author

lstein commented Aug 17, 2024 •
edited
Loading

The issue with v_prediction vs epsilon has been worked-around.
The issue with unconvertible sd 1.5 controlnets has been worked-around.
Loading of SDXL controlnet single files still does not work.

@lstein lstein marked this pull request as ready for review August 17, 2024 20:02
@psychedelicious psychedelicious mentioned this pull request Aug 20, 2024
Merged
8 tasks
@lstein lstein mentioned this pull request Sep 4, 2024
Open
3 tasks
brandonrising
brandonrising reviewed Oct 9, 2024
View reviewed changes
invokeai/backend/assets/model_base_conf_files/controlnet_sd15/config.json Outdated
{
"_class_name": "ControlNetModel",
"_diffusers_version": "0.16.0.dev0",
"_name_or_path": "/home/patrick/controlnet_v1_1/control_v11p_sd15_canny",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran an sd1 controlnet as a single file and it didn't seem to break as a result of this, but it seems weird to have this hard coded filepath here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _name_or_path field is ignored during file conversion, but these fields will be removed.

@brandonrising
Copy link
Collaborator

Are you still following this PR @lstein ? I can take it over and get it across the line if you'd like

@lstein
Copy link
Collaborator Author

lstein commented Oct 22, 2024

Are you still following this PR @lstein ? I can take it over and get it across the line if you'd like

I’m very happy to see renewed activity on this PR. I thought it was comatose. I’ll take it across the finish line.

@lstein
Copy link
Collaborator Author

lstein commented Oct 26, 2024

Are you still following this PR @lstein ? I can take it over and get it across the line if you'd like
@brandonrising
I've removed the absolute path name from the controlnet json file and have corrected a merge conflict. Should be ready for a re-review.

@lstein lstein changed the title Use Diffusers 0.30.0 / Enable offline single file model loading Enable offline single file model loading Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brandonrising brandonrising brandonrising left review comments

@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

@RyanJDick RyanJDick Awaiting requested review from RyanJDick RyanJDick is a code owner

@hipsterusername hipsterusername Awaiting requested review from hipsterusername hipsterusername is a code owner

@psychedelicious psychedelicious Awaiting requested review from psychedelicious

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
backend PRs that change backend files python PRs that change python files python-deps PRs that change python dependencies Root
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[bug]: Image generation fails with a server error when starting InvokeAI offline.
3 participants
@lstein @RyanJDick @brandonrising