Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image Processing Stuck #103

Open
yasakrami opened this issue Sep 16, 2024 · 4 comments
Open

Image Processing Stuck #103

yasakrami opened this issue Sep 16, 2024 · 4 comments

Comments

@yasakrami
Copy link

I've encountered an issue with the Unique3D API where the image processing gets stuck and does not proceed. Initially, I noticed that the model was not progressing beyond a certain point, so I updated and switched to the latest Stable Diffusion model available on Hugging Face for Unique3D (specifically the benjamin-paine/stable-diffusion-v1-5 model). However, despite the update, the issue persists, with the process hanging in the same spot, without returning any results. The log indicates a problem with the view direction (cos_angles.mean() error), and switching models did not resolve the issue. Additionally, the Hugging Face demo for this model is also not working, which suggests the problem may be more widespread.

Screenshot (570)

@josepmy
Copy link

josepmy commented Sep 16, 2024

This is an old issue. In my opinion, maybe adding some margin to the angle could bypass the error, although other people suggest simply commenting it out

@yasakrami
Copy link
Author

This is an old issue. In my opinion, maybe adding some margin to the angle could bypass the error, although other people suggest simply commenting it out

Thank you but the problem is that it gets stuck in this step and even after waiting for 45min, there is no result. Do you have any suggestions regarding this?

@josepmy
Copy link

josepmy commented Sep 16, 2024

This line in project_mesh.py

# find invalid faces cos_angles = (faces_normals * view_direction).sum(dim=1) assert cos_angles.mean() < 0, f"The view direction is not correct. cos_angles.mean()={cos_angles.mean()}" selected_faces = unique_faces[cos_angles < -eps]

To test individually

Option 1

cos_angles = (faces_normals * view_direction).sum(dim=1) if cos_angles.mean() >= 0: print(f"Warning: Unusual view direction detected. cos_angles.mean()={cos_angles.mean()}") # Optional invert normals to test faces_normals = -faces_normals cos_angles = (faces_normals * view_direction).sum(dim=1) selected_faces = unique_faces[cos_angles < -eps]

Option 2
If not use absolute value, should be work but need test
cos_angles = (faces_normals * view_direction).sum(dim=1) print(f"Info: cos_angles.mean()={cos_angles.mean()}") selected_faces = unique_faces[cos_angles.abs() > eps]

Option 3
or my option is add margin
cos_angles = (faces_normals * view_direction).sum(dim=1) threshold = -eps if cos_angles.mean() < 0 else eps selected_faces = unique_faces[cos_angles < threshold]

and add logger
print(f"Number of faces: {len(faces_normals)}")
print(f"View direction: {view_direction}")
print(f"Cos angles range: {cos_angles.min().item()} to {cos_angles.max().item()}")
print(f"Cos angles distribution: {torch.histc(cos_angles, bins=10)}")

test and show results

@josepmy
Copy link

josepmy commented Sep 18, 2024

@yasakrami did work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants