Image Processing Stuck

[yasakrami]

I've encountered an issue with the Unique3D API where the image processing gets stuck and does not proceed. Initially, I noticed that the model was not progressing beyond a certain point, so I updated and switched to the latest Stable Diffusion model available on Hugging Face for Unique3D (specifically the benjamin-paine/stable-diffusion-v1-5 model). However, despite the update, the issue persists, with the process hanging in the same spot, without returning any results. The log indicates a problem with the view direction (cos_angles.mean() error), and switching models did not resolve the issue. Additionally, the Hugging Face demo for this model is also not working, which suggests the problem may be more widespread.

[josepmy]

This is an old issue. In my opinion, maybe adding some margin to the angle could bypass the error, although other people suggest simply commenting it out

[yasakrami]

This is an old issue. In my opinion, maybe adding some margin to the angle could bypass the error, although other people suggest simply commenting it out

Thank you but the problem is that it gets stuck in this step and even after waiting for 45min, there is no result. Do you have any suggestions regarding this?

[josepmy]

This line in project_mesh.py

# find invalid faces cos_angles = (faces_normals * view_direction).sum(dim=1) assert cos_angles.mean() < 0, f"The view direction is not correct. cos_angles.mean()={cos_angles.mean()}" selected_faces = unique_faces[cos_angles < -eps]

To test individually

Option 1

cos_angles = (faces_normals * view_direction).sum(dim=1) if cos_angles.mean() >= 0: print(f"Warning: Unusual view direction detected. cos_angles.mean()={cos_angles.mean()}") # Optional invert normals to test faces_normals = -faces_normals cos_angles = (faces_normals * view_direction).sum(dim=1) selected_faces = unique_faces[cos_angles < -eps]

Option 2
If not use absolute value, should be work but need test
cos_angles = (faces_normals * view_direction).sum(dim=1) print(f"Info: cos_angles.mean()={cos_angles.mean()}") selected_faces = unique_faces[cos_angles.abs() > eps]

Option 3
or my option is add margin
cos_angles = (faces_normals * view_direction).sum(dim=1) threshold = -eps if cos_angles.mean() < 0 else eps selected_faces = unique_faces[cos_angles < threshold]

and add logger
print(f"Number of faces: {len(faces_normals)}")
print(f"View direction: {view_direction}")
print(f"Cos angles range: {cos_angles.min().item()} to {cos_angles.max().item()}")
print(f"Cos angles distribution: {torch.histc(cos_angles, bins=10)}")

test and show results

[josepmy]

@yasakrami did work?