Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

torch-scatter doesn't declare torch as a build-time dependency #455

Open
CompRhys opened this issue Aug 5, 2024 · 1 comment
Open

torch-scatter doesn't declare torch as a build-time dependency #455

CompRhys opened this issue Aug 5, 2024 · 1 comment

Comments

@CompRhys
Copy link

CompRhys commented Aug 5, 2024

when using pep 517 with pip/uv torch-scatter fails to install with the following error:

error: Failed to download and build `torch-scatter==2.1.2`
  Caused by: Failed to build: `torch-scatter==2.1.2`
  Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 14, in <module>
  File "/home/runner/.cache/uv/builds-v0/.tmp4LP0zO/lib/python3.11/site-packages/setuptools/build_meta.py", line 327, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=[])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.cache/uv/builds-v0/.tmp4LP0zO/lib/python3.11/site-packages/setuptools/build_meta.py", line 297, in _get_build_requires
    self.run_setup()
  File "/home/runner/.cache/uv/builds-v0/.tmp4LP0zO/lib/python3.11/site-packages/setuptools/build_meta.py", line 497, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/home/runner/.cache/uv/builds-v0/.tmp4LP0zO/lib/python3.11/site-packages/setuptools/build_meta.py", line 313, in run_setup
    exec(code, locals())
  File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'torch'
---
  Caused by: This error likely indicates that torch-scatter==2.1.2 depends on torch, but doesn't declare it as a build dependency. If torch-scatter==2.1.2 is a first-party package, consider adding torch to its `build-system.requires`. Otherwise, `uv pip install torch` into the environment and re-run with `--no-build-isolation`.

As this is fairly common (common enough that there is a custom error message in uv) would it be possible to declare torch as a build time dependency?

See reference:
https://peps.python.org/pep-0517/
astral-sh/uv#5040

@itrase
Copy link

itrase commented Sep 4, 2024

This actually ends up being a bit more complex. This comment gives a good summary of what's going on here - effectively, many packages that rely on torch and CUDA don't want build isolation, since they want to be compiled against the specific version of torch that is installed in the environment. If torch was declared as a build-time dependency, you'd likely end up with a torch-scatter that was compiled against a different version of torch than was actually installed. This is also why packages like torch-scatter tell you to do things like this:

pip install torch-scatter -f https://data.pyg.org/whl/torch-2.4.0+${CUDA}.html
Where you are trying to get the torch-scatter wheel that was compiled for your specific versions of torch and CUDA. When this works, it sidesteps the whole issue - but I've had trouble getting it to work with uv and torch-scatter.

A neater solution is this - allowing build isolation for everything except torch. I also haven't gotten this to work successfully in uv - I think it is new and a little buggy.

Ideally, packages like torch-scatter would have a way to say "I want build isolation except for these specific packages" within their own setup files, but I am not sure there is any syntax that supports that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants