can not install torch-scatter

[dragove]

Steps to Reproduce

mkdir mypojo && cd mypojo
rye init
rye add torch
rye sync
rye shell
rye add torch-scatter
rye sync

Expected Result

sync successfully

Actual Result

ModuleNotFoundError: No module named 'torch'

Version Info

rye 0.13.0
commit: 0.13.0 (8e01123 2023-08-29)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true

Stacktrace

Reusing already existing virtualenv
Generating production lockfile: /home/dove/Workspace/Python/mypojo/mypojo/requirements.lock
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/tmp/pip-resolve-y85p18ca/torch-scatter_4648624828ad49669686af72ba151a62/setup.py", line 8, in <module>
        import torch
    ModuleNotFoundError: No module named 'torch'
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

Traceback (most recent call last):
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 64, in generate_metadata
call_subprocess(
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
raise error
pip._internal.exceptions.InstallationSubprocessError: python setup.py egg_info exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/dove/.rye/pip-tools/[email protected]/bin/pip-compile", line 8, in
sys.exit(cli())
^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/piptools/scripts/compile.py", line 592, in cli
results = resolver.resolve(max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/piptools/resolver.py", line 593, in resolve
is_resolved = self._do_resolve(
^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/piptools/resolver.py", line 625, in _do_resolve
resolver.resolve(
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
result = self._result = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in bool
return bool(self._sequence)
^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool
return any(self)
^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in
return (c for c in iterator if id(c) not in self._incompatible_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in init
super().init(
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in init
self.dist = self._prepare()
^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 653, in _prepare_linked_requirement
dist = _get_prepared_distribution(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 568, in prepare_metadata
self.metadata_directory = generate_metadata_legacy(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dove/.rye/pip-tools/[email protected]/lib/python3.11/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 71, in generate_metadata
raise MetadataGenerationFailed(package_details=details) from error
pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
error: could not write production lockfile for project

Caused by:
failed to generate lockfile

[mitsuhiko]

It looks like pytorch-scatter requires torch as build dependency but does not actually declare it. My guess would be that this is a bug in pytorch-scatter. I'm not sure what a good solution is here. Kinda of curious how people use this with poetry or other package managers that should have the same issue.

[dragove]

Thanks for your reply. I find the import torch is in torch-scatter's setup.py but not in install_requires in setuptools' setup function.

In this situation, I wanna if I can use python in current virtual env to build torch-scatter. (That's why I enter rye shell to run rye sync)

[mitsuhiko]

Entering the shell won’t work. Builds are happening in an isolated environment.

[mitsuhiko]

This would need fixing in the upstream package. See rusty1s/pytorch_scatter#265

[dragove]

Thanks for pointing out that issue. I'll track that one.