Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pathos has strict dependencies #302

Open
dyollb opened this issue Dec 5, 2024 · 3 comments
Open

Pathos has strict dependencies #302

dyollb opened this issue Dec 5, 2024 · 3 comments

Comments

@dyollb
Copy link

dyollb commented Dec 5, 2024

I recently came across pathos as a dependency of another package. Adding pathos forced me to update many packages we already use to the latest version.

I see the dependencies were updated to the latest versions in a sweep here:
33e3f91

Enforcing those latest dependencies in a CI and your requirements.txt file, i.e. the "supported" dependency versions, is probably fine. However, I would argue that forcing these via the setup.py is a bit too strict. What I see in other popular packages is the opposite: "Only update strict minimum version requirements if necessary because of API or bugs"

@mmckerns
Copy link
Member

mmckerns commented Dec 6, 2024

I can appreciate what you are saying. In part why the requirements are so strict is that multiprocess is a simple fork of multiprocessing, and it only is guaranteed to be supported against the most recent minor release of python. Thus, whenever the multiprocessing code changes in the STL release, any prior release of multiprocess needs to be updated to the most recent version. Hence, pathos also has strict dependency requirements. I expect that the requirements with dill could be loosened a bit... but multiprocess would fundamentally need to change from being a fork of the current versions of multiprocessing in the current python releases to something that supports multiple recent versions.

@mmckerns
Copy link
Member

mmckerns commented Dec 6, 2024

Maybe moving the supported versions into requirements.txt is good. They are where they are because the requirements (as noted above) are fairly strict.

@dyollb
Copy link
Author

dyollb commented Dec 6, 2024

Thanks for the explanation. I now understand better.

If you feel there is room for decoupling the versions of dill, ppft, and pox from pathos that would be nice. Otherwise feel free to close the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants