Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support uv package manager #1339

Open
falkaer opened this issue Oct 17, 2024 · 0 comments
Open

Support uv package manager #1339

falkaer opened this issue Oct 17, 2024 · 0 comments

Comments

@falkaer
Copy link

falkaer commented Oct 17, 2024

Proposal Summary

It would be nice to have the option of using uv to manage dependencies instead of pip/conda/poetry.

The simplest support would be to just use uv pip as a drop-in replacement for pip when installing from a task's requirements in the clearml agent.

Motivation

Much faster environment build times on slow file systems.

On my university cluster all storage on most GPU nodes is network-attached with high throughput but very slow random access, and building my environment using pip hits a massive performance cliff and can easily take over an hour to complete even with all downloads already cached, compared to uv which can do this in about a minute.

I've resorted to handrolling my own clearml agent replacement for now because environment building takes so long with pip (and environments are rebuilt from scratch with every change)

Related Discussion

If this continues a slack thread, please provide a link to the original slack thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant