Use pip-tools (GSI 334) #160
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a print statement for each file being updated (as it takes a while) Include all-extras flag only if optional-deps is present in TOML file
Remove the setup files from that list too
Prevent redundant package dep resolution that may result in errors
Reqs should be installed exactly as specified in lock file
Fix error message capture/output if command fails
Make src dir next to TOML to satisfy build system
Create dev lock file before production lock file
KerstenBreuer
suggested changes
Sep 19, 2023
Cito
previously approved these changes
Sep 19, 2023
Add setup.cfg/.py to deprecated files Use no-deps in package install command in dev_install Use ~= on production deps < 1.0.0 in pyproject.toml Remove extra newlines in pyproject.toml Remove old comments from requirements-dev-common.in
Pull Request Test Coverage Report for Build 6247490190
💛 - Coveralls |
Require the lock files (mandatory) Remove requirements.txt from static files
Cito
approved these changes
Sep 20, 2023
KerstenBreuer
approved these changes
Sep 20, 2023
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Up until now, we have managed our project dependencies in a semi-loose fashion where we only pinned top-level dependencies and allowed pip to resolve everything else as it saw fit. This mostly worked okay but we really wanted a way to lock down dependencies to ensure reproducibility and avoid the it works on my machine problem. We evaluated a few options, including Poetry, PDM, and pip-tools, and ultimately decided on the latter because it meshed well with what we already had, allowed us to switch to pyproject.toml from setup.cfg, is still actively maintained and relatively popular.
Changes
Dependencies
The line in
dev_installthat installs dependencies now includes--no-depsand installs exclusively fromrequirements-dev.txt.The rationale for this is described in this PR.
Top-level dependencies have been moved as follows:
setup.cfg->pyproject.tomlrequirements-dev.txt->requirements-dev.inrequirements-dev-common.in->requirements-dev-common.inGithub Workflows
New
update_lock.pyin /scripts/ (originally written by @KerstenBreuer)requirements.txtandrequirements-dev.txt(the lock files).scripts/update_lock.py [--upgrade]--upgradewill tell pip-compile to reevaluate dependencies that are already in the lock file (if it exists).--checkwill compare the current and would-be files to see if an update is needed.pyproject.toml(PEP 621) is where project information and dependencies are defined. It replacessetup.cfg.requirements-dev.txt: lock file that contains all pinned dependencies for the repository, both top-level and transitive, and their hashes.requirements.txt: lock file containing the production-only dependencies. This is a subset ofrequirements-dev.txt.