Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace poetry with uv #19075

Merged
merged 1 commit into from
Oct 31, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions .github/workflows/dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
- name: Update dependencies
run: |
python -m venv .venv
make update-dependencies
run: make update-dependencies
- name: Create pull request
uses: peter-evans/create-pull-request@v6
with:
Expand Down
12 changes: 7 additions & 5 deletions doc/source/admin/framework_dependencies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ Adding additional Galaxy dependencies
-------------------------------------

New packages can be added to Galaxy, or the versions of existing packages can be
updated, using `poetry`_ and `Wheelforge`_.
updated, using `uv`_ and `Wheelforge`_.

If wheels already exist on PyPI for all platforms and Python versions supported
by Galaxy, you can skip to step 3 in the process below.
Expand All @@ -195,11 +195,13 @@ by Galaxy, you can skip to step 3 in the process below.

2. Submit a pull request to `Wheelforge`_.

3. Add the new dependency to the `[tool.poetry.dependencies]` (or to `[tool.poetry.group.dev.dependencies]` if only needed for Galaxy development) section of `pyproject.toml` .
4. Run `make update-dependencies` to update the requirements file in `lib/galaxy/dependencies`_.
3. Add the new dependency to the ``dependencies`` list in the ``[project]``
section (or to ``dev`` list in the ``[dependency-groups]`` section if only
needed for Galaxy development) of `pyproject.toml`_ .
4. Run ``make update-dependencies`` to update the requirements files in `lib/galaxy/dependencies`_.
5. Submit a pull request to Galaxy with your changes.

.. _poetry: https://python-poetry.org/
.. _uv: https://docs.astral.sh/uv/
.. _Wheelforge: https://github.com/galaxyproject/wheelforge/
.. _Pull Request #4891: https://github.com/galaxyproject/galaxy/pull/4891
.. _pyproject.toml: https://github.com/galaxyproject/galaxy/blob/dev/pyproject.toml
.. _lib/galaxy/dependencies: https://github.com/galaxyproject/galaxy/tree/dev/lib/galaxy/dependencies
376 changes: 188 additions & 188 deletions lib/galaxy/dependencies/dev-requirements.txt

Large diffs are not rendered by default.

434 changes: 216 additions & 218 deletions lib/galaxy/dependencies/pinned-requirements.txt

Large diffs are not rendered by default.

62 changes: 32 additions & 30 deletions lib/galaxy/dependencies/pinned-typecheck-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,30 +1,32 @@
annotated-types==0.7.0 ; python_version >= "3.8" and python_version < "3.13"
boto3-stubs[s3]==1.35.49 ; python_version >= "3.8" and python_version < "3.13"
botocore-stubs==1.35.49 ; python_version >= "3.8" and python_version < "3.13"
cffi==1.17.1 ; python_version >= "3.8" and python_version < "3.13" and platform_python_implementation != "PyPy"
cryptography==43.0.3 ; python_version >= "3.8" and python_version < "3.13"
lxml-stubs==0.5.1 ; python_version >= "3.8" and python_version < "3.13"
mypy-boto3-s3==1.35.46 ; python_version >= "3.8" and python_version < "3.13"
mypy-extensions==1.0.0 ; python_version >= "3.8" and python_version < "3.13"
mypy==1.13.0 ; python_version >= "3.8" and python_version < "3.13"
pycparser==2.22 ; python_version >= "3.8" and python_version < "3.13" and platform_python_implementation != "PyPy"
pydantic-core==2.23.4 ; python_version >= "3.8" and python_version < "3.13"
pydantic==2.9.2 ; python_version >= "3.8" and python_version < "3.13"
tomli==2.0.2 ; python_version >= "3.8" and python_version < "3.11"
types-awscrt==0.23.0 ; python_version >= "3.8" and python_version < "3.13"
types-bleach==6.1.0.20240331 ; python_version >= "3.8" and python_version < "3.13"
types-boto==2.49.18.20241019 ; python_version >= "3.8" and python_version < "3.13"
types-contextvars==2.4.7.3 ; python_version >= "3.8" and python_version < "3.13"
types-dataclasses==0.6.6 ; python_version >= "3.8" and python_version < "3.13"
types-docutils==0.21.0.20241005 ; python_version >= "3.8" and python_version < "3.13"
types-html5lib==1.1.11.20241018 ; python_version >= "3.8" and python_version < "3.13"
types-markdown==3.7.0.20240822 ; python_version >= "3.8" and python_version < "3.13"
types-paramiko==3.5.0.20240928 ; python_version >= "3.8" and python_version < "3.13"
types-python-dateutil==2.9.0.20241003 ; python_version >= "3.8" and python_version < "3.13"
types-pyyaml==6.0.12.20240917 ; python_version >= "3.8" and python_version < "3.13"
types-requests==2.31.0.6 ; python_version >= "3.8" and python_version < "3.13"
types-s3transfer==0.10.3 ; python_version >= "3.8" and python_version < "3.13"
types-setuptools==75.2.0.20241025 ; python_version >= "3.8" and python_version < "3.13"
types-six==1.16.21.20241009 ; python_version >= "3.8" and python_version < "3.13"
types-urllib3==1.26.25.14 ; python_version >= "3.8" and python_version < "3.13"
typing-extensions==4.12.2 ; python_version >= "3.8" and python_version < "3.13"
# This file was autogenerated by uv via the following command:
# uv export --no-hashes --only-group=typecheck
annotated-types==0.7.0
boto3-stubs==1.35.49
botocore-stubs==1.35.49
cffi==1.17.1 ; platform_python_implementation != 'PyPy'
cryptography==43.0.3
lxml-stubs==0.5.1
mypy==1.13.0
mypy-boto3-s3==1.35.46
mypy-extensions==1.0.0
pycparser==2.22 ; platform_python_implementation != 'PyPy'
pydantic==2.9.2
pydantic-core==2.23.4
tomli==2.0.2 ; python_full_version < '3.11'
types-awscrt==0.23.0
types-bleach==6.1.0.20240331
types-boto==2.49.18.20241019
types-contextvars==2.4.7.3
types-dataclasses==0.6.6
types-docutils==0.21.0.20241005
types-html5lib==1.1.11.20241018
types-markdown==3.7.0.20240822
types-paramiko==3.5.0.20240928
types-python-dateutil==2.9.0.20241003
types-pyyaml==6.0.12.20240917
types-requests==2.31.0.6
types-s3transfer==0.10.3
types-setuptools==75.2.0.20241025
types-six==1.16.21.20241009
types-urllib3==1.26.25.14
typing-extensions==4.12.2
92 changes: 22 additions & 70 deletions lib/galaxy/dependencies/update.sh
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@
#!/bin/sh

# This script installs poetry if it's not already installed in the user account.
# It then runs poetry inside Galaxy's root directory, potentially updating
# pyproject.toml and poetry.lock, and the pinned and dev requirements files.
# This script installs uv if it's not already installed in the user account.
# It then runs uv inside Galaxy's root directory, potentially updating
# pyproject.toml and uv.lock, and the pinned requirements files.

set -e

SUPPORTED_PYTHON_VERSIONS="3.8 3.9 3.10 3.11 3.12"
NOT_SUPPORTED_NEXT_PYTHON_VERSION="3.13"

this_directory="$(cd "$(dirname "$0")" > /dev/null && pwd)"

usage() {
printf "Usage: %s: [-a] [pkg_spec...]\n" "${0##*/}" >&2
printf "Usage: %s: [-p pkg]\n" "${0##*/}" >&2
}

add=
while getopts 'a' OPTION
pkg=
while getopts 'p:h' OPTION
do
case $OPTION in
a) add=1
p) pkg="$OPTARG"
;;
h) usage
exit 0
;;
?) usage
exit 2
Expand All @@ -28,68 +28,20 @@ do
done
shift $((OPTIND - 1))

if [ -n "$add" ] && [ $# -eq 0 ]; then
printf "When adding (-a), you must provide at least one package specification.\n" >&2
usage
exit 2
fi

# Install the latest version of poetry into the user account
curl -sSL https://install.python-poetry.org | python3 -
poetry self add 'poetry-plugin-export!=1.7.0' # https://github.com/python-poetry/poetry-plugin-export/issues/269
# Create a virtual environment in a tmp directory and install uv into it
uv_venv=$(mktemp -d "${TMPDIR:-/tmp}/uv_venv.XXXXXXXXXX")
python3 -m venv "${uv_venv}"
"${uv_venv}/bin/python" -m pip install uv
uv="${uv_venv}/bin/uv"

# Run poetry (this may update pyproject.toml and poetry.lock).
if [ -z "$add" ]; then
poetry update -vv --lock "$@"
# Run uv (this may update pyproject.toml and uv.lock).
if [ -n "$pkg" ]; then
${uv} lock --upgrade-package "$pkg"
else
poetry add -vv --lock "$@"
${uv} lock
fi

# Update pinned requirements files.
PINNED_REQUIREMENTS_FILE=$this_directory/pinned-requirements.txt
PINNED_DEV_REQUIREMENTS_FILE=$this_directory/dev-requirements.txt
poetry export -f requirements.txt --without-hashes --output "$PINNED_REQUIREMENTS_FILE"
poetry export --only dev -f requirements.txt --without-hashes --output "$PINNED_DEV_REQUIREMENTS_FILE"
poetry export --only typecheck -f requirements.txt --without-hashes --output "$this_directory/pinned-typecheck-requirements.txt"

# Fix requirements

latest_package_version_for_python_version () {
PIP_INDEX_OUT=$(pip index versions --python-version "$2" "$1")
echo "$PIP_INDEX_OUT" | head -n 1 | sed -e 's/.* (\(.*\)).*/\1/'
}

split_requirement () {
PACKAGE_NAME=$1

if ! pip index version --help >/dev/null; then
echo "pip >= 21.2.1 required for the 'index' command"
exit 1
fi

NEW_REQUIREMENTS=
PREVIOUS_LATEST_PACKAGE_VERSION=
for PYTHON_VERSION in $SUPPORTED_PYTHON_VERSIONS; do
LATEST_PACKAGE_VERSION=$(latest_package_version_for_python_version "$PACKAGE_NAME" "$PYTHON_VERSION")
if [ -z "$NEW_REQUIREMENTS" ]; then
NEW_REQUIREMENTS="$PACKAGE_NAME==$LATEST_PACKAGE_VERSION ; python_version >= \"$PYTHON_VERSION\""
elif [ "$LATEST_PACKAGE_VERSION" != "$PREVIOUS_LATEST_PACKAGE_VERSION" ]; then
NEW_REQUIREMENTS="$NEW_REQUIREMENTS and python_version < \"$PYTHON_VERSION\"\n$PACKAGE_NAME==$LATEST_PACKAGE_VERSION ; python_version >= \"$PYTHON_VERSION\""
fi
PREVIOUS_LATEST_PACKAGE_VERSION=$LATEST_PACKAGE_VERSION
done
if [ -n "$NOT_SUPPORTED_NEXT_PYTHON_VERSION" ]; then
NEW_REQUIREMENTS="$NEW_REQUIREMENTS and python_version < \"$NOT_SUPPORTED_NEXT_PYTHON_VERSION\""
fi

sed -i.orig -e "s/^$PACKAGE_NAME==.*/$NEW_REQUIREMENTS/" "$PINNED_REQUIREMENTS_FILE" "$PINNED_DEV_REQUIREMENTS_FILE"
}

# For some packages there is no recent version that works on all Python versions
# supported by Galaxy, so Poetry resorts to an old version. Here we replace any
# such requirement with multiple Python-version-specific requirements.
split_requirement contourpy
split_requirement h5py
split_requirement matplotlib
split_requirement numpy
split_requirement scipy
${uv} export --no-hashes --no-dev > "$this_directory/pinned-requirements.txt"
${uv} export --no-hashes --only-group=dev > "$this_directory/dev-requirements.txt"
${uv} export --no-hashes --only-group=typecheck > "$this_directory/pinned-typecheck-requirements.txt"
Loading
Loading