Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pants CI: Run all unit and pack tests #6269

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Oct 24, 2024

This PR updates pants CI so pants+pytest will run all unit and pack tests in CI.
Other things in this PR:

  • adjusts the CI workflow to ensure we're testing with the version of python we expect.
  • splits the pants test CI into 3 jobs: pants-plugins tests, unit tests (with 4 shards of tests), and pack tests.
  • adds some hints to use ST2TESTS_REDIS_HOST and ST2TESTS_REDIS_PORT env vars to error/warning messages when coordination backend isn't configured or when pants-plugins/uses_services can't find a running redis service. I added this because I forgot about that, which made tests fail.
  • adds ensure_indexes = True on one st2api triggertypes test that was failing because the index wasn't in mongo. My best guess is that the test relied on another test to create the indexes.

This cherry-picks the last changes from #6202 which I will leave as-is for historical reference.

Closes #6202

@cognifloyd cognifloyd added this to the pants milestone Oct 24, 2024
@cognifloyd cognifloyd requested review from nzlosh, guzzijones and a team October 24, 2024 20:00
@cognifloyd cognifloyd self-assigned this Oct 24, 2024
@pull-request-size pull-request-size bot added the size/M PR that changes 30-99 lines. Good size to review. label Oct 24, 2024
@@ -8,6 +8,7 @@ python_sources()

python_tests(
name="tests",
tags=["unit"],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allowed me to run all unit tests without explicitly running the pylint_plugins tests.

Comment on lines 128 to 129
--tag=unit
test '::'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:: is a glob that means everything. Using --tag=unit filters everything down to only those test targets that are tagged with unit.

Comment on lines +139 to +140
--tag=pack
test '::'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:: is a glob that means everything. Using --tag=pack filters everything down to only those test targets that are tagged with pack.

Using the Merge OK jobs simplifies setting up the branch protection rules.
@pull-request-size pull-request-size bot added size/L PR that changes 100-499 lines. Requires some effort to review. and removed size/M PR that changes 30-99 lines. Good size to review. labels Oct 24, 2024
@cognifloyd
Copy link
Member Author

cognifloyd commented Oct 24, 2024

Dang. I don't know why the unit tests are not passing any more. Maybe I missed cherry-picking something. I will have to debug and fix the tests before this can be merged.
edit: Oh. I didn't set ST2TESTS_REDIS_HOST and ST2TESTS_REDIS_PORT. Oops.

Comment on lines +168 to +182
- name: Checkout repository
uses: actions/checkout@v4
with:
# a test uses a submodule, and pants needs access to it to calculate deps.
submodules: 'recursive'
# sadly, the submodule will only have fetch-depth=1, which is what we want
# for st2.git, but not for the submodules. We still want actions/checkout
# to do the initial checkout, however, so that it adds auth for fetching
# in the submodule.

- name: Fetch repository submodules
run: |
git submodule status
git submodule foreach 'git fetch --all --tags'
git submodule foreach 'git tag'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried turning this block into a local action, but checkout has to happen first, before using local actions, so that did not work.

There is hope for a less repetitive workflow, however. actions/runner#1182 says YAML anchor/alias support for GHA workflows is on the roadmap for 2025.

@cognifloyd cognifloyd requested a review from a team October 25, 2024 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure: ci/cd pantsbuild size/L PR that changes 100-499 lines. Requires some effort to review. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants