Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.12 still failing with error similar to #291 #300

Closed
sroet opened this issue Nov 11, 2024 · 12 comments · May be fixed by #304
Closed

v1.12 still failing with error similar to #291 #300

sroet opened this issue Nov 11, 2024 · 12 comments · May be fixed by #304

Comments

@sroet
Copy link

sroet commented Nov 11, 2024

Dear developer,

Since the release of 1.12, our (self-hosted, rootless docker) workflow started failing with:
/opt/conda/bin/python3: can't open file '/home/githubrunner/actions-runner/_work/_actions/pypa/gh-action-pypi-publish/release/v1.12/create-docker-action.py': [Errno 2] No such file or directory

Which seems similar to #291 (the error), but wasn't fixed with the release of 1.12.2

Here's the failing workflow run
https://github.com/SBC-Utrecht/pytom-match-pick/actions/runs/11778522196/job/32805113391#step:7:58

Please let me know if we can help tracking down this issue.

xref: SBC-Utrecht/pytom-match-pick#236

@sroet sroet changed the title v1.12 still failing with error similar to 291 v1.12 still failing with error similar to #291 Nov 11, 2024
@rjdbcm
Copy link

rjdbcm commented Nov 14, 2024

Confirm 1.12.2 still fails for my nested composite action.

@webknjaz
Copy link
Member

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

Also, /opt/conda/bin/python3 is untypical for usual GH Runners too.

@ferhatys
Copy link

I'm getting the same issue for some reason.

Run pypa/gh-action-pypi-publish@release/v1
Run # Reset path if needed
Run # Set repo and ref from which to run Docker container action
Run # 🔎 Discover pre-installed Python
python-path=/usr/local/bin/python3
Run # Create Docker container action
  # Create Docker container action
  /usr/local/bin/python3 '/home/runner/work/_actions/pypa/gh-action-pypi-publish/release/v1/create-docker-action.py'
  shell: bash --noprofile --norc -e -o pipefail {0}
  env:
    REF: release/v1
    REPO: pypa/gh-action-pypi-publish
    REPO_ID: 609775869
/usr/local/bin/python3: can't open file '/home/runner/work/_actions/pypa/gh-action-pypi-publish/release/v1/create-docker-action.py': [Errno 2] No such file or directory
Error: Process completed with exit code 2.

@ferhatys
Copy link

Confirmed release/v1.11 is working for me as a workaround.

@sroet
Copy link
Author

sroet commented Nov 15, 2024

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

Also, /opt/conda/bin/python3 is untypical for usual GH Runners too.

Hey @webknjaz, thanks for looking into this!

To repeat: we are running this on self-hosted runners inside a rootless docker, using the continuumio/miniconda3 image.
That image has the default conda installation in /opt/conda, this is also where the /opt/conda/bin/python3 comes from.

Now for /home/githubrunner/actions-runner/_work/_actions/:

That directory exists on the host system of the runner, but looking at this line of the setup, specifically:

-v "/home/githubrunner/actions-runner/_work/_actions":"/__w/_actions"

it seems to get mounted at /__w/_actions instead.

I don't know enough about creating actions to know if this is a bug in the Runner software or if these variables that are not meant to be used inside actions after the container creation

@webknjaz
Copy link
Member

Yeah, I understand, but can you check what's actually on dist in the runner host and within container (just add some recursive ls or something at the beginning of the job).

@sroet
Copy link
Author

sroet commented Nov 15, 2024

@webknjaz, started a debug workflow in SBC-Utrecht/pytom-match-pick#241. Should we move any back and forth discussion about what folders / variables you want to check to there?

@nikaro
Copy link

nikaro commented Nov 18, 2024

@sroet have you tried checking the contents of /home/githubrunner/actions-runner/_work/_actions/? That path is constructed using ${{ github.action_path }} which is supposed to contain a checked out copy of the action, which in turn would have create-docker-action.py in it. And since it's not there, I'd assume a bug in the GitHub Runner software.

@webknjaz this is a known issue, the workaround seems to be using $GITHUB_ACTION_PATH instead of ${{ github.action_path }}.

nikaro added a commit to nikaro/sopsy that referenced this issue Nov 18, 2024
nikaro added a commit to nikaro/sopsy that referenced this issue Nov 18, 2024
nikaro added a commit to nikaro/sopsy that referenced this issue Nov 18, 2024
@sroet
Copy link
Author

sroet commented Nov 19, 2024

@webknjaz, started a debug workflow in SBC-Utrecht/pytom-match-pick#241. Should we move any back and forth discussion about what folders / variables you want to check to there?

To report back to here;
After some debugging we indeed ran into the issue mentioned by @nikaro and I opened #304 to implement the proposed solution.

For my project however, I decided to restructure my release workflow to follow to current best practices around not having the id-token: write at build time and also sidestep the container issue. In case anyone else is interested this is the current workflow file.

In general; the workflow now has 4 jobs each depending on the previous one to succeed, with the type of runner in []:

  1. [self-hosted] build the package and test with twine check, upload artifact
  2. [github] download artifact and upload the distribution to testPyPI
  3. [self-hosted] pull the latest version from testPyPI and run the test suite
  4. [github] download artifact and upload the distribution to PyPI

I am fine closing this issue, but maybe it is nice to keep open until a decision has been made on #304

@virtuald
Copy link
Contributor

This is broken on github actions runners that use containers. Using release/v1.11 as a workaround.

@webknjaz
Copy link
Member

webknjaz commented Dec 7, 2024

  1. [self-hosted] pull the latest version from testPyPI and run the test suite

@sroet I wanted to comment on this point. This is not directly related to the action, but I have an opinion to share. Relying on TestPyPI may be dangerous since the project owners are not the same as on PyPI — it's easy to fall a victim of a dependency confusion / poisoning when somebody registers a project with a name of one of your transitive deps. Another bit is that due to caching in PyPI's CDN, the newly published dist might not be available immediately, and it may take 10 minutes for the resolver to start “seeing” it.
For these reasons, I recommend just downloading the same dists from the GHA artifact and performing the tests with that. You don't really need to test that pip is able to talk to some other index than PyPI which is mostly what this workflow implies.
In my most sophisticated workflows I have this structure where the very first job builds the dists, they are stored and then the tests run against them and only after that, there's conditional bits for publishing and signing. There's really no need to test how well an installer talks to an index. The thing to focus on is testing your project itself.
Here's an example: https://github.com/ansible/awx-plugins/blob/e7d5733/.github/workflows/ci-cd.yml. I even made an action to substitute git clone with testing from an sdist that I've been using in a few places, in almost all the jobs that follow the build one: https://github.com/marketplace/actions/checkout-python-sdist.

@webknjaz
Copy link
Member

So, I've recorded this as unsupported in README: https://github.com/marketplace/actions/pypi-publish#Non-goals. I'm going to close this issue with understanding that the PR remains open in case @sroet gets to it, and we'll decide whether to merge it separately.

@webknjaz webknjaz closed this as not planned Won't fix, can't repro, duplicate, stale Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants