Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Build node images with cloud build #1746

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

lentzi90
Copy link
Contributor

What this PR does / why we need it:

Add cloud build configuration for building node images.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1502

Special notes for your reviewer:

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

  • squashed commits
  • if necessary:
    • includes documentation
    • adds unit tests

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 10, 2023
Copy link

netlify bot commented Nov 10, 2023

Deploy Preview for kubernetes-sigs-cluster-api-openstack ready!

Name Link
🔨 Latest commit 979fc28
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-cluster-api-openstack/deploys/65bca66dcd23360008846cf4
😎 Deploy Preview https://deploy-preview-1746--kubernetes-sigs-cluster-api-openstack.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 10, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lentzi90

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 10, 2023
@lentzi90
Copy link
Contributor Author

I did not manage to figure out how the builds are triggered. Any clue where to add a trigger for this? Or at least how to test it once?
@mdbooth @tormath1

@tormath1
Copy link
Contributor

@lentzi90 it has to be tested with a Google Cloud account: you need to enable cloudbuild and to add the repo / branch where the cloudbuild.yaml is present. (https://cloud.google.com/build/docs/overview)
I tested with my personal account back in the days - since you are a CAPO maintainer, you might have access to the Cluster API GCP account (or at least to some part) ?

@lentzi90
Copy link
Contributor Author

Yeah I can see the cloud console, I can see the existing builds, but they are not triggered from "triggers" in cloud build and it seems I don't have privileges to add any. My guess is that it is all managed externally somehow. Perhaps they are triggered from test-infra?

Co-authored-by: Mathieu Tortuyaux <[email protected]>
Co-authored-by: Matthew Booth <[email protected]>
Signed-off-by: Lennart Jern <[email protected]>
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 24, 2024
@lentzi90 lentzi90 force-pushed the lentzi90/osimage-cloudbuild branch 2 times, most recently from 9b9c117 to 08e7227 Compare January 24, 2024 13:45
@lentzi90 lentzi90 force-pushed the lentzi90/osimage-cloudbuild branch 2 times, most recently from 2ee34f2 to 21a7a2f Compare January 24, 2024 14:26
@lentzi90 lentzi90 force-pushed the lentzi90/osimage-cloudbuild branch 7 times, most recently from 39d7ca8 to d813e41 Compare January 26, 2024 09:05
@lentzi90
Copy link
Contributor Author

Any clue why make -C osimages -j2 ACCELERATOR=kvm \ osimage-flatcar-1.27.2 \ osimage-flatcar-1.28.2 \ osimage-ubuntu-2204-1.28.2 \ osimage-ubuntu-2204-1.27.2 becomes

 docker run --rm --privileged \
	--network=host \
	-v "/home/runner/work/cluster-api-provider-openstack/cluster-api-provider-openstack/_artifacts/osimages:/output" \
	-e PACKER_LOG=1 \
	-e PACKER_FLAGS=" \
	    --var 'boot_wait=300s' \
	    --var 'accelerator=kvm' \
	    --var 'kubernetes_semver=v1.27.2.o' \
	    --var 'kubernetes_series=v1.27.o' \

(note the .o in v1.27.2.0)
@tormath1

@tormath1
Copy link
Contributor

Any clue why make -C osimages -j2 ACCELERATOR=kvm \ osimage-flatcar-1.27.2 \ osimage-flatcar-1.28.2 \ osimage-ubuntu-2204-1.28.2 \ osimage-ubuntu-2204-1.27.2 becomes

 docker run --rm --privileged \
	--network=host \
	-v "/home/runner/work/cluster-api-provider-openstack/cluster-api-provider-openstack/_artifacts/osimages:/output" \
	-e PACKER_LOG=1 \
	-e PACKER_FLAGS=" \
	    --var 'boot_wait=300s' \
	    --var 'accelerator=kvm' \
	    --var 'kubernetes_semver=v1.27.2.o' \
	    --var 'kubernetes_series=v1.27.o' \

(note the .o in v1.27.2.0) @tormath1

@lentzi90 very strange. It seems to come from the \ - while I'm checking this you can try to use a single line:

make -j2 ACCELERATOR=none osimage-flatcar-1.27.2 osimage-flatcar-1.28.2 osimage-ubuntu-2204-1.28.2 osimage-ubuntu-2204-1.27.2

docker run --rm $(privileged) \
--network=host \
-v "$(OSIMAGE_DIR):/output" \
-e PACKER_LOG=1 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: Now it seems to work, we can remove this line that generates tones of logs.

@lentzi90
Copy link
Contributor Author

lentzi90 commented Jan 29, 2024

From what I can see, it managed to build both flatcar images, but then it timed out for ubuntu.
https://github.com/Nordix/cluster-api-provider-openstack/actions/runs/7693606423/job/20962692694#step:6:2425

@tormath1
Copy link
Contributor

From what I can see, it managed to build both flatcar images, but then it timed out for ubuntu. https://github.com/Nordix/cluster-api-provider-openstack/actions/runs/7693606423/job/20962692694#step:6:2425

@lentzi90 commands over VNC... We already faced this: https://kubernetes.slack.com/archives/CFKJB65G9/p1679317697144409?thread_ts=1679302801.869119&cid=CFKJB65G9
I suggested back in the days to use 20s for Ubuntu before typing commands 🤷

@lentzi90
Copy link
Contributor Author

I suggested back in the days to use 20s for Ubuntu before typing commands 🤷

Where/how would I change this?

--network=host \
-v "$(OSIMAGE_DIR):/output" \
-e PACKER_FLAGS=" \
--var 'boot_wait=300s' \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lentzi90 #1746 (comment) you can change the value here.

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 2, 2024
@mdbooth
Copy link
Contributor

mdbooth commented May 2, 2024

@lentzi90 @tormath1 this has fallen through the cracks. Do we want to resurrect this?

@lentzi90
Copy link
Contributor Author

lentzi90 commented May 3, 2024

I guess we do. My last attempt was to use GitHub actions. It looked promising at first, but kept failing from running out of disk space 🙁
The standard runners have only 14 GB disks. I guess that is just not enough.

@mdbooth
Copy link
Contributor

mdbooth commented May 3, 2024

Incidentally, I think I saw one of the infra folks mention that they didn't want anybody publishing container images from the staging repo because we get charged egress for it (but registry.k8s is ok). I'm guessing publishing OS images from our staging bucket would be even worse. The way we use them in our CI is presumably fine because they never leave GCP, but I suspect we should not routinely direct users to them for fear of blowing the credit card.

@dims Do I have that right?

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jun 2, 2024
@lentzi90
Copy link
Contributor Author

lentzi90 commented Jun 3, 2024

/remove-lifecycle rotten

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Jun 3, 2024
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 1, 2024
@EmilienM
Copy link
Contributor

EmilienM commented Sep 2, 2024

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build CAPO supported images
6 participants