Shared reusable workflows for GitHub Actions.
We currently have 4 reusable workflows (i.e. run-terraform and run-security-scans (DEPRECATED)) available for use.
See Ideal Use of Workflows for an example of how to optimally use all 3 workflows together.
See Tips and Tricks for supporting information regarding usage of the reusable workflows.
Allows running kubectl commands against a Kubernetes cluster. This is useful for doing restarts of deployments for example.
- Connects to a google cluster as a deploy service account
- Will always use connect gateway
- Runs specified kubectl commands against the cluster
- Your gcp project is set up and given required permissions in skip-core-infrastructure and gcp-service-accounts
Example usage in .github/workflows/auto-merge.yml:
name: Restart deployment
on: pull_request_target
jobs:
sandbox:
name: restart-app
uses: kartverket/github-workflows/.github/workflows/run-kubectl.yaml@latest
with:
cluster_name: atkv1-dev
service_account: [email protected]
kubernetes_project_id: kube-dev-4329023
kubernetes_project_number: 43290432893
namespace: default
command: |
restart deployment my-deployment| Key | Type | Required | Description |
|---|---|---|---|
| cluster_name | string | X | Cluster name. Found with gcloud container fleet memberships list |
| service_account | string | X | The projects deploy service account in full format. |
| kubernetes_project_id | string | X | The kubernetes GCP project id. |
| project_number | string | X | A 12-digit number used as a unique identifier for the product project. |
| namespace | string | X | which namespace to execute the command in |
| kubectl_version | string | X | which kubectl version to use. format: v1.30.0. latest stable is default |
| commands | multiline string | X | The kubectl commands you want to run, exclude kubectl. example: https://skip.kartverket.no/docs/github-actions/kubectl-fra-github |
Allows auto-merging dependabot PRs that match given patterns. Useful when you are drowning in PRs and have built up trust in a set of dependencies that release often and never break. It's recommended to have a sane CI setup so that anything merged to main at least passes CI tests before going into prod
- Allows configuring a set of dependencies in a configfile that can be merged
- Each dependency will allow either major, minor or patch updates (only supports semver)
- A bot approves and merges the PR
A few requirements are necessary in order to make this work in addition to the example below.
- Legacy branch protection rules are not supported. Your repo needs to use the more modern branch rulesets
- The Octo STS app needs to be added to the rulesets bypass list so that it can merge the PR
- A trust file called
.github/chainguard/auto-update.sts.yamlneeds to exist to allow the workflow to get a valid GitHub token
Example usage in .github/workflows/auto-merge.yml:
name: Dependabot auto-merge
on: pull_request_target
jobs:
auto-merge-dependabot:
permissions:
id-token: write
contents: write
pull-requests: write
uses: kartverket/github-workflows/.github/workflows/auto-merge-dependabot.yml@<release tag>Example configfile in .github/auto-update.json:
[{
"match": {
"dependency_name": "hashicorp/google",
"update_type": "semver:minor"
}
}, {
"match": {
"dependency_name": "hashicorp/google-beta",
"update_type": "semver:minor"
}
}]Example STS trust file in .github/chainguard/auto-update.sts.yaml:
issuer: https://token.actions.githubusercontent.com
subject: repo:kartverket/gcp-service-accounts:pull_request
permissions:
contents: write
pull_requests: writeThe configfile is currently the only input. The configfile at .github/auto-merge.json supports the following values:
| Key | Type | Required | Description |
|---|---|---|---|
[].match.dependency_name |
string | true | The name of the dependency as it appears on the Dependabot PR |
[].match.update_type |
string | true | Which changes should be merged. Currently supports semver:patch, semver:minor and semver:major. The type includes all lower tiers, for example semver:minor includes patch changes |
This workflow plans and applies Terraform config to deploy to an environment.
- Logs in to GCP and Kubernetes automatically with Workload Identity Federation
- Posts comments with relevant information to Pull Requests
- Runs validate, format and plan to verify deployment follows conventions before running apply
- Posts summaries on relevant steps on the Action pipeline view to improve visibility
- Supports manual approval and delay periods in deploy using GitHub Environments protection rules
- Re-uses plan from plan step in apply step to ensure apply always executes the diff generated by plan no matter when it is run
- Prevents deploys running in parallel against the same environment crashing due to failing to acquire state lock
- Allows for the choice of deploying and/or destroying terraform config
- Will only deploy on push or workflow_dispatch event to main by default. Can be configured to deploy on a different branch using the
deploy_oninput. - Logs into tailscale if using the postgresql provider
jobs:
build:
# Builds an image of the format <registry>/<repository>:<tag> or <registry>/<repository>@<digest>
# and pushes it to the github registry.
# See 'All Workflows Together' section for an example build job.
dev:
needs: [build]
name: Deploy to dev
permissions:
# For logging on to Vault, GCP
id-token: write
# For writing comments on PR
pull-requests: write
# For fetching git repo
contents: read
# For accessing repository
packages: write
uses: kartverket/github-workflows/.github/workflows/run-terraform.yml@<release tag>
secrets: inherit # Optional, but required if you need to use tailscale
with:
runner: ubuntu-latest
environment: dev
kubernetes_cluster: atkv1-dev
terraform_workspace: dev
terraform_option_1: -var-file=dev.tfvars
terraform_option_2: -var=image=${{ needs.build.outputs.image_url}}
terraform_init_option_1: -backend-config=dev.gcs.tfbackend
working_directory: terraform
auth_project_number: "123456789123"
service_account: [email protected]
project_id: project-dev-123
destroy: <optional boolean>
unlock: <optional LOCK_ID>
test:
needs: [build, dev]
# approximately the same as 'dev' but for the test environment
prod:
needs: [build, dev, test]
# approximately the same as 'dev' but for the prod environment| Key | Type | Required | Description |
|---|---|---|---|
| service_account | string | X | The GCP service account connected to the identity pool that will be used by Terraform. Service account and auth_project_number environment must coincide. |
| auth_project_number | string | X | The GCP Project Number used as the active project. A 12-digit number used as a unique identifier for the project. Used to find workload identity pool. Project number and SA environment must coincide |
| workload_identity_provider_override | string | The ID of the provider to use for authentication. Only used for overriding the default workload identity provider based on project number. It should be in the format of projects/{{project_number}}/locations/global/workloadIdentityPools/{{workload_identity_pool_id}}/providers/{{workload_identity_pool_provider_id}}. |
|
| runner | string | X | The GitHub runner to use when running the deploy. This can for example be ubuntu-latest. |
| deploy_on | string | Which branch will be the only branch allowed to deploy. This defaults to the main branch so that other branches only run check and plan. Defaults to refs/heads/main. |
|
| working_directory | string | The directory in which to run terraform, i.e. where the Terraform files are placed. The path is relative to the root of the repository. | |
| project_id | string | The GCP Project ID to use as the "active project" when running Terraform. When deploying to Kubernetes, this must match the project in which the Kubernetes cluster is registered. | |
| kubernetes_cluster | string | An optional kubernetes cluster to authenticate to. Note that the project_id must match where the cluster is registered. | |
| environment | string | The GitHub environment to use when deploying. See using environments for deployment for more info on this. | |
| terraform_workspace | string | When provided will set a workspace as the active workspace when planning and deploying. | |
| terraform_option_X | string | An additional terraform option to be passed to plan and apply. For example -var-file=dev.tfvars and -var=<variableName>=<variableValue>. X may be an integer between 1-3, which allows at most 3 options. |
|
| terraform_init_option_Y | string | An additional config to be passed to terraform init. For example -backend-config=dev.gcs.tfbackend. Y may be an integer between 1-3, which allows at most 3 init options. |
|
| add_comment_on_pr | boolean | Setting this to false disables the creation of comments with info of the Terraform run on Pull Requests. When true the pull-request permission is required to be set to write. Defaults to true. |
|
| destroy | boolean | An optional boolean that determines whether terraform will be destroyed. Defaults to 'false'. | |
| unlock | string | An optional string which runs terraform force-unlock on the provided LOCK_ID, if set. |
If the cyrilgdn/postgresql provider is present, the secrets: inherit input is required to use the tailscale provider.
The provider will set the environment variable NEED_TAILSCALE to true, which will trigger the tailscale login.
As long as your repository is internal, the tailscale secrets should be present on your repository.
DEPRECATED: THIS WORKFLOW IS DEPRECATED AND WILL NOT RECEIVE FURTHER UPDATES. PLEASE CHECK OUT Pharos FOR SECURITY SCANS
This workflow runs security scans on a repository. Note, in order to not limit/interfere with the development process, the scans do not run on draft pull requests. Additionally, if image_url is not supplied Trivy scan will not be performed (i.e. only TFSec scan will run).
- Runs TFSec, a static analysis security scanner for your Terraform code. Does not run on draft pull requests.
- Runs Trivy, a comprehensive security scanner. Does not run on draft pull requests.
- Calls the GitHub Security Code Scanning API and fails workflow if there are any high or critical errors.
- The above step can be configured by specifying the
allow_severity_levelinput parameter.
- Code Scanning must be appropriately set up in the Github Security tab.
- Note that the image built during your build-job must be pushed to the registry on all but draft PRs for the workflow to work as intended (see example build job below, paying extra attention to lines following
# Note: ...)
jobs:
build:
# Builds an image of the format <registry>/<repository>:<tag> or <registry>/<repository>@<digest>
# and pushes it to the github registry. This is the image that must be used in all following jobs.
# See 'All Workflows Together' section for an example build job.
security-scans:
needs: [build]
name: Security Scans
permissions:
# For fetching git repo
contents: read
# For accessing repository
packages: write
# required for authentication to GCP
id-token: write
actions: read
security-events: write
uses: kartverket/github-workflows/.github/workflows/run-security-scans.yml@<release tag>
with:
auth_project_number: "123456789123"
service_account: [email protected]
image_url: <registry>/<repository>:<tag> or <registry>/<repository>@<digest> # the image created by build job
trivy: true # default true, optional
tfsec: true # default true, optional
allow_severity_level: medium
dev:
needs: [build]
# call to run-terraform.yml for dev environment with build image
test:
needs: [build, dev]
# call to run-terraform.yml for test environment with build image
prod:
needs: [build, dev, test, security-scans]
# call to run-terraform.yml for prod environment only after security-scans job, with build image| Key | Type | Required | Description |
|---|---|---|---|
| auth_project_number | string | X | The GCP Project Number used as the active project. A 12-digit number used as a unique identifier for the project. Used to find workload identity pool. This project should be your dev environment project, as this is the environment where the attestors are located |
| workload_identity_provider_override | string | The ID of the provider to use for authentication. Only used for overriding the default workload identity provider based on project number. It should be in the format of projects/{{project_number}}/locations/global/workloadIdentityPools/{{workload_identity_pool_id}}/providers/{{workload_identity_pool_provider_id}}. |
|
| service_account | string | X | The GCP service account connected to the identity pool that will be used by Terraform. Should be the dev environment deploy service account |
| image_url | string | The Docker image url must be of the form registry/repository:tag or registry/repository@digest for run-security-scans. It is not required; however, in order to run Trivy and aquire attestations an image_url must be supplied. |
|
| trivy | boolean | An optional boolean that determines whether trivy-scan will be run. Defaults to 'true'. | |
| tfsec | boolean | An optional boolean that determines whether tfsec-scan will be run. Defaults to 'true'. | |
| allow_severity_level | string | A string which determines the highest level of severity the security scans can find while still succeeding workflows. Only "medium", "high" and "critical" values are allowed. Note that these values are case sensitive. |
Below are examples of how to use the reusable workflows for different scenarios
The following is an example of how to use run-terraform and run-security-scans together to deploy to dev, test and prod environments.
Note the use of jobs.<job_id>.needs to specify the order in which jobs run.
For details on how to use jobs.<job_id>.outputs see Using outputs.
name: "Ideal" workflow
# The workflow runs when (1) there is a push to the 'main' branch or (2) there is a push to a pull request onto main branch
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
jobs:
build:
# Example of how to build an image and supply appropriate inputs to run-security-scans reusable workflow
# Note outputs derived in the setOutput step and 'tags' set in the meta step
name: Build Docker Image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
outputs:
image_url: ${{ steps.setOutput.outputs.image_url }}
image_tag_url: ${{ steps.setOutput.outputs.image_tag_url }}
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Sets tag 'latest' for images built on main/master branch and tag 'prebuild-temp' on all other image builds
- name: Set tag
id: set-tag
env:
BRANCH: ${{ github.ref_name }}
run: |
if [[ "$BRANCH" == "main" || "$BRANCH" == "master" ]]; then
echo "image_tag=latest" >> $GITHUB_OUTPUT
else
echo "image_tag=prebuild-temp" >> $GITHUB_OUTPUT
fi
# Login against a Docker registry except on draft-PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }}
if: github.event.pull_request.draft == false
uses: docker/login-action@<version>
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@<version>
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Note: checkout https://github.com/docker/metadata-action#tags-input for tag format options
tags: |
type=sha,format=long
type=raw,value=${{ steps.set-tag.outputs.image_tag }}
# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
id: build-docker
uses: docker/build-push-action@<version>
with:
context: .
# Note: The image must be pushed to registry in all cases except for draft PRs
push: ${{ !github.event.pull_request.draft }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Set output with build values
id: setOutput
# Note: The image url is output in both `registry/repository:tag` and `registry/repository@digest` formats
run: |
echo "image_url=${{ env.REGISTRY }}/${{ github.repository }}@${{ steps.build-docker.outputs.digest }}" >> $GITHUB_OUTPUT
echo "image_tag_url=${{ env.REGISTRY }}/${{ github.repository }}:${{ steps.meta.outputs.version }}" >> $GITHUB_OUTPUT
security-scans:
needs: [build]
name: Security Scans
permissions:
contents: read
packages: write
# required for authentication to GCP
id-token: write
actions: read
security-events: write
uses: kartverket/github-workflows/.github/workflows/run-security-scans.yml@<release tag>
with:
auth_project_number: "123456789123"
service_account: [email protected]
image_url: ${{ needs.build.outputs.image_tag_url}}
dev:
needs: [security-scans]
name: Deploy to dev
permissions:
# For logging on to Vault, GCP
id-token: write
# For writing comments on PR
pull-requests: write
# For fetching git repo
contents: read
# For accessing repository
packages: write
uses: kartverket/github-workflows/.github/workflows/run-terraform.yml@<release tag>
with:
runner: ubuntu-latest
environment: dev
kubernetes_cluster: atkv1-dev
terraform_workspace: dev
terraform_option_1: -var-file=dev.tfvars
terraform_option_2: -var=image=${{ needs.build.outputs.image_url}}
terraform_init_option_1: -backend-config=dev.gcs.tfbackend
working_directory: terraform
auth_project_number: "123456789123"
service_account: [email protected]
project_id: project-dev-123
test:
needs: [dev]
name: Deploy to test
permissions:
# For logging on to Vault, GCP
id-token: write
# For writing comments on PR
pull-requests: write
# For fetching git repo
contents: read
# For accessing repository
packages: write
uses: kartverket/github-workflows/.github/workflows/run-terraform.yml@<release tag>
with:
runner: ubuntu-latest
environment: test
kubernetes_cluster: atkv1-test
terraform_workspace: test
terraform_option_1: -var-file=test.tfvars
terraform_option_2: -var=image=${{ needs.build.outputs.image_url}}
terraform_init_option_1: -backend-config=test.gcs.tfbackend
working_directory: terraform
auth_project_number: "123456789123"
service_account: [email protected]
project_id: project-test-123
prod:
needs: [build, dev, test, security-scans]
name: Deploy to prod
permissions:
# For logging on to Vault, GCP
id-token: write
# For writing comments on PR
pull-requests: write
# For fetching git repo
contents: read
# For accessing repository
packages: write
uses: kartverket/github-workflows/.github/workflows/run-terraform.yml@<release tag>
with:
runner: ubuntu-latest
environment: prod
kubernetes_cluster: atkv1-prod
terraform_workspace: prod
terraform_option_1: -var-file=prod.tfvars
terraform_option_2: -var=image=${{ needs.build.outputs.image_url}}
terraform_init_option_1: -backend-config=prod.gcs.tfbackend
working_directory: terraform
auth_project_number: "123456789123"
service_account: [email protected]
project_id: project-prod-123If you want to deploy a selected branch to the dev-environment on workflow dispatch.
You will need to take care not to interfere with others working in the same repo/environment.
Note that the deploy_on input is set to ${{ github.ref }} and that the only on specified is workflow_dispatch
name: Deploy to dev on workflow dispatch
on: workflow_dispatch
jobs:
build:
# Builds an image of the format <registry>/<repository>:<tag> or <registry>/<repository>@<digest>
# and pushes it to the github registry. This is the image that must be used in all following jobs.
# See 'Using outputs' section for suggestions on how to do this.
security-scans:
needs: [build]
# call to run-security-scans.yml with build image
dev:
name: Deploy to dev
needs: [security-scans]
permissions:
# For logging on to Vault, GCP
id-token: write
# For writing comments on PR
pull-requests: write
# For fetching git repo
contents: read
# For accessing repository
packages: write
uses: kartverket/github-workflows/.github/workflows/run-terraform.yml@<release tag>
with:
runner: ubuntu-latest
environment: dev
kubernetes_cluster: atkv1-dev
terraform_workspace: dev
terraform_option_1: -var-file=dev.tfvars -var=image=${{ needs.build.outputs.image_url}}
terraform_init_option_1: -backend-config=dev.gcs.tfbackend
working_directory: terraform
auth_project_number: "123456789123"
service_account: [email protected]
project_id: project-dev-123
deploy_on: ${{ github.ref }}Helpful "tips and tricks" for using the reusable workflows can be found below.
If you want jobs to run in a particular order:
Use
jobs.<job_id>.needsto identify any jobs that must complete successfully before this job will run. [2]
See Github Doc: jobs.<job_id>.needs
If you want to use outputs from one job in some following job:
You can use
jobs.<job_id>.outputsto create a map of outputs for a job. Job outputs are available to all downstream jobs that depend on this job. [1]
See Github Doc: Defining outputs for jobs for more information.
If you want to use environment variables, passing them to a reusable workflow is a little tricky as this behavior is not currently supported by GitHub. This is a requested feature. The current workaround for this is to use a setup-job to consume env vars and provide an output that can be mapped to the arguments of the job.
Click here to see an example of this
env:
AUTH_PROJECT_NUMBER: "123456789123"
SERVICE_ACCOUNT: [email protected]
PROJECT_ID: project-name-123
jobs:
setup-env:
runs-on: ubuntu-latest
outputs:
auth_project_number: ${{ steps.set-output.outputs.auth_project_number }}
service_account: ${{ steps.set-output.outputs.service_account }}
project_id: ${{ steps.set-output.outputs.project_id }}
steps:
- name: set outputs with default values
id: set-output
run: |
echo "auth_project_number=${{ env.AUTH_PROJECT_NUMBER }}" >> $GITHUB_OUTPUT
echo "service_account=${{ env.SERVICE_ACCOUNT }}" >> $GITHUB_OUTPUT
echo "project_id=${{ env.PROJECT_ID }}" >> $GITHUB_OUTPUT
dev:
name: Deploy to dev
needs: setup-env
permissions:
id-token: write
contents: read
pull-requests: write
packages: write
uses: kartverket/github-workflows/.github/workflows/[email protected]
with:
runner: ubuntu-latest
environment: dev
kubernetes_cluster: atkv1-dev
terraform_workspace: dev
terraform_option_1: -var-file=dev.tfvars -var=image=/: or /@
working_directory: terraform
auth_project_number: ${{ needs.setup-env.outputs.auth_project_number }}
service_account: ${{ needs.setup-env.outputs.service_account }}
project_id: ${{ needs.setup-env.outputs.project_id }}
Secrets should be administered through Google Secret Manager and thus there is no explicit
support for taking GitHub secrets into the action. GitHub will also stop you
from sending secrets as arguments using the with argument, so this will not
work either.
See TROUBLESHOOTING.md. If you experience and fix an issue that isn't mentioned there, feel free to add it.
Get in touch with SKIP if you have any contribution suggestions, and feel free to create a pull-request.