Skip to content

Automation for deployment of the Interim Data Facility on Google Cloud Platform.

License

Notifications You must be signed in to change notification settings

lsst/idf_deploy

Repository files navigation

Terraform GCP Foundation and Deployment

This code repo is intended to deploy a solid Google Cloud Platform foundation based off of Google's Cloud security foundation guide. This guide provides our opinionated security foundations blueprint and captures a step-by-step view of how to configure and deploy your Google Cloud estate. This document can provide a good reference and starting point because we highlight key topics to consider. In each topic, we provide background and discussion of why we made each of our choices.

If what is desired is a new environment parallel to the existing ones at the IDF, please consult the New Environment documentation.

Repo Structure

The .github workflows directory contains the build steps used when a pipeline is initiated. All of the pipelines are located in this directory.

The deployments directory is used as the main directory to place new applications. Each new application will have its own dedicated directory with a subdirectory with the different *.tfvars files for differences between different environments like dev,int, and stable. These *.tfvars files help differentiate between projects and supply the inputs for the different modules.

The modules directory is where the blueprints of the infrastructure are stored.

The runbook directory is used for documentation.

Where to Begin

To start, you will need to go into the foundation directory. This directoy is the building block to deploying a solid and secure GCP foundation. The foundation directory has its own readme with steps.

Where to Continue

After all the steps have been completed from the foundation directory, next is day-to-day operations. Most of the time, deployments are decentralized meaning a project is created and handed over to a PI or researcher to be used for their initiatives. Terraform may never be used again to manage the project, but is used for consistency and repeatability.

To build new projects with new infrastructure, these should be built under the modules directory. To separate out different inputs or to have different environments these will go under the deployments directory. Additional folders under deployments can be used if desired.


Runbook

The GCP Organization, Organization Policies, Organization Level IAM, projects, monitoring, and logging are in the Foundation Terraform code and GitHub Actions. Input below for creating folders, modifying IAM roles and modifiying Projects created by Terraform. Also below is how to manually create GCP projects in the scratch folder.

Folders

IAM

Projects

Overview of Github Actions YAML files and input on how to modify them.

Input below for working with GKE.

Connecting to GKE Cluster

Monitoring and Logging

Node Pools

Storage

VPC Peering

About

Automation for deployment of the Interim Data Facility on Google Cloud Platform.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages