Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(infra): environment-specific TF configurations #388

Merged
merged 1 commit into from
Feb 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,13 @@ yarn-error.log*
*.tfstate.*
*.tfvars
*.tfvars.json
.terraformrc
override.tf
override.tf.json
*_override.tf
*_override.tf.json
crash.log
crash.*.log
.terraformrc

# WebStorm
.idea
Expand Down
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,17 @@
"**/yarn.lock": true
},
"terminal.integrated.env.linux": {
"TF_CLI_CONFIG_FILE": "${workspaceFolder}/.terraformrc",
"CLOUDSDK_ACTIVE_CONFIG_NAME": "default",
"CACHE_DIR": "${workspaceFolder}/.cache"
},
"terminal.integrated.env.osx": {
"TF_CLI_CONFIG_FILE": "${workspaceFolder}/.terraformrc",
"CLOUDSDK_ACTIVE_CONFIG_NAME": "default",
"CACHE_DIR": "${workspaceFolder}/.cache"
},
"terminal.integrated.env.windows": {
"TF_CLI_CONFIG_FILE": "${workspaceFolder}/.terraformrc",
"CLOUDSDK_ACTIVE_CONFIG_NAME": "default",
"CACHE_DIR": "${workspaceFolder}\\.cache"
},
Expand Down
62 changes: 0 additions & 62 deletions infra/.terraform.lock.hcl

This file was deleted.

95 changes: 32 additions & 63 deletions infra/README.md
Original file line number Diff line number Diff line change
@@ -1,85 +1,54 @@
# Cloud Infrastructure
# Terraform Cloud Project

The Google Cloud Platform (GCP) and Cloudflare infrastructure resources required
by the app and that can be bootstrapped via [Terraform](https://www.terraform.io/).
This folder contains the Terraform configurations for our project's infrastructure, managed through Terraform Cloud. The infrastructure is divided into multiple workspaces to handle different environments and shared resources.

## Requirements
## Directory Structure

- [Node.js](https://nodejs.org/en/) v18+ with [Yarn](https://yarnpkg.com/) package manager
- [Google Cloud SDK](https://cloud.google.com/sdk/docs/install) and [Terraform CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli)
- Access to [Google Cloud Projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects) and [Terraform Cloud](https://cloud.hashicorp.com/products/terraform) workspaces
The repository is organized into the following directories, each corresponding to a specific Terraform Cloud workspace:

<details>
<summary>How to install Terraform CLI on macOS?</summary><br>
- **[`/core`](./core/)** - This directory contains the Terraform configurations for the global/shared resources used across all environments. These may include VPCs, shared databases, IAM roles, etc.

```bash
$ brew tap hashicorp/tap
$ brew install hashicorp/tap/terraform
$ brew update
$ brew upgrade hashicorp/tap/terraform
$ yarn tf -version
```
- **[`/server-prod`](./server-prod/)** - Contains the Terraform configurations for the production web server. This workspace should be configured with production-grade settings, ensuring high availability and security.

</details>
- **[`/server-test`](./server-test/)** - Holds the configurations for the testing/QA web server. This environment mirrors production closely and is used for final testing before deploying to production.

<details>
<summary>How to create Google Cloud Platform projects?</summary><br>
- **[`/server-preview`](./server-preview/)** - This directory is for the preview web server, typically used for staging and pre-release reviews. It might contain configurations that are under testing or not yet approved for the testing environment.

Simply navigate to [Google Cloud Resource Manager](https://console.cloud.google.com/cloud-resource-manager)
and create two GCP projects for both `test` (QA) and `prod` (production)
environments, e.g. "example" and "example-test".
## Usage

Fore more information visit https://cloud.google.com/resource-manager/docs/creating-managing-projects<br>
### Prerequisites

</details>
- Terraform
- Access to the Terraform Cloud workspace

<details>
<summary>How to configure Terraform Cloud workspaces?</summary><br>
### Setting Up Workspaces in Terraform Cloud

1. Sign in to [Terraform Cloud](https://cloud.hashicorp.com/products/terraform) dashboard.
2. Create or join an organization.
3. Create two workspaces — `app-test` and `app-prod` for test/QA and production environments.
4. In each of these workspaces create an environment variable called `GOOGLE_CREDENTIALS` with the value containing JSON key of a GCP [service account](https://cloud.google.com/iam/docs/service-accounts). Note, this GCP service account needs to have `Owner` or `Editor` + `Service Usage Admin` roles.
1. Log in to Terraform Cloud.
2. Create a workspace for each directory/environment.
3. Link each workspace to the corresponding directory in this repository.
4. Save Terraform API token to the `../.terraformrc` file.

For more information visit https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference<br>
### Working with Terraform

</details>
To work with Terraform configurations:

<details>
<summary>How to authenticate Terraform CLI in Terraform Cloud?</summary><br>
1. Navigate to the appropriate directory (e.g., `cd server-prod`).
2. Initialize Terraform: `terraform init`.
3. Apply configurations: `terraform apply`.

1. Create a personal or team [API Token](https://learn.hashicorp.com/tutorials/terraform/cloud-login) via [Terraform Cloud](https://app.terraform.io/app/) dashboard → [Settings](https://app.terraform.io/app/settings/tokens).
2. Save API token to the `.terraformrc` file in root of the project:
Ensure that you are working in the correct workspace to avoid misconfigurations.

```
credentials "app.terraform.io" {
token = "xxxxxx.atlasv1.zzzzzzzzzzzzz"
}
```
### Contributions

**NOTE**: This would allow to using different Terraform credentials per software project if you want to.<br>
Please follow our contribution guidelines for making changes or adding new configurations. Ensure you test configurations in the test and preview environments before applying them to production.

</details>
## Support

## Getting Started
For any issues or questions related to this Terraform setup, please contact [@koistya](https://github.com/koistya) on our [Discord server](https://discord.com/invite/bSsv7XM).

- `yarn tf init -upgrade` — Initializes a Terraform workspace
- `yarn tf plan` — creates an execution plan
- `yarn tf apply` — executes the actions proposed by the `yarn tf plan` command
### References

**NOTE**: By default the `app-test` Terraform workspace is used. In order to use
the production workspace, set `TF_WORKSPACE` environment variable to `prod`. For
example:

```bash
$ TF_WORKSPACE=prod tf plan
$ TF_WORKSPACE=prod tf apply
```

**NOTE**: You need to run Terraform commands via `yarn tf <command> [...args]`.

<p align="center">
<a href="https://www.youtube.com/watch?v=tomUWcQ0P3k"><img src="https://user-images.githubusercontent.com/197134/151321818-d47fe54f-c19e-4d4c-9834-c33e589a33e1.png" alt="" width="640" height="360" /></a>
</p>

Fore more information visit https://learn.hashicorp.com/terraform
- https://learn.hashicorp.com/terraform
- https://cloud.google.com/docs/terraform/best-practices-for-terraform
- https://cloud.google.com/iam/docs/workload-identity-federation-with-deployment-pipelines
- https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference.html
12 changes: 0 additions & 12 deletions infra/backend.tf

This file was deleted.

64 changes: 64 additions & 0 deletions infra/core/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions infra/core/database.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Google Cloud SQL
# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database

resource "random_id" "db_name_suffix" {
byte_length = 4
}

resource "google_project_service" "compute" {
service = "compute.googleapis.com"
disable_on_destroy = false
}

resource "google_project_service" "sqladmin" {
service = "sqladmin.googleapis.com"
disable_on_destroy = false
}

resource "google_sql_database_instance" "db" {
name = "db-${random_id.db_name_suffix.hex}"
database_version = "POSTGRES_15"
deletion_protection = false

settings {
tier = "db-f1-micro"

database_flags {
name = "cloudsql.iam_authentication"
value = "on"
}
}

depends_on = [google_project_service.sqladmin]
}

resource "google_sql_user" "developer" {
name = trimsuffix(google_service_account.developer.email, ".gserviceaccount.com")
instance = google_sql_database_instance.db.name
type = "CLOUD_IAM_SERVICE_ACCOUNT"
depends_on = [google_service_account.developer]
}
8 changes: 8 additions & 0 deletions infra/core/iam.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Google Cloud Service Account
# https://cloud.google.com/compute/docs/access/service-accounts
# https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/google_service_account

resource "google_service_account" "developer" {
account_id = "developer"
display_name = "Developer"
}
12 changes: 12 additions & 0 deletions infra/core/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Output Values
# https://developer.hashicorp.com/terraform/language/values/outputs

output "db_instance_name" {
value = google_sql_database_instance.db.name
sensitive = false
}

output "developer_service_account_email" {
value = google_service_account.developer.email
sensitive = false
}
10 changes: 10 additions & 0 deletions infra/core/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Provider Configuration
# https://developer.hashicorp.com/terraform/language/providers/configuration

# Google Cloud Platform Provider
# https://registry.terraform.io/providers/hashicorp/google/latest/docs
provider "google" {
project = var.gcp_project
region = var.gcp_region
zone = var.gcp_zone
}
17 changes: 17 additions & 0 deletions infra/core/storage.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Google Cloud Storage
#
# NOTE: The Google account under which the Terraform Cloud agents run must have
# the owner role on the target domain name in order to create Google Cloud
# Storage buckets using that domain name. You can update the list of owner
# members in the Google Search Console at the following URL:
# https://search.google.com/search-console/welcome?new_domain_name=example.com
# https://cloud.google.com/storage/docs/domain-name-verification

resource "google_storage_bucket" "pkg" {
name = "pkg.${var.root_level_domain}"
location = "US"
force_destroy = false

uniform_bucket_level_access = true
public_access_prevention = "enforced"
}
Loading
Loading