Fixing various GCP issues

Makefile

@@ -53,7 +53,7 @@ destroy:
 # Get pods details of the running cluster.
 .PHONY: getpods
 getpods:
-	python ${ROOT}/test/kube-pods.py
+	python3 ${ROOT}/test/kube-pods.py
 
 .PHONY: lint
 lint: check_shell check_terraform check_shebangs check_trailing_whitespace

README.md

@@ -12,14 +12,14 @@ This repo contains a Terraform modules for creating Kubernetes clusters on Googl
         * [iam](./gcp/modules/iam/README.md): Identity Access Management modules.
 * [aws](./aws/README.md): Amazon Web Services terraform module to create kubernetes cluster using Elastic Kubernetes Service(EKS).
 * [azure](./azure/README.md): Azure terraform module to create kubernetes cluster using Azure Kubernetes Service(AKS).
-* [aws](./aws/README.md): Tanzu terraform module to create kubernetes cluster using Elastic Kubernetes Service.
+* [tanzu](./tanzu/README.md): Tanzu terraform module to create kubernetes cluster using Elastic Kubernetes Service.
 * [test](./test): Automated tests for the files in this project repository.
 
 
 ## What is Kubernetes?
 [Kubernetes](https://kubernetes.io/) is an open source container management system for deploying, scaling, and managing containerized applications. Kubernetes is built by Google based on their internal proprietary container management systems (Borg and Omega). Kubernetes provides a cloud agnostic platform to deploy your containerized applications with built in support for common operational tasks such as replication, autoscaling, self-healing, and rolling deployments.
 
-## What is Manged Kubernetes services?
+## What is Managed Kubernetes services?
 Managed Kubernetes is when third-party providers take over responsibility for some or all of the work necessary for the successful set-up and operation of K8s. Depending on the vendor, “managed” can refer to anything from dedicated support, to hosting with pre-configured environments, to full hosting and operation. We will be using GKE, AKS, EKS, Tanzu. 
 
 ## What is Terraform?
@@ -102,10 +102,10 @@ k8ssandra-terraform/
 |       └── variables.tf 
 |       └── outputs.tf
 |       └── README.md
+|    ├──<a href="gcp/scripts/README.md">scripts</a>
 |  azure/
 |  tanzu/
 |  test/
-|  scripts/
 |  LICENSE
 |  Makefile
 |  README.md

gcp/README.md

@@ -59,10 +59,10 @@ gcp/
       ../modules/gke
       ../modules/gcs
     ├── version.tf 
-    └── backend.tf 
     └── variables.tf 
     └── outputs.tf
     └── README.md
+ ├──<a href="gcp/scripts/README.md">scripts</a>
 </pre>
 
 ## Prerequisites
@@ -78,7 +78,7 @@ gcp/
 |  gsutil           |    4.60    |
 |  kubectl          |  1.17.17   |
 
-The steps to create kubernetes cluster in this document require the following tools installation and configuration to access Google cloud resources.
+The steps to create Kubernetes cluster in this document require the following tools installation and configuration to access Google cloud resources.
 
 ### Cloud project
 
@@ -96,7 +96,7 @@ The following APIs are enabled when the terraform is utilized:
 * Stackdriver Monitoring API
 * IAM Service Account Credentials API
 
-Execute the following commands on the linux machine in order to setup gcloud cli.
+Execute the following commands on the Linux machine in order to setup gcloud cli.
 
 ```console
 gcloud init
@@ -108,7 +108,7 @@ If you created your Google cloud account newly, Google Compute Engine enforces q
 ### Backend
   * Terraform uses persistent state data to keep track of the resources it manages. Since it needs the state in order to know which real-world infrastructure objects correspond to the resources in a configuration, everyone working with a given collection of infrastructure resources must be able to access the same state data.
   * Terraform backend configuration: 
-  [Configuring your backend in aws gcs](https://www.terraform.io/docs/language/settings/backends/gcs.html)
+  [Configuring your backend in gcs](https://www.terraform.io/docs/language/settings/backends/gcs.html)
   * Terraform state
   [How Terraform state works](https://www.terraform.io/docs/language/state/index.html)
 

gcp/env/README.md

@@ -111,10 +111,10 @@ No resources.
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_environment"></a> [environment](#input\_environment) | The environment of the infrastructure being built. | `any` | n/a | yes |
+| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment where infrastructure being built. | `any` | n/a | yes |
 | <a name="input_k8s_namespace"></a> [k8s\_namespace](#input\_k8s\_namespace) | The namespace to use for the deployment and workload identity binding | `string` | `"default"` | no |
 | <a name="input_machine_type"></a> [machine\_type](#input\_machine\_type) | Type of machines which are used by cluster node pool | `string` | `"e2-highmem-8"` | no |
-| <a name="input_name"></a> [name](#input\_name) | Name of the cluster resources | `string` | `"k8ssandra"` | no |
+| <a name="input_name"></a> [name](#input\_name) | Name is the prefix to use for resources that needs to be created. | `string` | `"k8ssandra"` | no |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The GCP project in which the components are created. | `string` | `"k8ssandra-testing"` | no |
 | <a name="input_project_services"></a> [project\_services](#input\_project\_services) | The GCP APIs that should be enabled in this project. | `list(string)` | <pre>[<br>  "cloudresourcemanager.googleapis.com",<br>  "servicenetworking.googleapis.com",<br>  "container.googleapis.com",<br>  "compute.googleapis.com",<br>  "iam.googleapis.com",<br>  "logging.googleapis.com",<br>  "monitoring.googleapis.com",<br>  "sqladmin.googleapis.com",<br>  "securetoken.googleapis.com"<br>]</pre> | no |
 | <a name="input_region"></a> [region](#input\_region) | The region in which to create the VPC network | `string` | `"us-central1"` | no |

gcp/env/outputs.tf

@@ -43,6 +43,7 @@ output "service_account" {
 output "service_account_key" {
   description = "The service Account Key to configure Medusa backups to use GCS bucket"
   value       = module.iam.service_account_key
+  sensitive   = true
 }
 
 # Configuring GKE cluster access for kubectl.

gcp/env/variables.tf

@@ -13,24 +13,23 @@
 # limitations under the License.
 
 variable "name" {
-  description = "Name of the cluster resources"
-  default     = "k8ssandra"
+  description = "Name is the prefix to use for resources that needs to be created."
+  type        = string
 }
 
 variable "environment" {
-  description = "The environment of the infrastructure being built."
+  description = "Name of the environment where infrastructure being built."
+  type        = string
 }
 
 variable "region" {
   description = "The region in which to create the VPC network"
   type        = string
-  default     = "us-central1"
 }
 
 variable "project_id" {
   description = "The GCP project in which the components are created."
   type        = string
-  default     = "k8ssandra-testing"
 }
 
 variable "zone" {

gcp/env/version.tf

@@ -14,7 +14,7 @@
 
 # Terraform provider
 terraform {
-  required_version = ">= 0.12"
+  required_version = ">= 0.14"
 
   required_providers {
     google = {

gcp/modules/gcs/README.md

@@ -31,10 +31,10 @@ No modules.
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_bucket_policy_only"></a> [bucket\_policy\_only](#input\_bucket\_policy\_only) | Enables Bucket Policy Only access to a bucket. | `bool` | `true` | no |
-| <a name="input_environment"></a> [environment](#input\_environment) | The environment of the infrastructure being built. | `string` | n/a | yes |
-| <a name="input_name"></a> [name](#input\_name) | Globally unique name of the bucket | `string` | n/a | yes |
+| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment where infrastructure being built. | `string` | n/a | yes |
+| <a name="input_name"></a> [name](#input\_name) | Name is the prefix to use for resources that needs to be created. | `string` | n/a | yes |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The ID of the project to create the bucket in. | `string` | n/a | yes |
-| <a name="input_region"></a> [region](#input\_region) | location of the bucket | `string` | n/a | yes |
+| <a name="input_region"></a> [region](#input\_region) | The region where terraform builds resources. | `string` | n/a | yes |
 | <a name="input_role"></a> [role](#input\_role) | Role of the google storage bucket iam member | `string` | `"roles/storage.admin"` | no |
 | <a name="input_service_account"></a> [service\_account](#input\_service\_account) | service account email address | `string` | n/a | yes |
 | <a name="input_storage_class"></a> [storage\_class](#input\_storage\_class) | Storage class for the gcs bucket | `string` | `null` | no |

gcp/modules/gcs/variables.tf

@@ -13,12 +13,12 @@
 # limitations under the License.
 
 variable "name" {
-  description = "Globally unique name of the bucket"
+  description = "Name is the prefix to use for resources that needs to be created."
   type        = string
 }
 
 variable "environment" {
-  description = "The environment of the infrastructure being built."
+  description = "Name of the environment where infrastructure being built."
   type        = string
 }
 
@@ -28,7 +28,7 @@ variable "project_id" {
 }
 
 variable "region" {
-  description = "location of the bucket"
+  description = "The region where terraform builds resources."
   type        = string
 }
 

gcp/modules/gke/README.md

@@ -32,11 +32,11 @@ No modules.
 |------|-------------|------|---------|:--------:|
 | <a name="input_enable_private_endpoint"></a> [enable\_private\_endpoint](#input\_enable\_private\_endpoint) | (Beta) Whether the master's internal IP address is used as the cluster endpoint | `bool` | `false` | no |
 | <a name="input_enable_private_nodes"></a> [enable\_private\_nodes](#input\_enable\_private\_nodes) | (Beta) Whether nodes have internal IP addresses only | `bool` | `false` | no |
-| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment the infrastructure is being built. | `string` | n/a | yes |
+| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment where infrastructure being built. | `string` | n/a | yes |
 | <a name="input_initial_node_count"></a> [initial\_node\_count](#input\_initial\_node\_count) | n/a | `number` | `1` | no |
 | <a name="input_machine_type"></a> [machine\_type](#input\_machine\_type) | Type of machines which are used by cluster node pool | `string` | `"e2-highmem-8"` | no |
 | <a name="input_master_ipv4_cidr_block"></a> [master\_ipv4\_cidr\_block](#input\_master\_ipv4\_cidr\_block) | The IP range in CIDR notation (size must be /28) to use for the hosted master network. This range will be used for assigning internal IP addresses to the master or set of masters, as well as the ILB VIP. This range must not overlap with any other ranges in use within the cluster's network. | `string` | `"10.0.0.0/28"` | no |
-| <a name="input_name"></a> [name](#input\_name) | Name of the cluster | `string` | n/a | yes |
+| <a name="input_name"></a> [name](#input\_name) | Name is the prefix to use for resources that needs to be created. | `string` | n/a | yes |
 | <a name="input_network_link"></a> [network\_link](#input\_network\_link) | network link variable from vpc module outputs | `string` | `""` | no |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The project ID where all resources will be launched. | `string` | n/a | yes |
 | <a name="input_region"></a> [region](#input\_region) | The location of the GKE cluster. | `string` | n/a | yes |

gcp/modules/gke/variables.tf

@@ -13,12 +13,12 @@
 # limitations under the License.
 
 variable "name" {
-  description = "Name of the cluster"
+  description = "Name is the prefix to use for resources that needs to be created."
   type        = string
 }
 
 variable "environment" {
-  description = "Name of the environment the infrasstructure is being built."
+  description = "Name of the environment where infrastructure being built."
   type        = string
 }
 

gcp/modules/iam/README.md

@@ -33,7 +33,7 @@ No modules.
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_name"></a> [name](#input\_name) | name of the cluster | `string` | n/a | yes |
+| <a name="input_name"></a> [name](#input\_name) | Name is the prefix to use for resources that needs to be created. | `string` | n/a | yes |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The project in which to hold the components | `string` | n/a | yes |
 | <a name="input_project_services"></a> [project\_services](#input\_project\_services) | n/a | `list(string)` | `[]` | no |
 | <a name="input_region"></a> [region](#input\_region) | The region in which to create the VPC network | `string` | n/a | yes |

gcp/modules/iam/variable.tf

@@ -13,7 +13,7 @@
 # limitations under the License.
 
 variable "name" {
-  description = "name of the cluster"
+  description = "Name is the prefix to use for resources that needs to be created."
   type        = string
 }
 

gcp/modules/vpc/README.md

@@ -42,8 +42,8 @@ No modules.
 | <a name="input_cloud_nat_logging_filter"></a> [cloud\_nat\_logging\_filter](#input\_cloud\_nat\_logging\_filter) | What filtering should be applied to logs for this NAT. Valid values are: 'ERRORS\_ONLY', 'TRANSLATIONS\_ONLY', 'ALL'. Defaults to 'ERRORS\_ONLY'. | `string` | `"ERRORS_ONLY"` | no |
 | <a name="input_enable_cloud_nat"></a> [enable\_cloud\_nat](#input\_enable\_cloud\_nat) | Whether to enable Cloud NAT. This can be used to allow private cluster nodes to accesss the internet. Defaults to 'true' | `bool` | `true` | no |
 | <a name="input_enable_cloud_nat_logging"></a> [enable\_cloud\_nat\_logging](#input\_enable\_cloud\_nat\_logging) | Whether the NAT should export logs. Defaults to 'true'. | `bool` | `true` | no |
-| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment the infrasstructure is being built. | `string` | n/a | yes |
-| <a name="input_name"></a> [name](#input\_name) | name of the cluster | `string` | n/a | yes |
+| <a name="input_environment"></a> [environment](#input\_environment) | Name of the environment where infrastructure being built. | `string` | n/a | yes |
+| <a name="input_name"></a> [name](#input\_name) | Name is the prefix to use for resources that needs to be created. | `string` | n/a | yes |
 | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The project in which to hold the components | `string` | n/a | yes |
 | <a name="input_region"></a> [region](#input\_region) | The region in which to create the VPC network | `string` | n/a | yes |
 | <a name="input_secondary_cidr_block"></a> [secondary\_cidr\_block](#input\_secondary\_cidr\_block) | The IP address range of the VPC's secondary address range in CIDR notation. A prefix of /16 is recommended. Do not use a prefix higher than /27. | `string` | `"10.1.0.0/16"` | no |

gcp/modules/vpc/variables.tf

@@ -13,12 +13,12 @@
 # limitations under the License.
 
 variable "name" {
-  description = "name of the cluster "
+  description = "Name is the prefix to use for resources that needs to be created."
   type        = string
 }
 
 variable "environment" {
-  description = "Name of the environment the infrasstructure is being built."
+  description = "Name of the environment where infrastructure being built."
   type        = string
 }
 

gcp/scripts/apply.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-
+# shellcheck disable=SC1091
 # Copyright 2021 DataStax, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");

gcp/scripts/destroy.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-
+# shellcheck disable=SC1091,SC2154
 # Copyright 2021 DataStax, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -40,8 +40,7 @@ readonly backend_config="terraform {
 cd "${ROOT}/env"
 echo -e "${backend_config}" > backend.tf
 
-# Terraform initinalize the back
-end bucket
+# Terraform initinalize the backend bucket.
 terraform init -input=false
 
 # Select the environment workspace where you want destroy all your resources

gcp/scripts/init.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# shellcheck disable=SC1091,SC2154
 
 # Copyright 2021 DataStax, Inc.
 #
@@ -31,7 +32,7 @@ source "${ROOT}/scripts/common.sh"
 export bucket_name="${TF_VAR_name}-${TF_VAR_project_id}-statefiles"
 
 # Create Google cloud storage bucket to store the state files. 
-source "${ROOT}/gcp/scripts/make_bucket.py"
+python3 "${ROOT}/scripts/make_bucket.py"
 
 # Generate Backend Template to store Terraform State files.
 readonly backend_config="terraform {

gcp/scripts/plan.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/env bash
+# shellcheck disable=SC1091
 
 # Copyright 2021 DataStax, Inc.
 #
@@ -32,4 +33,4 @@ cd "${ROOT}"/env
 terraform validate
 
 # Terraform plan will create a plan file in your current repository. Verify the all the resource it create by using plan. 
-terraform plan -input=false -no-color -out=./plan.json
+terraform plan -no-color