Version upgrades and incremental improvements (#3)

* cluster_endpoint_public_access = false * documentation * kubernetes_namespace.namespace depends on module.eks * modify setup instructions * parameterize environment_name * parameterize environment_name * parameterize environment_name * parameterize environment_name * add ci_deploy_hastexo_tutor_contrib_s3_version * add ci_deploy_hastexo_tutor_contrib_s3_version * documentation * documentation * documentation * add eks_managed_node_groups defaults, set fargate profile to default * documentation * documentation * documentation * enable_irsa defaults to true * refactor the service_account_role_arn for vpc-cni * only depends on module.eks * simplify module.eks. remove all addons. only one selector for fargate_profile. * add cluster_addons coredns * remove eks_managed_node_groups * add terraform_aws_modules_vpc * add terraform_aws_modules_vpc * reconfigure module with sponsor example * configure node groups * create separate profiles for openedx, kube-system * lint * add the data sources and the kubernetes provider block * add helm provider. hard code the name space to 'kube-system' * add dependency to module.eks * parameterize terraform_aws_modules_s3 * update change notes * setup an simple 1-container app for deployment testing purposes * refactor kubernetes data blocks * terragrunt plan passes * remove annotations from kubernetes_ingress.app * documentation * add test apps * refactor route53 resources * testing * more route53 refactoring * move alb controller back into eks * move nginx and alb into alb namespace * setup nginx deployment * set depends_on * terragrunt testing * testing * add kubernetes_ingress.nginx * cleanup * documentation * add fargate pod execution role * wrap up EKS Fargate pod execution role * scaffold our own aws alb conroller module * configure aws alb controller based on https://aws.amazon.com/premiumsupport/knowledge-center/eks-alb-ingress-controller-fargate/ * syntax error * rename pod_execution role * terragrunt testing * terragrunt testing * terragrunt plan created on this commit * additional helm config * wait_for_load_balancer = true * add node_security_group_additional_rules to resolve problem with alb creation * documentation * add dependencies * move alb code to its own module * namespace is now kube-system * namespace is now kube-system * move coredns to fargate * terragrunt testing * terragrunt testing * terragrunt testing * terragrunt testing * terragrunt testing * terragrunt testing * add namespaces for application and openedx * move deployments into separate module * move nginx back to alb controller module * alb configuration * add dependencies * testing * healthy port range is 200-399 * node_security_group_additional_rules * open port 80 from anywhere * rename parameter * add module vpc_cni_irsa * set create timeout to 30 minutes * listen ports: 80, 443 * add ssh key name * remove kube-proxy add_on. it doesn't do anything * reassign service ports * change service type to ClusterIP * add ALB diagram * add ALB diagram * documentation * remove port 80 from node_security_group_additional_rules * open port 80 in the EKS created sg * documentation * add dependencies * attribution * consolidate security group into ingress.tf * remove dependency from module.vpc_cni_irsa * create environment_namespace in fargate profile * add namespace for environment * eks is now eks_fargate * add an ec2 worker node configuration * remove eks_worker_group parameters * parameterize eks compute node option * fix hard-coded domain name in spec tls * add eks cluster options * add eks cluster options * restrict port 80 ingress of aws_security_group_rule.nginx to the vpc * documentation * create ec2 and fargate versions of terragrunt.hcl * rename the ingress to alb * documentation * documentation * fire for effect * configure for prod * parameterize version constraint * parameterize version * rename kubernetes_ingress to alb * rename kubernetes_ingress to alb * terragrunt testing * testing * terragrunt testing * refactor post_gen_hook * refactor post_gen_hook * refactor post_gen_hook * refactor post_gen_hook * refactor post_gen_hook * refactor post_gen_hook * assign a value for db_subnet_group_name * wrap booleans in full quotes * move aws_db_subnet_group from vpc to mysql * lint * remove dead code * doesn't really depend on kubernetes * doesn't really depend on kubernetes * reduce non-system namespaces to only openedx, default * rename eks to kubernetes * rename eks to kubernetes * rename secrets to kubernetes_secrets * refactor s3 buckets * eks is now kubernetes * bastion is now ec2_bastion * review dependencies * terragrunt testing * add mock outputs for eks cluster * allow mock outputs on validate and init * add mock outputs * documentatin * replication_group_description is deprecated. changed to description * number_cache_clusters is deprecated. changed to num_cache_clusters * linter * make bastion a cookiecutter option * documentation * add the kubernetes provider * documentation * switch namespace to openedx * fix syntax error * put all secrets in the openedx namespace * application namespace changed to openedx * documentation * documentation * add dependencies * documentation * documentation * documentation * set namespace to openedx * set namespace to openedx * add aws-efs-csi-driver * scaffold kubernetes_persistent_volume_claim * test * documentation * renamed * code pv and pvc * terragrunt testing * remove all persistent volume resources, and the eks add-on * move nginx deployments to ci * move nginx service back to terraform * testing * testing * testing * renamed * rename stuff * remove all ci manifests. TUTOR_RUN_NGINX=true, ENABLE_WEB_PROXY=false * configure lms host * configure lms host * configure lms host * add nginx configs for lms, cms * only deploy lms, cms, smtp * testing * test * test * lint json * disable elasticsearch * refactor kubernetes fargate to use ec2 as primary * add ec2 variables * add ec2 variables * move port 80 sg to alb controller * move coredns back to fargate * remove cluster_security_group_additional_rules * terragrunt testing * testing * reduce eks definition to bare minimum * add coredns and vpc-cni * remove atomic flag * attempt simplest possible ec2-only config * delete the alb controller * remove all compute type option processing * remove deployment options * revert to main, but add in cookiecutter enviornment variables * remove compute specific environments * 86 the clb * re-scaffold from main * tweak name * arg * fix hosts * arrrrrrrrrrrrrr * set namespace to openedx * set namespace to openedx * set namespace to ingress-nginx * add map_users for additional kubernetes admins. add security rule to open port 8443. * parameterize aws account number * parameteriz map_users * standarize required provider block * testing * add map_roles * kill auth_map mod * set ec2 sizes and cluster * fix sg name * add descriptions to all sg rules * add kubernetes as a dependency * add better descriptions to sg resources * best descriptions ever * switch cidrs to strings * fix identifier * tweak description * remove s3 data backup * switch acm module to data declaration * remove all data declarations * documentation * namespace must be openedx * remove certs for environment domain * move us-east-1 certs to cloudfront * add a Usage tag * add a Usage tag * aws_acm_certificate depends on the module
cookiecutter-openedx · Apr 4, 2022 · b588bff · b588bff
1 parent dbd239f
commit b588bff
Show file tree

Hide file tree

Showing 128 changed files with 1,327 additions and 1,856 deletions.
diff --git a/.gitignore b/.gitignore
@@ -2,3 +2,4 @@
 .DS_Store
 .terraform.lock.hcl
 .terraform
+*.out
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -6,7 +6,7 @@ repos:
     rev: v4.1.0
     hooks:
       # See https://pre-commit.com/hooks.html for more hooks
-      - id: check-added-large-files
+      #- id: check-added-large-files
       - id: check-byte-order-marker
       - id: check-case-conflict
       - id: check-executables-have-shebangs

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,43 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/)
+and this project adheres to [Semantic Versioning](http://semver.org/).
+
+
+## [0.0.4]
+
+- parameterized deployment yaml manifests with cookiecutter
+- refactored VPC and EKS modules based on the current latest version of terraform-aws-modules modules
+- upgraded AWS RDS Terraform module to v4
+- added AWS certficates in us-east-1 and the aws region specified in environments/global.hcl
+- added new module for Cloudfront distribution and DNS record for 'cdn' subdomain
+- added new optional module for EC2 Bastion and DNS record for subdomain
+- added version constraint parameters to cookiecutter for all terraform-aws-modules
+- added mock outputs to terragrunt scripts to facilitate `run-all` init and validate operations in environments
+- added this change log
+- resolved deprecation warnings in all modules
+- restructured terraform folders
+- fixed a bug that was causing multiple SSL/TLS certificates to be created in both us-east-1 as well as the environment region
+- added the text 'openedx_devops' to the descriptions of all security groups, IAM roles, and IAM policies resources that are explicitly created by this repository
+
+
+## [0.0.3] - 2022-03-20
+
+- added Cookiecutter parameters for environment_subdomain, ci_build_open_edx_version, ci_build_tutor_version, all teraform version constraints
+- split environment_name and environment_subdomain
+- added Cookiecutter post hook to process selection of EKS Load Balancer configuration
+- added scripts to make, test, lint
+- more sensible defaults in cookiecutter.json
+- expanded README.md documentation
+- added git pre-commit
+- added AUTHORS.md
+
+## [0.0.2] - 2022-03-11
+
+- Additional documentation
+
+## [0.0.1] - 2022-03-10
+
+Initial release
diff --git a/README.rst b/README.rst
diff --git a/cookiecutter.json b/cookiecutter.json
@@ -3,6 +3,7 @@
     "github_repo_name": "openedx_devops",
     "environment_name": "prod",
     "environment_subdomain": "courses",
+    "environment_add_bastion": ["N", "Y"],
     "global_platform_name": "yourschool",
     "global_platform_description": "Your School",
     "global_platform_region": "usa_east",
@@ -21,21 +22,23 @@
     "ci_build_xblock_repository": "edx-ora2",
     "ci_build_xblock_ref": "master",
     "ci_deploy_OPENEDX_COMMON_VERSION": "open-release/{{ cookiecutter.ci_build_open_edx_version }}",
+    "ci_deploy_hastexo_tutor_contrib_s3_version": "v0.2.0",
     "ci_deploy_EMAIL_HOST": "email-smtp.{{ cookiecutter.global_aws_region|lower|replace(' ', '-') }}.amazonaws.com",
     "ci_deploy_EMAIL_PORT": 587,
-    "ci_deploy_EMAIL_USE_TLS": true,
-    "eks_cluster_version": "1.21",
-    "eks_cluster_compute_type": ["CLB_EC2"],
-    "eks_cluster_alb_ingress_controller_version": "v2.4.1",
+    "ci_deploy_EMAIL_USE_TLS": "true",
+    "kubernetes_cluster_version": "1.21",
+    "kubernetes_cluster_compute_type": ["EC2", "Fargate"],
+    "kubernetes_cluster_load_balancer_type": ["ALB", "CLB"],
+    "kubernetes_cluster_ingress_controller_version": "v2.4.1",
     "mongodb_master_username": "root",
     "mongodb_db_port": 27017,
-    "mongodb_deletion_protection": false,
+    "mongodb_deletion_protection": "false",
     "mongodb_engine": "docdb",
     "mongodb_engine_version": "3.6.0",
     "mongodb_retention_period": 7,
     "mongodb_preferred_maintenance_window": "",
     "mongodb_preferred_backup_window": "07:00-09:00",
-    "mongodb_auto_minor_version_upgrade": true,
+    "mongodb_auto_minor_version_upgrade": "true",
     "mysql_username": "root",
     "mysql_port": 3306,
     "mysql_engine": "mysql",
@@ -44,18 +47,30 @@
     "mysql_engine_version": "5.7.33",
     "mysql_allocated_storage": 10,
     "mysql_create_random_password": "true",
-    "mysql_iam_database_authentication_enabled": false,
+    "mysql_iam_database_authentication_enabled": "false",
     "mysql_maintenance_window": "Sun:00:00-Sun:03:00",
     "mysql_backup_window": "03:00-06:00",
     "mysql_backup_retention_period": 7,
-    "mysql_deletion_protection": false,
-    "mysql_skip_final_snapshot": true,
+    "mysql_deletion_protection": "false",
+    "mysql_skip_final_snapshot": "true",
     "redis_engine_version": "6.x",
-    "redis_number_cache_clusters": 1,
+    "redis_num_cache_clusters": 1,
     "redis_port": 6379,
     "redis_family": "redis6.x",
     "terraform_required_version": "~> 1.1",
-    "terraform_provider_kubernetes_version": "~> 2.8",
+    "terraform_aws_modules_acm": "~> 3.4",
+    "terraform_aws_modules_cloudfront": "~> 2.9",
+    "terraform_aws_modules_eks": "~> 18.15",
+    "terraform_aws_modules_iam": "~> 4.14",
+    "terraform_aws_modules_rds": "~> 4.2.0",
+    "terraform_aws_modules_s3": "~> 3.0",
+    "terraform_aws_modules_sg": "~> 4.9",
+    "terraform_aws_modules_vpc": "~> 3.13",
+    "terraform_helm_ingress_nginx": "~> 4",
+    "terraform_helm_cert_manager": "v1.7.1",
+    "terraform_helm_alb_controller_chart_version": "1.4.1",
+    "terraform_helm_aws_efs_csi_driver_version": "1.3.6",
+    "terraform_provider_kubernetes_version": "~> 2.9",
     "terraform_provider_hashicorp_aws_version": "~> 4.6",
     "terraform_provider_hashicorp_helm_version": "~> 2.4",
     "terraform_provider_hashicorp_local_version": "~> 2.2",

diff --git a/hooks/post_gen_project.py b/hooks/post_gen_project.py
@@ -11,73 +11,18 @@
 HINT = "\x1b[3;33m"
 SUCCESS = "\x1b[1;32m [SUCCESS]: "
 
+def remove_bastion():
+    module_dir_path = os.path.join("terraform", "modules", "ec2_bastion")
+    if os.path.exists(module_dir_path):
+        shutil.rmtree(module_dir_path)
 
-def remove_eks_clb_ec2_files():
-    component_dir_path = os.path.join("terraform", "components", "eks_clb_ec2")
-    if os.path.exists(component_dir_path):
-        shutil.rmtree(component_dir_path)
-
-    terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_clb_ec2")
-    if os.path.exists(terragrunt_dir_path):
-        shutil.rmtree(terragrunt_dir_path)
-
-    ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_clb_ec2")
-    if os.path.exists(ci_dir_path):
-        shutil.rmtree(ci_dir_path)
-
-def remove_eks_alb_ec2_files():
-    component_dir_path = os.path.join("terraform", "components", "eks_alb_ec2")
-    if os.path.exists(component_dir_path):
-        shutil.rmtree(component_dir_path)
-
-    terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_alb_ec2")
-    if os.path.exists(terragrunt_dir_path):
-        shutil.rmtree(terragrunt_dir_path)
-
-    ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_alb_ec2")
-    if os.path.exists(ci_dir_path):
-        shutil.rmtree(ci_dir_path)
-
-def remove_eks_abl_fargate_files():
-    component_dir_path = os.path.join("terraform", "components", "eks_alb_fargate")
-    if os.path.exists(component_dir_path):
-        shutil.rmtree(component_dir_path)
-
-    terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_alb_fargate")
+    terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "ec2_bastion")
     if os.path.exists(terragrunt_dir_path):
         shutil.rmtree(terragrunt_dir_path)
 
-    ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_alb_fargate")
-    if os.path.exists(ci_dir_path):
-        shutil.rmtree(ci_dir_path)
-
-# move kubernetes manifests into the k8s folder and remove the original source folder.
-def move_manifests(folder = ""):
-    source = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", folder)
-    destination = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s")
-    src_files = os.listdir(source)
-    for file_name in src_files:
-        full_file_name = os.path.join(source, file_name)
-        if os.path.isfile(full_file_name):
-            shutil.copy(full_file_name, destination)
-    shutil.rmtree(source)
-
 def main():
-
-    if "{{ cookiecutter.eks_cluster_compute_type }}" == "CLB_EC2":
-        remove_eks_abl_fargate_files()
-        remove_eks_alb_ec2_files()
-        move_manifests("eks_clb_ec2")
-
-    if "{{ cookiecutter.eks_cluster_compute_type }}" == "ALB_EC2":
-        remove_eks_abl_fargate_files()
-        remove_eks_clb_ec2_files()
-        move_manifests("eks_alb_ec2")
-
-    if "{{ cookiecutter.eks_cluster_compute_type }}" == "ALB_Fargate":
-        remove_eks_clb_ec2_files()
-        remove_eks_alb_ec2_files()
-        move_manifests("eks_alb_fargate")
+    if "{{ cookiecutter.environment_add_bastion }}".upper() != "Y":
+        remove_bastion()
 
     print(SUCCESS + "Your Open edX devops repo has been initialized." + TERMINATOR)
 

diff --git a/tests/test.sh b/tests/test.sh
@@ -10,19 +10,23 @@
 #------------------------------------------------------------------------------
 
 GITHUB_REPO="gh:lpm0073/cookiecutter-openedx-devops"
-GITHUB_BRANCH="eks-fargate"
+GITHUB_BRANCH="eks_alb"
 OUTPUT_FOLDER="/Users/mcdaniel/cookiecutter/"
 
 cookiecutter --checkout $GITHUB_BRANCH \
              --output-dir $OUTPUT_FOLDER \
              --overwrite-if-exists \
              --no-input \
              $GITHUB_REPO \
-             global_platform_name=sandbox \
-             global_platform_region=ohio \
+             global_platform_name=stepwisemath \
+             global_platform_region=mexico \
              global_aws_region=us-east-2 \
              global_account_id=320713933456 \
              global_root_domain=stepwisemath.ai \
-             global_aws_route53_hosted_zone_id=Z049210026A5G6XHV84CF \
-             environment_name=fargate \
-             environment_subdomain=fargate
+             global_aws_route53_hosted_zone_id=Z0232691KVI7Y7U23HBD \
+             global_ec2_ssh_key_name=stepwisemath-ohio \
+             environment_name=prod \
+             environment_add_bastion=N \
+             environment_subdomain=web \
+             kubernetes_cluster_compute_type=Fargate \
+             kubernetes_cluster_load_balancer_type=ALB
diff --git a/...}/.github/workflows/tutor_deploy_prod.yml → ...loy_{{cookiecutter.environment_name}}.yml b/...}/.github/workflows/tutor_deploy_prod.yml → ...loy_{{cookiecutter.environment_name}}.yml
@@ -16,7 +16,7 @@
 #        7. use Tutor to startup the edxapp platform
 #        8. initialize Kubernetes, and then let it take over
 #------------------------------------------------------------------------------
-name: Tutor Deploy Prod
+name: Tutor Deploy {{ cookiecutter.environment_name }}
 
 on: [workflow_dispatch]
 
@@ -25,8 +25,8 @@ jobs:
     runs-on: ubuntu-20.04
     env:
       KUBECONFIG: /home/runner/.kube/config
-      ENVIRONMENT_ID: prod
-      NAMESPACE: {{ cookiecutter.environment_name }}-{{ cookiecutter.global_platform_name }}-{{ cookiecutter.global_platform_region }}
+      ENVIRONMENT_ID: {{ cookiecutter.environment_name }}
+      NAMESPACE: openedx
       TUTOR_VERSION: {{ cookiecutter.ci_build_tutor_version }}
       OPENEDX_COMMON_VERSION: {{ cookiecutter.ci_deploy_OPENEDX_COMMON_VERSION }}
 
@@ -44,7 +44,7 @@ jobs:
           aws-region: {{ cookiecutter.global_aws_region }}
 
       - name: Get Kube config
-        run: aws eks --region {{ cookiecutter.global_aws_region }} update-kubeconfig --name {{ cookiecutter.environment_name }}-{{ cookiecutter.global_platform_name }}-{{ cookiecutter.global_platform_region }} --alias eks-prod
+        run: aws eks --region {{ cookiecutter.global_aws_region }} update-kubeconfig --name {{ cookiecutter.environment_name }}-{{ cookiecutter.global_platform_name }}-{{ cookiecutter.global_platform_region }} --alias eks-{{ cookiecutter.environment_name }}
 
       - name: Install kubectl
         uses: azure/setup-kubectl@v1
@@ -67,13 +67,13 @@ jobs:
         run: |-
           echo "OPENEDX_COMMON_VERSION=$OPENEDX_COMMON_VERSION" >> $GITHUB_ENV
 
-      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/prod/jwt
+      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/jwt
       - name: Fetch JWT token
         run: |-
           ### Fetch secrets from Kubernetes into Environment
           kubectl get secret jwt -n $NAMESPACE -o json |  jq  '.data| map_values(@base64d)'  | jq -r 'keys[] as $k | "\(.[$k])"' > jwt_private_key
 
-      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/prod/rds
+      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/rds
       - name: MySQL
         run: |-
           echo "TUTOR_RUN_MYSQL=false" >> $GITHUB_ENV
@@ -85,7 +85,7 @@ jobs:
           echo "TUTOR_RUN_MONGODB=false" >> $GITHUB_ENV
           kubectl get secret mongodb-admin -n $NAMESPACE  -o json | jq  '.data | map_values(@base64d)' | jq -r 'keys[] as $k | "TUTOR_\($k|ascii_upcase)=\(.[$k])"' >> $GITHUB_ENV
 
-      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/prod/redis
+      # see: https://github.com/{{ cookiecutter.github_account_name }}/terraform-openedx/tree/main/ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/redis
       - name: Redis
         run: |-
           echo "TUTOR_RUN_REDIS=false" >> $GITHUB_ENV
@@ -116,7 +116,7 @@ jobs:
           cat ci/tutor-deploy/environments/$ENVIRONMENT_ID/config.yml >> $GITHUB_ENV
 
       # note that values like $LMS_HOSTNAME come from this repo
-      # in /ci/tutor-deploy/environments/prod/config.yml
+      # in /ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/config.yml
       - name: Load additional environment specific settings  (computed)
         run: |-
           # We don't want to run these services as we are using the Kubernetes ingress instead.
@@ -129,7 +129,7 @@ jobs:
           echo "TUTOR_RUN_NGINX=false" >> $GITHUB_ENV
 
       # note that the Kubernetes additional config data is locally
-      # stored in ci/tutor-deploy/environments/prod/k8s/
+      # stored in ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/k8s/
       - name: Create Kubernetes add-on resources
         run:  |-
           # Create kubernetes ingress and other environment resources
@@ -192,7 +192,7 @@ jobs:
           cat $TUTOR_ROOT/config.yml
 
       # in this step we're combining our custom configuration data
-      # from ci/tutor-deploy/environments/prod/settings_merge.json with the default
+      # from ci/tutor-deploy/environments/{{ cookiecutter.environment_name }}/settings_merge.json with the default
       # config that was created when we built the openedx docker image with tutor
       - name: Patch Generated Configuration (Static)
         run:  |-