-
-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Cookiecutter parameter additions, developer workflow tools (#2)
* documentation * scaffolding from Harshet Jain - Medium.com * rename the folder * add eks_cluster_compute_type choice type * add CookieCutter hooks to process eks_cluster_compute_type choice * add narrative comments from Harshet Jain * documentation * attribution * rename CookieCutter to Cookiecutter * scaffold terragrunt template. surface and declare most module variables * add/remove variable declarations * add/remove variable declarations * remove variable eks_node_group_instance_types * add Fargate * setup a simple automated test using a shell script * add project_slug * set cookiecutter.json --directory * inject personalized test settings into the command line * inject personalized test settings into the command line * add an AUTHORS markdown file * documentation * set version constraint for aws provider to a cookiecutter parameter * pin all versions to a cookiecutter parameter * bump all terraform versions up to current latest * repo_name is now github_repo_name * rename stuff. fix capitalizations * better defaults * better success message * Fargate is the default compute cluster * refactor module to latest from terraform registry docs * allow minor updates * terragrunt testing * owncloud is now openedx * pivot to terraform eks fargate profile example * its called eks_node_group_instance_type * fix stupid syntax error * add iam roles * remove variable clust_name * move IAM roles to VPC * terragrunt testing * move more IAM resources to VPC * move more IAM resources to VPC * move more IAM resources to VPC * move more IAM resources to VPC * testing * folder 'eks' is now 'eks_ec2' * documentation * revert to keeping all Fargate-related resources inside of eks_fargate * refactor aws_eks_node_group into its own resource declaration * terragrunt testing - aws_eks_node_group * SAVEPOINT: THIS GENERATES A VALID TERRAFORM PLAN * eks_fargate is now eks_fargate_official * clone eks_ec2 into a new eks_fargate * use new Terraform built-in function to render template * upgrade to v18 * remove all ec2 node management variables * acm.tf is now cloudfront.tf * remove dead code * ingress.tf is now ingress_nginx.tf * scaffold kubernetes and alb * restore v18 mods * add data aws_caller_identity * cert-manager.tf is now acm.tf * cert-manager-values.yaml.tpl is now acm-values.yaml.tpl * add terragrunt variables * add aws_region * documentation * refactor IAM json into external files * refactor IAM json into external files * refactor kubernetes deployment resources into a separate file * cloudfront.tf is now acm_cloudfront.tf * configure kubernetes_ingress. replace iam_policy eks_alb_ingress_controller w AWS json * parameterize fargate_namespace * also issue ssl certs in environment aws region for ALB * also issue ssl certs in environment aws region for ALB * documentation * remove legacy acm.tf. acm_cloudfront.tf is now acm.tf * attach ssl/tls cert to ALB * documentation * eks_ec2 is now eks_ec2_clb * delete eks_fargate_official * iam folder is now json * change name scheme so that we will be able to add eks_alb_ec2 later * lint * add Terraform pre-commit hooks * terrform fmt * lint * pre-commit checks * setup tflint * alb.tf is now application_load_balancer.tf * add a command-line linter script * lint * refactor folder structure to accomodate three types of eks implementation * refactor folder structure to accomodate three types of eks implementation * scaffold eks with alb * parameterize eks_alb_ec2 modules * terragrunt testing * scaffold kubernetes, route53, acm * add handling for eks_alb_ec2 * add eks_alb_ec2 * add ingress for nginx * remove variable subnet_ids * fix ci paths * restructure k8s folders * disable everything but the nginx ingress * documentation * add parameter one_nat_gateway_per_az * lint * update to v3.13.0 * remove vpc_endpoints * Terraform testing * disable instance_types * add depends_on = [aws_eks_cluster.eks] * parameterize environment name * parameterize environment name * parameterize environment name * add Name to tags * add Name to tags * add eks-?? suffix * add helm * add alb config * terragrunt testing * replace clb ingress with alb. * rename stuff * move dns resources to vpc * re-declare subdomains * move certificate management to vpc * point dns to alb host name * revert aws_iam_role.eks_alb_ingress_controller to inline * documentation * split environment_name from environment_subdomain * split environment_name from environment_subdomain * move manifests to k8s folder * prod_environment is now environment_name * testing the test * testing the test * testing the test * testing the test * testing the test * add aws_launch_template and aws_security_group * comment out most parameters in aws_launch_template * terragrunt testing * terragrunt testing * remove aws_launch_template instance_initiated_shutdown_behavior * documentation * documentation * documentation * parameterize eks cluster version * alphabetize inputs * update aws_iam_role.nodes_general * documentation * resequence execution plan * resequence execution plan * add explicit depends_on. documentation. * add explicit depends_on. documentation. * add explicit depends_on. documentation. * documentation. * documentation. * documentation. * documentation. * documentation. * syntax error * formatting * parameterize open edx version * add a make file * work on cookiecutter manifest * work on cookiecutter manifest * work on cookiecutter manifest * work on cookiecutter manifest * work on cookiecutter manifest * remove depends_on. these break * reverse engineer https://raw.githubusercontent.com/iplabs/terraform-kubernetes-alb-ingress-controller/master/main.tf * migrated from terraform/components/kubernetes * documentatino * terragrunt testing * archived failed attempt * switch to a module approach based on https://registry.terraform.io/modules/Young-ook/eks/aws/ * switch to a module approach based on https://registry.terraform.io/modules/Young-ook/eks/aws/ * add local submodule of Young-ook/terraform-aws-eks * add local submodule of Young-ook/terraform-aws-eks * make CLB_EC2 the default * remove all eks alb attempts * remove git submodule * purge eks alb deployment manifests
- Loading branch information
Showing
152 changed files
with
1,977 additions
and
1,716 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,4 @@ | ||
| .terragrunt-cache | ||
| .DS_Store | ||
| .DS_Store | ||
| .terraform.lock.hcl | ||
| .terraform |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| default_language_version: | ||
| # default language version for each language | ||
| python: python3.9 | ||
| repos: | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: v4.1.0 | ||
| hooks: | ||
| # See https://pre-commit.com/hooks.html for more hooks | ||
| - id: check-added-large-files | ||
| - id: check-byte-order-marker | ||
| - id: check-case-conflict | ||
| - id: check-executables-have-shebangs | ||
| - id: check-json | ||
| - id: check-shebang-scripts-are-executable | ||
| - id: check-merge-conflict | ||
| - id: check-symlinks | ||
| - id: check-toml | ||
| - id: check-xml | ||
| - id: destroyed-symlinks | ||
| - id: detect-aws-credentials | ||
| - id: detect-private-key | ||
| - id: end-of-file-fixer | ||
| - id: trailing-whitespace | ||
| # - id: pretty-format-json | ||
| # - id: check-yaml | ||
| - repo: https://github.com/gruntwork-io/pre-commit | ||
| rev: v0.1.17 # Get the latest from: https://github.com/gruntwork-io/pre-commit/releases | ||
| hooks: | ||
| - id: terraform-fmt | ||
| - id: helmlint | ||
| # - id: terraform-validate | ||
| # - id: terragrunt-hclfmt | ||
| # - id: tflint | ||
| # - id: shellcheck | ||
| # - id: yapf | ||
| # - id: markdown-link-check |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Credits | ||
|
|
||
| ## Development Leads | ||
|
|
||
| - Lawrence McDaniel ([@lpm0073](https://github.com/lpm0073)) | ||
|
|
||
| ## Core Committers | ||
|
|
||
| Add yourself here. | ||
|
|
||
| ## Contributors | ||
|
|
||
| Add yourself here. |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| """ | ||
| """ | ||
|
|
||
| import os | ||
| import shutil | ||
|
|
||
|
|
||
| TERMINATOR = "\x1b[0m" | ||
| WARNING = "\x1b[1;33m [WARNING]: " | ||
| INFO = "\x1b[1;33m [INFO]: " | ||
| HINT = "\x1b[3;33m" | ||
| SUCCESS = "\x1b[1;32m [SUCCESS]: " | ||
|
|
||
|
|
||
| def remove_eks_clb_ec2_files(): | ||
| component_dir_path = os.path.join("terraform", "components", "eks_clb_ec2") | ||
| if os.path.exists(component_dir_path): | ||
| shutil.rmtree(component_dir_path) | ||
|
|
||
| terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_clb_ec2") | ||
| if os.path.exists(terragrunt_dir_path): | ||
| shutil.rmtree(terragrunt_dir_path) | ||
|
|
||
| ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_clb_ec2") | ||
| if os.path.exists(ci_dir_path): | ||
| shutil.rmtree(ci_dir_path) | ||
|
|
||
| def remove_eks_alb_ec2_files(): | ||
| component_dir_path = os.path.join("terraform", "components", "eks_alb_ec2") | ||
| if os.path.exists(component_dir_path): | ||
| shutil.rmtree(component_dir_path) | ||
|
|
||
| terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_alb_ec2") | ||
| if os.path.exists(terragrunt_dir_path): | ||
| shutil.rmtree(terragrunt_dir_path) | ||
|
|
||
| ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_alb_ec2") | ||
| if os.path.exists(ci_dir_path): | ||
| shutil.rmtree(ci_dir_path) | ||
|
|
||
| def remove_eks_abl_fargate_files(): | ||
| component_dir_path = os.path.join("terraform", "components", "eks_alb_fargate") | ||
| if os.path.exists(component_dir_path): | ||
| shutil.rmtree(component_dir_path) | ||
|
|
||
| terragrunt_dir_path = os.path.join("terraform", "environments", "{{ cookiecutter.environment_name }}", "eks_alb_fargate") | ||
| if os.path.exists(terragrunt_dir_path): | ||
| shutil.rmtree(terragrunt_dir_path) | ||
|
|
||
| ci_dir_path = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", "eks_alb_fargate") | ||
| if os.path.exists(ci_dir_path): | ||
| shutil.rmtree(ci_dir_path) | ||
|
|
||
| # move kubernetes manifests into the k8s folder and remove the original source folder. | ||
| def move_manifests(folder = ""): | ||
| source = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s", folder) | ||
| destination = os.path.join("ci", "tutor-deploy", "environments", "{{ cookiecutter.environment_name }}", "k8s") | ||
| src_files = os.listdir(source) | ||
| for file_name in src_files: | ||
| full_file_name = os.path.join(source, file_name) | ||
| if os.path.isfile(full_file_name): | ||
| shutil.copy(full_file_name, destination) | ||
| shutil.rmtree(source) | ||
|
|
||
| def main(): | ||
|
|
||
| if "{{ cookiecutter.eks_cluster_compute_type }}" == "CLB_EC2": | ||
| remove_eks_abl_fargate_files() | ||
| remove_eks_alb_ec2_files() | ||
| move_manifests("eks_clb_ec2") | ||
|
|
||
| if "{{ cookiecutter.eks_cluster_compute_type }}" == "ALB_EC2": | ||
| remove_eks_abl_fargate_files() | ||
| remove_eks_clb_ec2_files() | ||
| move_manifests("eks_alb_ec2") | ||
|
|
||
| if "{{ cookiecutter.eks_cluster_compute_type }}" == "ALB_Fargate": | ||
| remove_eks_clb_ec2_files() | ||
| remove_eks_alb_ec2_files() | ||
| move_manifests("eks_alb_fargate") | ||
|
|
||
| print(SUCCESS + "Your Open edX devops repo has been initialized." + TERMINATOR) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| """ | ||
| """ | ||
|
|
||
| TERMINATOR = "\x1b[0m" | ||
| WARNING = "\x1b[1;33m [WARNING]: " | ||
| INFO = "\x1b[1;33m [INFO]: " | ||
| HINT = "\x1b[3;33m" | ||
| SUCCESS = "\x1b[1;32m [SUCCESS]: " | ||
|
|
||
| github_repo_name = "{{ cookiecutter.github_repo_name }}" | ||
| if hasattr(github_repo_name, "isidentifier"): | ||
| assert ( | ||
| github_repo_name.isidentifier() | ||
| ), "'{}' project slug is not a valid Python identifier.".format(github_repo_name) | ||
|
|
||
| assert ( | ||
| github_repo_name == github_repo_name.lower() | ||
| ), "'{}' project slug should be all lowercase".format(github_repo_name) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| #!/bin/sh | ||
| #------------------------------------------------------------------------------ | ||
| # written by: mcdaniel | ||
| # https://lawrencemcdaniel.com | ||
| # | ||
| # date: mar-2022 | ||
| # | ||
| # usage: Runs terraform fmt -recursive | ||
| #------------------------------------------------------------------------------ | ||
|
|
||
| terraform fmt -recursive | ||
| #pre-commit run --all-files |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
|
|
||
| FIX NOTE: what is the correct way to test terraform from Cookiecutter? | ||
|
|
||
| mcdaniel: i'm currently "testing" by generating a Cookiecutter and then running this against the Stepwise Math AWS account, using a sandbox environment to place the VPC and all resources. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| #!/bin/sh | ||
| #------------------------------------------------------------------------------ | ||
| # written by: mcdaniel | ||
| # https://lawrencemcdaniel.com | ||
| # | ||
| # date: mar-2022 | ||
| # | ||
| # usage: Runs the Cookiecutter. | ||
| # Inject your own parameters to override cookiecutter.json defaults | ||
| #------------------------------------------------------------------------------ | ||
|
|
||
| GITHUB_REPO="gh:lpm0073/cookiecutter-openedx-devops" | ||
| GITHUB_BRANCH="eks-fargate" | ||
| OUTPUT_FOLDER="/Users/mcdaniel/cookiecutter/" | ||
|
|
||
| cookiecutter --checkout $GITHUB_BRANCH \ | ||
| --output-dir $OUTPUT_FOLDER \ | ||
| --overwrite-if-exists \ | ||
| --no-input \ | ||
| $GITHUB_REPO \ | ||
| global_platform_name=sandbox \ | ||
| global_platform_region=ohio \ | ||
| global_aws_region=us-east-2 \ | ||
| global_account_id=320713933456 \ | ||
| global_root_domain=stepwisemath.ai \ | ||
| global_aws_route53_hosted_zone_id=Z049210026A5G6XHV84CF \ | ||
| environment_name=fargate \ | ||
| environment_subdomain=fargate |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
Oops, something went wrong.