Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include AWS Solution cloud formation template #1101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -188,11 +188,17 @@ jobs:
docker stop $(docker ps -q) && docker system prune --volumes -f
docker image ls --format '{{.Repository}}:{{.Tag}}' | grep '^migrations/' | xargs -I {} docker image rm {}
cdk-tests:
node-tests:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for including 👍

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
npm-project:
- ./deployment/cdk/opensearch-service-migration
- ./deployment/migration-assistant-solution
defaults:
run:
working-directory: ./deployment/cdk/opensearch-service-migration
working-directory: ${{ matrix.npm-project }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
Expand Down Expand Up @@ -227,7 +233,7 @@ jobs:
all-ci-checks-pass:
needs:
- style-check
- cdk-tests
- node-tests
- gradle-tests
- link-checker
- python-e2e-tests
Expand Down
4 changes: 4 additions & 0 deletions deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ This directory is aimed at housing deployment/distribution methods for various m
A containerized end-to-end solution (including a source and target cluster as well as the migration services) can be deployed locally using the
[Docker Solution](../TrafficCapture/dockerSolution/README.md).

### Deploying the Migration Assistant Solution to AWS

The Migration Assistant is avaliable as an AWS Solution, [learn more](./migration-assistant-solution/README.md).

### Deploying Migration solution to AWS

**Note**: These features are still under development and subject to change
Expand Down
12 changes: 12 additions & 0 deletions deployment/migration-assistant-solution/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
*.js
!jest.config.js
*.d.ts
node_modules
cdk.context.json
coverage
dist
# CDK asset staging directory
.cdk.staging
cdk.out
certs
cdk-synth-output
6 changes: 6 additions & 0 deletions deployment/migration-assistant-solution/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.ts
!*.d.ts

# CDK asset staging directory
.cdk.staging
cdk.out
43 changes: 43 additions & 0 deletions deployment/migration-assistant-solution/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# AWS Solutions Deployment

OpenSearch Migration assistant is distributed in AWS Solutions Library, see the most recent release in https://aws.amazon.com/solutions/implementations/migration-assistant-for-amazon-opensearch-service/

## Solutions Architecture

For the solutions project to allow customization of the feature used by Migration assistant first a bootstrap environment is deployed then a secondary step is used to deploy the configured version of Migration Assistant. See more details about this configuration options from [options.md](../cdk/opensearch-service-migration/options.md).

```mermaid
sequenceDiagram
participant User
participant Cfn as Cloud Formation
participant AWS as AWS Solution Environment
participant MA as Migration Assistant Environment

User ->> Cfn: (Optional) Deploy Bootstrap
User ->>+ Cfn: Deploy AWS Solution via Template
Cfn ->>- AWS: Deploy Resources
User ->>+ AWS: Build Migration Assisant (on EC2 instance)
Note over AWS: Migration Assistant is built<br/>with `./initBootstrap.sh` script<br/>and deployed via `cdk` tool
AWS ->> User: Build Complete
User ->> AWS: Configure & Deploy Migration Assisant
AWS ->>+ Cfn: Deploy Migration Assisant
Cfn ->>- MA: Deploy Resources
deactivate AWS
Note over MA: Migration Assistant is ready
User ->> MA: Log into Migration Assistant Console
User ->> MA: Migration Actions
```

### Migration Assistant

The full range of functionality offered by the migration assistant deployed through the opensearch-service-migration project, see its [README.MD](../cdk/opensearch-service-migration/README.md) for additon details.

## Development

This project is writen in TypeScript and uses the cloud developer tookit (CDK) to produce its build artifacts, cloud formation templates that can be used to deploy onto Amazon Web Services.

### Quick Start Guide

* Install Node 18+ & Npm 10+ https://docs.npmjs.com/downloading-and-installing-node-js-and-npm
* Build the project `npm run build`
* Creat the deployment artifacts `npm run snyth`
42 changes: 42 additions & 0 deletions deployment/migration-assistant-solution/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"app": "npx ts-node --prefer-ts-exts bin/app.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk/core:checkSecretUsage": true,
"@aws-cdk/aws-iam:minimizePolicies": true,
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
"@aws-cdk/core:enablePartitionLiterals": true,
"@aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
]
}
}
14 changes: 14 additions & 0 deletions deployment/migration-assistant-solution/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import eslint from '@eslint/js';
import tseslint from 'typescript-eslint';

export default tseslint.config(
eslint.configs.recommended,
...tseslint.configs.strict,
...tseslint.configs.stylistic,
{
ignores: ['**/*.js'],
"env": {
"jest": true
}
}
);
1 change: 1 addition & 0 deletions deployment/migration-assistant-solution/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './lib/solutions-stack'
55 changes: 55 additions & 0 deletions deployment/migration-assistant-solution/initBootstrap.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash

usage() {
echo "Usage: $0 [--tag <tag_name>] [--branch <branch_name>]"
exit 1
}

tag=""
branch=""

# Parse options
while [[ "$#" -gt 0 ]]; do
case $1 in
--tag)
if [ -n "$branch" ]; then
echo "Error: You cannot specify both --tag and --branch."
usage
fi
tag="$2"
shift
;;
--branch)
if [ -n "$tag" ]; then
echo "Error: You cannot specify both --tag and --branch."
usage
fi
branch="$2"
shift
;;
*)
echo "Unknown parameter passed: $1"
usage
;;
esac
shift
done

yum update && yum install -y git java-11-amazon-corretto-devel docker nodejs https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm
systemctl start docker
git init
git remote | grep "origin" || git remote add -f origin https://github.com/opensearch-project/opensearch-migrations.git

if [ -n "$branch" ]; then
git checkout $branch
elif [ -n "$tag" ]; then
git checkout tags/$tag
else
latest_release_tag=$(curl -s https://api.github.com/repos/opensearch-project/opensearch-migrations/releases/latest | jq -r ".tag_name")
git checkout tags/$latest_release_tag
fi

cd deployment/cdk/opensearch-service-migration || exit
npm install -g aws-cdk 2>&1
npm install 2>&1
./buildDockerImages.sh
7 changes: 7 additions & 0 deletions deployment/migration-assistant-solution/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/** @type {import('ts-jest').JestConfigWithTsJest} **/
module.exports = {
testEnvironment: "node",
transform: {
"^.+.tsx?$": ["ts-jest",{}],
},
};
Loading