Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility install option for SDLFv2 #281

Open
5 tasks
cnfait opened this issue Feb 26, 2024 · 5 comments
Open
5 tasks

Compatibility install option for SDLFv2 #281

cnfait opened this issue Feb 26, 2024 · 5 comments
Assignees
Labels
enhancement New feature or request

Comments

@cnfait
Copy link
Contributor

cnfait commented Feb 26, 2024

Having SDLFv1 and SDLFv2 side-by-side in the same accounts. [more details to follow]

  • SSM parameters prefixed with /SDLF2/ instead of /SDLF/
  • KMS aliases starting with sdlf2- instead of sdlf-
  • CodeCommit repositories starting with sdlf2- instead of sdlf-
  • DynamoDB table names using a prefix distinct from octagon- (such as sdlf2-)
  • import team keys when deploying CloudFormation stacks
@cnfait cnfait added the enhancement New feature or request label Feb 26, 2024
@cnfait cnfait self-assigned this Feb 26, 2024
@cnfait
Copy link
Contributor Author

cnfait commented Mar 25, 2024

#290 helps handling the CodeCommit part of this issue.

@cnfait
Copy link
Contributor Author

cnfait commented Mar 25, 2024

#291 helps handling the DynamoDB part - but there is more work required.

@cnfait
Copy link
Contributor Author

cnfait commented Mar 26, 2024

#295 and #296 are also linked to this task.

@cnfait
Copy link
Contributor Author

cnfait commented Mar 26, 2024

There is a branch with the changes necessary to install SDLFv2 side-by-side with SDLFv1:
https://github.com/awslabs/aws-serverless-data-lake-framework/tree/sdlf1-backcompatibility

Some of the changes have been incorporated into main (see previous comments) but not all are relevant/desirable for all installations of SDLFv2. This is an upgrade path more than a good solution long-term.

Important caveats:

  • when deploying foundations, do not use datalake as the domain name (we suggest lake).
  • when deploying stageA and stageB in SDLFv2, do not use the names a and b (lower-case) for pStageName.

@cnfait
Copy link
Contributor Author

cnfait commented Mar 26, 2024

Other changes that may become part of main:

removing FunctionName for the replicate lambda in sdlf-foundations
c968be5

retrieving the Glue crawler name from SSM in sdlf-stageB fetch-metadata Lambda function:
612e154

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant