Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Utilities for Interacting with Ingest Status table #26

Open
medley56 opened this issue Oct 8, 2024 · 0 comments
Open

Add Utilities for Interacting with Ingest Status table #26

medley56 opened this issue Oct 8, 2024 · 0 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@medley56
Copy link
Collaborator

medley56 commented Oct 8, 2024

Context

The ingest status table is written to and read from during ingest processing by a Lambda whose code is externally defined by the user. We should provide utilities for interacting with this table as part of the Construct Library so users can use standard reads/writes on the table without having to write their own DynamoDB code.

Implementation

We could use a library like PynamoDB for this, but then we would be levying that requirement on the users. It's easier if we just directly abstract the Dynamo queries into functions that the user Lambdas can use to interact with the status table.

We should also document the schema of the ingest status table as part of this issue. Write a markdown file that describes how the ingest status table should be used to track processing of files. Note that since Dynamo is schema on write, the user can pretty much do whatever they want with the table during processing.

@medley56 medley56 added documentation Improvements or additions to documentation enhancement New feature or request labels Oct 8, 2024
@medley56 medley56 changed the title Add Utilities for Interactiong with Ingest Status table Add Utilities for Interacting with Ingest Status table Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant