Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GDT-54 Address mismatches with OpenSearch mapping #112

Merged
merged 1 commit into from
Jan 8, 2024

Conversation

ghukill
Copy link
Contributor

@ghukill ghukill commented Jan 8, 2024

Helpful background context

After an MITAardvark transformation was mostly completed, a couple of small bugs were discovered when attempting to index the transformed TIMDEX records into OpenSearch. This commit adddresses those bugs, both full fixes and workarounds, to allow continued pipeline testing and improvements to the MITAardvark transformation.

How this addresses that need:

  • adds kind property to Dates and Identifiers
  • uses locations.value instead of locations.geodata

How can a reviewer manually see the effects of these changes?

Not readily testable as the StepFunction pipeline is not fully formed to attempt indexing of these documents. But, these changes have been confirmed via a local solution that applied these updates to the transformation process, then successfully indexed documents into OpenSearch.

What are the relevant tickets?

Developer

  • All new ENV is documented in README
  • Stakeholder approval has been confirmed (or is not needed)

Code Reviewer

  • The commit message is clear and follows our guidelines
    (not just this pull request message)
  • There are appropriate tests covering any new functionality
  • The documentation has been updated or is unnecessary
  • The changes have been verified
  • New dependencies are appropriate or there were no changes

Includes new or updated dependencies?

YES | NO

@ghukill ghukill requested a review from ehanson8 January 8, 2024 16:04
Why these changes are being introduced:

After an MITAardvark transformation was mostly completed, a couple of small bugs were discovered when attempting to index the
transformed TIMDEX records into OpenSearch.  This commit adddresses those bugs, both full fixes and workarounds, to allow
continued pipeline testing and improvements to the MITAardvark transformation.

How this addresses that need:
* adds kind property to Dates and Identifiers
* set get_locations() field method to return empty list until mapping approach confirmed

Side effects of this change:
* Anticipating successful indexing attempts from transformed records

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/GDT-54
@ghukill ghukill force-pushed the GDT-54-workarounds-and-bug-fixes branch from 79e553f to 6a97912 Compare January 8, 2024 16:40
@ghukill ghukill merged commit cbddb32 into main Jan 8, 2024
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants