As a user, I want to know if the datasets available in the registry are published and visible in the API

[tloubrieu-jpl]

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

Data Users

💪 Motivation

...so that I can actually see what is publicly visible to the end-users among what has been loaded in the registry.

📖 Additional Details

Using the legacy_registry index in opensearch, we want to consider the field "archive_status" for products which are available in the registry.
2 options to choose from:

• if status is "staged" we will consider the data is not available in the registry, found_in_registry will then be false.
• add a new archive_status field in the legacy registry index.

It depends on what we want to show in the kibana dashbaard: data loaded == staged or archived or data loaded == only archived. For what we want to show to the end-users we would have more flexibility if we have 2 fields: found_in_registry and archive status. Or actually with the archive status we might be able to have histogram showing proportions of data not loaded (red), loaded not published (orange), published (green). If that is whaat we want

Acceptance Criteria

Given
When I perform
Then I expect

⚙️ Engineering Details

No response

[tloubrieu-jpl]

Jordan's comment on slack:
Sounds good. per the histogram, that is nice for reporting sake, but in the end, the nodes what the output XLSX (edited)
which right now, is super manual for us to make work right now. I had to convert the exported CSV to enable filtering on the sheet. we may eventually need individual node reports that narrow this a bit, so we can just export the CSV versus having to convert to XLSX to enable filtering on the columns

if I were a node, I feel like I would most want to know what are the bundles and collections that still need to be loaded.
What are the bundles/collections loaded but staged.

[jordanpadams]

Remove this from initial dashboard Epic, as this can be considered a next phase improvement