Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STAC Items API #183

Open
mahimatics opened this issue Aug 22, 2024 · 9 comments
Open

STAC Items API #183

mahimatics opened this issue Aug 22, 2024 · 9 comments
Assignees

Comments

@mahimatics
Copy link
Collaborator

Adding best practice fields
Test using CARTOSAT-3 metadata and assets

@mahimatics
Copy link
Collaborator Author

mahimatics commented Aug 22, 2024

Sandeep and Akhil have had initial discussions and we have come to a working consensus about how to model CARTOSAT or other similar datasets. Akhil needs to work on the generation tool to test out the implementation of STAC Items including /items, /search. Additionally we may need to add extra STAC extensions such as Transactions, CQL, EO, Grid etc.

Akhil also to explore Rakshit's suggestion about what to do regarding existing STAC entries for vector datasets. There are 4 options:

  • Do nothing
  • Add a dummy STAC item under each collection which will have the full asset file associated
  • Remove the STAC collection entries for vector datasets and explore using just the OGC Features
  • Remove the STAC collection entries for vector datasets and explore using something new like OGC Records

@mahimatics
Copy link
Collaborator Author

Open question: Is it possible for the properties objects of two STAC items within the same collection to adhere to different JSON schemas?

@mahimatics
Copy link
Collaborator Author

In the properties object, we have to decide which fields needs to be indexed for search API as queryables. Sandeep needs to be consulted for more information about which fields are important for CARTOSAT. Since this is resource specific, this has broader implications to the STAC onboarding flows.

@mahimatics
Copy link
Collaborator Author

Provider information, licence, and band information are going to be queryables for CARTOSAT. Metadata can also be included in the assets, however that will not be searchable.
Sandeep's input is needed for deciding what metadata goes where.

From the API perspective, this may not be important. Work on /search is in progress. /items is complete.

@ThorodanBrom
Copy link
Collaborator

  • Added Flyway migration for STAC Items tables
  • Will be adding more commits to PR later on

@mahimatics
Copy link
Collaborator Author

Item search API testing is in progress
Extra extensions for Sentinel required
Sentinel Item templates to be shared
Ingesting the assets is yet to be completed
Item ID is yet to be decided based on the GIS usecase

@mahimatics
Copy link
Collaborator Author

NRSC data to be onboarded. Points to be decided

  • Items template
  • Items ID
  • Item count
  • Band information

@mahimatics
Copy link
Collaborator Author

mahimatics commented Sep 30, 2024

https://docs.google.com/spreadsheets/d/1HvMHycZ_-WtOM0EqD0iZvIA3OAPW1p2JdV6p9A2r-V4/edit?usp=sharing

Item ID and metadata decided. STAC extensions can be generated with the metadata.
Merged bands as a generated asset is good, but are they feasible within the timeline?

District name to be included as a generated attribute in the properties

@mahimatics
Copy link
Collaborator Author

Data has been uploaded and extracted into a new EC2 instance with a massive EBS volume. Total unzipped data size is 583G.

Varanasi have a few zip files which are broken (around 9 of them). They have been noted.

Symlinks resolution works in s3sync. So the script has been modified to create all the symlinks.

The script needs to be tested a few times and then run on the production instance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants