-
-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨ PySTACItemReaderIterDataPipe for reading STAC Items #46
Conversation
An iterable-style DataPipe for STAC items! Uses pystac for reading the files or URLs. Included a doctest that checks the metadata within the pystac.item.Item object. Added a new section in the API docs and an intersphinx mapping.
Ensure that zen3geo works even when `pystac` is not installed and add `pystac` to the spatial section of the extras dependencies in pyproject.toml.
92c26e0
to
a686a72
Compare
zen3geo/datapipes/pystac.py
Outdated
@functional_datapipe("read_from_pystac_item") | ||
class PySTACItemReaderIterDataPipe(IterDataPipe): | ||
""" | ||
Takes files from local disk or URLs (as long as they can be read by pystac) | ||
and yields :py:class:`pystac.item.Item` objects (functional name: | ||
``read_from_pystac_item``). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe read_to_pystac_item
instead, since we're reading URLs/files to a pystac.Item
? See also https://pytorch.org/data/0.4/tutorial.html#naming
@functional_datapipe("read_from_pystac_item") | |
class PySTACItemReaderIterDataPipe(IterDataPipe): | |
""" | |
Takes files from local disk or URLs (as long as they can be read by pystac) | |
and yields :py:class:`pystac.item.Item` objects (functional name: | |
``read_from_pystac_item``). | |
@functional_datapipe("read_to_pystac_item") | |
class PySTACItemReaderIterDataPipe(IterDataPipe): | |
""" | |
Takes files from local disk or URLs (as long as they can be read by pystac) | |
and yields :py:class:`pystac.item.Item` objects (functional name: | |
``read_to_pystac_item``). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in d66f1f3
New poetry 1.2.0 resolver and other optional dependencies like adlfs and contextily!
Decided that since the returned object is a `pystac.Item`, it should probably be `read_to_pystac_item`.
Ensure that a JSON STAC item can be read into a pystac.Item object that contains various spatiotemporal metadata.
Use pytest.importorskip to skip running the doctest when pystac cannot be imported.
An iterable-style DataPipe for STAC items! Uses pystac for reading the files or URLs.
Preview at https://zen3geo--46.org.readthedocs.build/en/46/api.html#zen3geo.datapipes.PySTACItemReader
Usage
This
dp_pystac
DataPipe could then be chained toRioxarrayReader
to load the data into anxarray.DataArray
, see walkthrough at https://zen3geo.readthedocs.io/en/v0.3.0/walkthrough.html#find-cloud-optimized-geotiffs.TODO:
Part of #48.
References: