Download only selected bands/files on cop_dataspace

[Rapsodia86]

Hello,
I would like to ask if there is a chance at some point to extend some functionality for specific providers. Here, that would be Copernicus Dataspace.
Some datasets like e.g., Sentinel-1 or -2 can be pretty heavy, while a user may only need just a few bands (#910).
When querying thru oData e.g. "https://catalogue.dataspace.copernicus.eu/odata/v1" instead of downloading a whole zipped *.SAFE file, a user can download manifest XML only:
url = f"{catalogue_odata_url}/Products({product_identifier})/Nodes({product_name})/Nodes(MTD_MSIL1C.xml)/$value"

After taking band names from XML, a user can download files (bands) of interest. The whole approach is in https://github.com/eu-cdse/notebook-samples/blob/main/geo/odata_basics.ipynb (although there is a typo in the search (eu-cdse/notebook-samples#49).

Would that be something to consider?

Thanks!

[sbrunato]

Hello @Rapsodia86 and thanks for this feature request and the associated inputs!

Single assets download is already implemented on the develop branch and will be shipped in the incoming v2.12.0.

You will be able to download individual assets from the new creodias_s3 provider or also STAC providers like astraea_eod, earth_search, planetary_computer, ....

Not yet with cop_dataspace, but we will plan to make it available using your inputs!

[sbrunato]

For this feature, OData should be used for creodias and cop_dataspace instead of OpenSearch

[sbrunato]

For instance, assets can be found through https://download.dataspace.copernicus.eu/odata/v1/Products(7fb094ad-ce1d-5e6e-91ed-7eeacb51e617)/Nodes
Then, following links recursively, we can find bands here https://download.dataspace.copernicus.eu/odata/v1/Products(7fb094ad-ce1d-5e6e-91ed-7eeacb51e617)/Nodes(S2A_MSIL1C_20200801T105631_N0209_R094_T30TYP_20200801T113256.SAFE)/Nodes(GRANULE)/Nodes(L1C_T30TYP_A026689_20200801T110247)/Nodes(IMG_DATA)/Nodes
Download links will be like https://download.dataspace.copernicus.eu/odata/v1/Products(7fb094ad-ce1d-5e6e-91ed-7eeacb51e617)/Nodes(S2A_MSIL1C_20200801T105631_N0209_R094_T30TYP_20200801T113256.SAFE)/Nodes(GRANULE)/Nodes(L1C_T30TYP_A026689_20200801T110247)/Nodes(IMG_DATA)/Nodes(T30TYP_20200801T105631_B01.jp2)/$value

See documentation here

[dalpasso]

Walking through the node tree of the products to discover the assets requires a high number of HTTP request (~10 request per product), leading to a high execution time. As alternative solution, the manifest file can be used to get all the assets at once.

The file name of the manifest is different among different Sentinel products:

• for Sentinel-1 and Sentinel-2 the file is manifest.safe
• for Sentinel-3 the file is xfdumanifest.xml
• no manifest file for Sentinel-5

A post on the Copernicus Data Space forum has been created to have a clarification on the different behaviors: https://forum.dataspace.copernicus.eu/t/location-of-the-manifest-of-sentinel-products/933/1