Add Time Series Workflow

CHANGELOG.md

@@ -6,9 +6,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/)
 and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.7.1]
+## [0.8.0]
 
 ### Added
+* New `time_series` workflow for time series processing of GSLC stacks.
+
+### Changed
+* The `back_projection` workflow now accepts an optional `--bounds` parameter to specify the DEM extent
 * The back-projection product now includes the elevation.dem.rsc file.
 
 ## [0.7.0]
@@ -79,4 +83,3 @@ and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Added
 * Initial version of repository.
-

Dockerfile

@@ -1,4 +1,4 @@
 FROM condaforge/mambaforge:latest as builder
 
 ENV USEGPU=false
 ENV FFTW_LIB=/usr/lib/x86_64-linux-gnu/libfftw3f.a
@@ -12,7 +12,7 @@
 COPY ./scripts/build_proc.sh ./srg
 RUN cd /srg && ./build_proc.sh && cd /
 
 FROM condaforge/mambaforge:latest as runner
 
 # For opencontainers label definitions, see:
 #    https://github.com/opencontainers/image-spec/blob/master/annotations.md
@@ -47,6 +47,7 @@
 USER ${CONDA_UID}
 WORKDIR /home/conda/
 
+COPY --chown=${CONDA_UID}:${CONDA_GID} --from=builder /srg/snaphu_v2.0b0.0.0/bin/snaphu /srg/bin/snaphu
 COPY --chown=${CONDA_UID}:${CONDA_GID} --from=builder /srg /srg
 COPY --chown=${CONDA_UID}:${CONDA_GID} --from=builder /hyp3-srg /hyp3-srg
 

README.md

@@ -6,9 +6,10 @@ HyP3 plugin for Stanford Radar Group (SRG) SAR Processor
 > [!WARNING]
 > Running the workflows in this repository requires a compiled version of the [Stanford Radar Group Processor](https://github.com/asfhyp3/srg). For this reason, running this repository's workflows in a standard Python is not implemented yet. Instead, we recommend running the workflows from the docker container as outlined below.
 
-The HyP3-SRG plugin provides a set of workflows (currently only accessible via the docker container) that can be used to process SAR data using the [Stanford Radar Group Processor](https://github.com/asfhyp3/srg). The workflows currently included in this plugin are:
+The HyP3-SRG plugin provides a set of workflows (currently only accessible via the docker container) that can be used to process SAR data using the [Stanford Radar Group Processor](https://github.com/asfhyp3/srg). This set of workflow uses the [SRG alogorithms]((https://doi.org/10.1109/LGRS.2017.2753580)) to process Level-0 Sentinel-1 (S1) data to geocoded, user-friendly products that can be used for time-series analysis. The workflows currently included in this plugin are:
 
-- `back_projection`: A workflow for creating geocoded Sentinel-1 SLCs from Level-0 data using the [back-projection methodology](https://doi.org/10.1109/LGRS.2017.2753580).
+- [`back_projection`](#back-projection): A workflow for creating geocoded Sentinel-1 SLCs,
+- [`time_series`](#time-series): A workflow for creating a deformation timeseries of geocoded Sentinel-1 SLCs. 
 
 To run a workflow, you'll first need to build the docker container:
 ```bash
@@ -23,7 +24,10 @@ docker run -it --rm \
     ++process [WORKFLOW_NAME] \
     [WORKFLOW_ARGS]
 ```
-Here is an example command for the `back_projection` workflow:
+
+### Back-projection 
+The `back_projection` processing type produces geocoded SLCs for raw S1 data. The workflow takes a list of Level-0 S1 granule names and outputs them as geocoded SLCs (GSLCs).
+An example command for the `back_projection` workflow is:
 ```bash
 docker run -it --rm \
     -e EARTHDATA_USERNAME=[YOUR_USERNAME_HERE] \
@@ -34,7 +38,21 @@ docker run -it --rm \
     S1A_IW_RAW__0SDV_20231229T134404_20231229T134436_051870_064437_5F38-RAW
 ```
 
-## Earthdata Login
+### Time-series
+The `time_series` workflow takes a list of up to 50 Level-0 S1 granule names and produces a time-series of the respective geocoded SLCs. Stacks are created with looks `10x10`and baselines of `1000x1000`. A  trophospheric correction is applied using an elevation-dependent regression. 
+This workflow will output interferograms and time-series files for all input granules.
+ The following command will run the `time_series` workflow: 
+```
+docker run -it --rm \
+    -e EARTHDATA_USERNAME=[YOUR_USERNAME_HERE] \
+    -e EARTHDATA_PASSWORD=[YOUR_PASSWORD_HERE] \
+    hyp3-srg:latest \
+    ++process time_series \
+   S1A_IW_RAW__0SDV_20240828T020812_20240828T020844_055407_06C206_6EA7 \
+   S1A_IW_RAW__0SDV_20240816T020812_20240816T020844_055232_06BB8A_C7CA \
+   S1A_IW_RAW__0SDV_20240804T020812_20240804T020844_055057_06B527_1346
+```
+### Earthdata Login
 
 For all workflows, the user must provide their Earthdata Login credentials in order to download input data.
 
@@ -45,7 +63,9 @@ Your credentials can be passed to the workflows via environment variables (`EART
 If you haven't set up a `.netrc` file
 before, check out this [guide](https://harmony.earthdata.nasa.gov/docs#getting-started) to get started.
 
-## GPU Setup:
+
+## Developer setup
+### GPU Setup
 In order for Docker to be able to use the host's GPU, the host must have the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html) installed and configured. 
 The process is different for different OS's and Linux distros. The setup process for the most common distros, including Ubuntu, 
 can be found [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuration). Make sure to follow the [Docker configuration steps](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuration) after installing the package.

pyproject.toml

@@ -43,6 +43,7 @@ Documentation = "https://hyp3-docs.asf.alaska.edu"
 
 [project.entry-points.hyp3]
 back_projection = "hyp3_srg.back_projection:main"
+time_series = "hyp3_srg.time_series:main"
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

src/hyp3_srg/__main__.py

@@ -14,7 +14,7 @@ def main():
     parser = argparse.ArgumentParser(prefix_chars='+', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
     parser.add_argument(
         '++process',
-        choices=['back_projection'],
+        choices=['back_projection', 'time_series'],
         default='back_projection',
         help='Select the HyP3 entrypoint to use',  # HyP3 entrypoints are specified in `pyproject.toml`
     )

src/hyp3_srg/back_projection.py

@@ -18,19 +18,6 @@
 log = logging.getLogger(__name__)
 
 
-def create_param_file(dem_path: Path, dem_rsc_path: Path, output_dir: Path):
-    """Create a parameter file for the processor.
-
-    Args:
-        dem_path: Path to the DEM file
-        dem_rsc_path: Path to the DEM RSC file
-        output_dir: Directory to save the parameter file in
-    """
-    lines = [str(dem_path), str(dem_rsc_path)]
-    with open(output_dir / 'params', 'w') as f:
-        f.write('\n'.join(lines))
-
-
 def check_required_files(required_files: Iterable, work_dir: Path) -> None:
     for file in required_files:
         if not (work_dir / file).exists():
@@ -78,6 +65,7 @@ def create_product(work_dir) -> Path:
     product_name = gslc_path.with_suffix('').name
     orbit_path = work_dir / f'{product_name}.orbtiming'
     rsc_path = work_dir / 'elevation.dem.rsc'
+    bounds_path = work_dir / 'bounds'
     zip_path = work_dir / f'{product_name}.zip'
 
     parameter_file = work_dir / f'{product_name}.txt'
@@ -91,13 +79,15 @@ def create_product(work_dir) -> Path:
         z.write(gslc_path, gslc_path.name)
         z.write(orbit_path, orbit_path.name)
         z.write(rsc_path, rsc_path.name)
+        z.write(bounds_path, bounds_path.name)
         z.write(parameter_file, parameter_file.name)
 
     return zip_path
 
 
 def back_project(
     granules: Iterable[str],
+    bounds: list[float] = None,
     earthdata_username: str = None,
     earthdata_password: str = None,
     bucket: str = None,
@@ -128,10 +118,12 @@ def back_project(
         orbit_path = utils.download_orbit(granule, work_dir)
         bboxs.append(granule_bbox)
         granule_orbit_pairs.append((granule_path, orbit_path))
-
-    full_bbox = unary_union(bboxs).buffer(0.1)
-    dem_path = dem.download_dem_for_srg(full_bbox, work_dir)
-    create_param_file(dem_path, dem_path.with_suffix('.dem.rsc'), work_dir)
+    if bounds is None:
+        full_bbox = unary_union(bboxs).buffer(0.1)
+        dem_path = dem.download_dem_for_srg(full_bbox, work_dir)
+    else:
+        dem_path = dem.download_dem_from_bounds(bounds, work_dir)
+    utils.create_param_file(dem_path, dem_path.with_suffix('.dem.rsc'), work_dir)
 
     back_project_granules(granule_orbit_pairs, work_dir=work_dir, gpu=gpu)
 
@@ -158,6 +150,9 @@ def main():
     parser.add_argument('--bucket', help='AWS S3 bucket HyP3 for upload the final product(s)')
     parser.add_argument('--bucket-prefix', default='', help='Add a bucket prefix to product(s)')
     parser.add_argument('--gpu', default=False, action='store_true', help='Use the GPU-based version of the workflow.')
+    parser.add_argument(
+        '--bounds', default=None, type=float, nargs=4, help='Bounds for DEM (max lat, min lat, min lon, max lon)'
+    )
     parser.add_argument('granules', type=str.split, nargs='+', help='Level-0 S1 granule(s) to back-project.')
     args = parser.parse_args()
     args.granules = [item for sublist in args.granules for item in sublist]

src/hyp3_srg/dem.py

@@ -1,6 +1,7 @@
 """Prepare a Copernicus GLO-30 DEM virtual raster (VRT) covering a given geometry"""
 import logging
 from pathlib import Path
+from typing import Optional
 
 import requests
 from shapely.geometry import Polygon
@@ -40,13 +41,33 @@ def download_dem_for_srg(
     Returns:
         The path to the downloaded DEM
     """
+    stanford_bounds = [footprint.bounds[i] for i in [3, 1, 0, 2]]
+    return download_dem_from_bounds(stanford_bounds, work_dir)
+
+
+def download_dem_from_bounds(bounds: list[float], work_dir: Optional[Path]):
+    """Download the DEM for the given stanford bounds.
+
+    Args:
+        footprint: The footprint to download a DEM for
+        work_dir: The directory to save create the DEM in
+
+    Returns:
+        The path to the downloaded DEM
+    """
+    if (bounds[0] <= bounds[1] or bounds[2] >= bounds[3]):
+        raise ValueError(
+            "Improper bounding box formatting, should be [max latitude, min latitude, min longitude, max longitude]."
+        )
+
     dem_path = work_dir / 'elevation.dem'
     dem_rsc = work_dir / 'elevation.dem.rsc'
 
+    with open(work_dir / 'bounds', 'w') as bounds_file:
+        bounds_file.write(' '.join([str(bound) for bound in bounds]))
+
     ensure_egm_model_available()
 
-    # bounds produces min x, min y, max x, max y; stanford wants toplat, botlat, leftlon, rightlon
-    stanford_bounds = [footprint.bounds[i] for i in [3, 1, 0, 2]]
-    args = [str(dem_path), str(dem_rsc), *stanford_bounds]
+    args = [str(dem_path), str(dem_rsc), *bounds]
     utils.call_stanford_module('DEM/createDEMcop.py', args, work_dir=work_dir)
     return dem_path