Suggestions for MintPy

[falkamelung]

Description of the desired feature

Just want to share a few ideas. Please feel free to move this elsewhere.

• start all scripts with mp_*. This allows to see the script names by typing mp_ TAB

mp_t
mp_timeseries_rms.py         mp_tropo_gacos.py            mp_tropo_phase_elevation.py

(it would make it easy to combine MIntPy with Non-Mintpy scripts)

• write all executed commands in a .commands.log file. This would make it easy to rerun a command from the workflow using different options. And it keeps a log of manually executed commands.

• support --subset-lalo 19.43:19.51,-155.65:-155.54 format (currently only used in *template control files). This is described in detail in Enhancement suggestion: --subset-lalo LATMIN:LATMAX,LONMIN LONMAX option for view.py #1045.

[avalentino]

Dear @falkamelung I'm pretty sure that the TAB completion function (first point) is already available as soon as you install the (argcomplete optional package) and use the main script named mintpy.

$ mintpy [TAB]
% mintpy 
--help                      -h  -- show this help message and exit                                                                                                                            
--version                   -v  -- show program's version number and exit                                                                                                                     
add                             -- Generate the sum of multiple input files.                                                                                                                  
asc_desc2horz_vert              -- Project Asc and Desc LOS displacement to Horizontal and Vertical direction                                                                                 
closure_phase_bias              -- Phase non-closure related biases correction                                                                                                                
dem_error                       -- DEM Error (Topographic Residual) Correction                                                                                                                
dem_gsi                         -- Prepare DEM from GSI (Japan) DEHM grib files.                                                                                                              
diff                            -- Generate the difference of two input files.                                                                                                                
generate_mask                   -- Generate mask file from input file                                                                                                                         
geocode                         -- Resample radar-coded files into geo-coordinates or vice versa                                                                                              
ifgram_inversion                -- Invert network of interferograms into time-series.                                                                                                         
image_math                      -- Basic Mathmatic Operation of file                                                                                                                          
image_stitch                    -- Stitch/mosaic multiple geocoded datasets into one.                                                                                                         
info                            -- Display Metadata / Structure information of ANY File                                                                                                       
iono_tec                        -- Calculate ionospheric ramps using Global Iono Maps  from GNSS-based TEC products.                                                                          
load_data                       -- Load stacks of interferograms to HDF5 files                                                                                                                
load_gbis                       -- Load GBIS inversion result to HDF5 format.                                                                                                                 
local_oscilator_drift           -- Local Oscilator Drift (LOD) correction of Envisat                                                                                                          
lookup_geo2radar                -- Convert lookup table from geo-coord (GAMMA, ROI_PAC) into radar-coord (ISCE)                                                                               
mask                            -- Mask file                                                                                                                                                  
modify_network                  -- Modify the network of interferograms                                                                                                                       
multilook                       -- Multilook the input file                                                                                                                                   
plate_motion                    -- Plate Motion Correction.                                                                                                                                   
plot_coherence_matrix           -- Plot the coherence matrix of one pixel (interactive)                                                                                                       
plot_network                    -- Display Network of Interferograms                                                                                                                          
plot_transection                -- Generate transect/profile along a line                                                                                                                     
prep_aria                       -- Prepare ARIA processed products for MintPy.                                                                                                                
prep_cosicorr                   -- Prepare attributes file for COSI-Corr pixel offset product.                                                                                                
prep_fringe                     -- Prepare FRInGE products for MintPy                                                                                                                         
prep_gamma                      -- Prepare attributes file for Gamma product.                                                                                                                 
prep_gmtsar                     -- Prepare GMTSAR metadata files.                                                                                                                             
prep_hyp3                       -- Prepare attributes file for HyP3 InSAR product.                                                                                                            
prep_isce                       -- Prepare ISCE-2 metadata files.                                                                                                                             
prep_roipac                     -- Prepare attributes file for ROI_PAC products.                                                                                                              
prep_snap                       -- Prepare attributes file for SNAP products.                                                                                                                 
reference_date                  -- Change reference date of time-series HDF5 file.                                                                                                            
reference_point                 -- Reference to the same pixel in space.                                                                                                                      
remove_hdf5_dset                -- Remove an existing dataset from HDF5 file                                                                                                                  
remove_ramp                     -- Remove 2D ramp(s) from the input file.                                                                                                                     
s1ab_range_bias                 -- Sentinel-1 A/B range bias correction                                                                                                                       
save_gbis                       -- Convert MintPy product to GBIS .mat format.                                                                                                                
save_gdal                       -- Generate GDAL raster from MintPy h5 file.                                                                                                                  
save_gmt                        -- Export geocoded file to GMT grd file                                                                                                                       
save_hdfeos5                    -- Convert MintPy timeseries product into HDF-EOS5 format                                                                                                     
save_kite                       -- Generate KITE (https://github.com/pyrocko/kite) npz and yaml from MintPy HDF5 file.                                                                        
save_kmz                        -- Generate Google Earth KMZ file (overlay / placemarks for files in geo / radar coordinates).                                                                
save_kmz_timeseries             -- Generare Google Earth KMZ file for time-series file.                                                                                                       
save_qgis                       -- Convert to QGIS compatible ps time-series                                                                                                                  
save_roipac                     -- Convert MintPy HDF5 file to ROI_PAC format.                                                                                                                
smallbaselineApp                -- Routine Time Series Analysis for Small Baseline InSAR Stack                                                                                                
solid_earth_tides               -- Solid Earth tides (SET) correction via PySolid                                                                                                             
spatial_average                 -- Calculate the spatial average.                                                                                                                             
spatial_filter                  -- Spatial filtering of 2D image.                                                                                                                             
subset                          -- Generate a subset from file/dataset                                                                                                                        
temporal_average                -- Calculate temporal average (stacking) of multi-temporal datasets                                                                                           
temporal_derivative             -- Calculate the temporal derivative of time-series.                                                                                                          
temporal_filter                 -- Smoothing timeseries in time domain with a moving filter                                                                                                   
timeseries2velocity             -- Estimate velocity / time functions from time-series.                                                                                                       
timeseries_rms                  -- Calculate Root Mean Square (RMS) of deramped residual phase time-series.                                                                                   
tropo_gacos                     -- Tropospheric correction using GACOS (http://www.gacos.net) delays                                                                                          
tropo_phase_elevation           -- Correct Topo-correlated Stratified tropospheric delay                                                                                                      
tropo_pyaps3                    -- Tropospheric correction using weather models via PyAPS                                                                                                     
tsview                          -- Interactive time-series viewer                                                                                                                             
unwrap_error_bridging           -- Unwrapping Error Correction with Bridging                                                                                                                  
unwrap_error_phase_closure      -- Unwrapping Error Correction based on Phase Closure                                                                                                         
view                            -- Plot InSAR Product in 2D    

or

$ mintpy pre [TAB]
 mintpy prep_
prep_aria      -- Prepare ARIA processed products for MintPy.
prep_cosicorr  -- Prepare attributes file for COSI-Corr pixel offset product.
prep_fringe    -- Prepare FRInGE products for MintPy
prep_gamma     -- Prepare attributes file for Gamma product.
prep_gmtsar    -- Prepare GMTSAR metadata files.
prep_hyp3      -- Prepare attributes file for HyP3 InSAR product.
prep_isce      -- Prepare ISCE-2 metadata files.
prep_roipac    -- Prepare attributes file for ROI_PAC products.
prep_snap      -- Prepare attributes file for SNAP products.

I use it both on GNU/Linux and Mac.
Please note that the tab completion shall be activated as described here.

[yunjunz]

Hi @avalentino, I tried to use the argcomplete as below but was unsuccessful on the activation. I am using bash on a Mac with an M2 chip.

After mamba install argcomplete, if I run activate-global-python-argcomplete --user and restart a new shell (and activate the custom environment), the auto completion does not seem to work. While if I run eval "$(register-python-argcomplete mintpy)", it works as below, but only in the current shell, and the effect is gone when I start a new shell; plus, I don't see the nice script explanation on the right as yours. What did I miss?

(insar) yunjunz:~>$ mintpy prep_
prep_aria      prep_cosicorr  prep_fringe    prep_gamma     prep_gmtsar    prep_hyp3      prep_isce      prep_roipac    prep_snap

argcomplete seems super light on the dependency: only standard python. Let's add it to mintpy's dependencies.

We should add a note for this at the end of the installation as an optional setup.

It would be even nicer if we could automate this activation during the installation via conda or pip. It seems that argcomplete does not work out of the box for Windows, but should work for bash with Windows Subsystem for Linux (WSL), so I don't know if including somehow a script/code body for this will impact the functionality for regular Windows users or not. Now I understand this is not a mintpy issue, but a systemwide choice for the user.

[falkamelung]

I quickly tried on the same system and it did not work either for me.

[avalentino]

The @yunjunz I think that activate-global-python-argcomplete --user puts a script in .bash_completion.d, but than that script shall be sourced in the .bashrc.
If i remember correctly, some linux distribution does it automatically in the default configuration, while in other cases the .bashrc have to be modified to do it.

See, e.g., https://serverfault.com/questions/506612/standard-place-for-user-defined-bash-completion-d-scripts

[yunjunz]

Thanks @avalentino, you are right: the activate-global-python-argcomplete --user command works for Linux, because the .bashrc file sources the .bash_completion by default.

For macOS, there is a bash version compatibility issue, after installing the latest version of bash, the same activation command works as well! @falkamelung here is my note for macOS if you would like to give it a try.

I still could not get the nice script explanation though. Any idea?
UPDATE: After reading the argcomplete doc, I realize this is provided by argcomplete's zsh support.