Recreating files in preparation step

[esrivas17]

When running preparation for the second time. It would be good if the software either replaces the coordinates_utm.h5 and background_map.h5 files or detects if the files are old enough to replace them, given that if the second time we process an area with different dimensions, at some step the process will crash.

This is the log when running: sarvey_mti.py -f config.json 1 1 (no issues with step 0)

I just noticed it triggers different error messages when coordinates_utm.h5 and background_map.h5 are not replaced and they are smaller or larger than inputs

coordinates_utm.h5 and background_map.h5 larger than inputs

2024-10-22 00:05:45,437 - INFO - ---------------------------------------------------------------------------------
2024-10-22 00:05:45,438 - INFO - STEP 1: CONSISTENCY CHECK
2024-10-22 00:05:45,438 - INFO - ---------------------------------------------------------------------------------
2024-10-22 00:05:45,438 - INFO - Parameter value default
2024-10-22 00:05:45,438 - INFO - _________ _____ _______
2024-10-22 00:05:45,438 - INFO - coherence_p1 0.9 0.9
2024-10-22 00:05:45,438 - INFO - grid_size 200 200
2024-10-22 00:05:45,439 - INFO - spatial_mask_file_p1 None <---
2024-10-22 00:05:45,439 - INFO - knn 30 30
2024-10-22 00:05:45,439 - INFO - max_arc_length 999999 <--- None
2024-10-22 00:05:45,439 - INFO - velocity_bound 0.1 0.1
2024-10-22 00:05:45,439 - INFO - dem_error_bound 100.0 100.0
2024-10-22 00:05:45,439 - INFO - num_samples 100 100
2024-10-22 00:05:45,439 - INFO - arc_coherence 0.6 0.6
2024-10-22 00:05:45,440 - INFO - min_num_arc 3 3
2024-10-22 00:05:45,440 - INFO -
2024-10-22 00:05:45,442 - INFO - reading box None from file: /data/share/rivas/SARvey_input_data_Masjed_Soleyman_dam_S1_dsc_2015_2018/sbas/temporal_coherence.h5 ...
Traceback (most recent call last):
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 287, in
main()
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 283, in main
run(config=config, args=args, logger=logger)
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 108, in run
proc_obj.runConsistencyCheck()
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/processing.py", line 219, in runConsistencyCheck
cand_mask1 = selectPixels(
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/preparation.py", line 183, in selectPixels
cand_mask_sparse = ut.selectBestPointsInGrid(box_list=box_list, quality=quality, sel_min=grid_min_val)
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/utils.py", line 537, in selectBestPointsInGrid
idx_box = np.where(np.max(qual_box) == qual_box)
File "/home/rivas/miniforge3/envs/sarvey/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2810, in max
return _wrapreduction(a, np.maximum, 'max', axis, None, out,
File "/home/rivas/miniforge3/envs/sarvey/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 88, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity

coordinates_utm.h5 and background_map.h5 smaller than inputs

2024-10-22 00:13:52,311 - INFO - ---------------------------------------------------------------------------------
2024-10-22 00:13:52,311 - INFO - STEP 1: CONSISTENCY CHECK
2024-10-22 00:13:52,311 - INFO - ---------------------------------------------------------------------------------
2024-10-22 00:13:52,311 - INFO - Parameter value default
2024-10-22 00:13:52,311 - INFO - _________ _____ _______
2024-10-22 00:13:52,311 - INFO - coherence_p1 0.9 0.9
2024-10-22 00:13:52,311 - INFO - grid_size 200 200
2024-10-22 00:13:52,311 - INFO - spatial_mask_file_p1 None <---
2024-10-22 00:13:52,312 - INFO - knn 30 30
2024-10-22 00:13:52,312 - INFO - max_arc_length 999999 <--- None
2024-10-22 00:13:52,312 - INFO - velocity_bound 0.1 0.1
2024-10-22 00:13:52,312 - INFO - dem_error_bound 100.0 100.0
2024-10-22 00:13:52,312 - INFO - num_samples 100 100
2024-10-22 00:13:52,312 - INFO - arc_coherence 0.6 0.6
2024-10-22 00:13:52,312 - INFO - min_num_arc 3 3
2024-10-22 00:13:52,312 - INFO -
2024-10-22 00:13:52,313 - INFO - reading box None from file: /data/share/rivas/SARvey_input_data_Masjed_Soleyman_dam_S1_dsc_2015_2018/sbas/temporal_coherence.h5 ...
2024-10-22 00:13:53,207 - INFO - Number of invalid pixels found in image: 1519
2024-10-22 00:13:53,207 - INFO - No mask for area of interest given.
Traceback (most recent call last):
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 287, in
main()
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 283, in main
run(config=config, args=args, logger=logger)
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/sarvey_mti.py", line 108, in run
proc_obj.runConsistencyCheck()
File "/home/rivas/SoftwareInSAR/sarvey/sarvey/processing.py", line 235, in runConsistencyCheck
cand_mask1 &= mask_valid_area
ValueError: operands could not be broadcast together with shapes (100,500) (100,400) (100,500)

[mahmud1]

@esrivas17, could you provide the error traceback here?

[esrivas17]

@esrivas17, could you provide the error traceback here?

I edited the original issue with the error tracebacks