README.md

SEDNet

This is the Python implementation of the SEDNet with GwcNet backbone. (Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation. Published at CVPR 2023) [Paper]

Environment

• python 3.7
• Pytorch >= 0.10.2
• Cuda >= 11.0
• Anaconda
• Create environment by conda env create -f sednet.yml or conda create --name myenv --file sednet.txt

Data Preparation

Download datasets at:

• Scene Flow Datasets
• KITTI 2015
• Virtual KITTI 2
• DrivingStereo

Training

Training List

• Training/Test split used in the paper.
• You can also use files in generate_datas to generate your own split.
• Please save the split files to ./filenames.

Training Scripts

• main.py is used to training the SEDNet.
• Training scripts are saved in ./scripts
• For --losstype, smooth_l1 is the smooth L1 loss in Guo et al., KG is the log-likelihood loss in Kendall and Gal., UC is our novel divergence loss with the log-likelihood loss.
• To train the LAF baseline, you need to run ./generate_datas/generate_laf_data.py to save the cost volumn of stereo network at first.

Example of Scene Flow Datasets

• run the scripts ./scripts/sceneflow.sh to traing on Scene Flow datasets
• Please update DATAPATH and SAVEPATH as your train data path and the log/checkpoints save path.
• You can use --loadckpt to specific the pre-trained checkpoint file.

Evaluation

• Files in post_process are used to evaluate the models.
• generate_statistic.py is to compute the evaluation metrics.
• generate_conf_and_depth.py can covert the disparity maps and uncertainty maps to depth maps and the confidence maps via gaussian error function.
• Run ./scripts/sceneflow_analysis.sh to generate the evaluation metric of models trained with Scene Flow datasets.

Save Outputs

• Run ./scripts/kitti15_save.sh to save the disparity maps of the model is fine-tunned on KIITI 2015 dataset. Please update the --loadckpt as your checkpoint file to generate the disparity maps.

Pretrained Models

• SceneFlow: SEDNet with a soft inlier threshold of 3 sigma and 11 bins in logspace.
• VKITT2: SEDNet with a soft inlier threshold of 3 sigma and 11 bins in logspace.

Citation

@inproceedings{chen2023learning,
  title={Learning the Distribution of Errors in Stereo Matching for Joint Disparity and Uncertainty Estimation},
  author={Chen, Liyan and Wang, Weihan and Mordohai, Philippos},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={17235--17244},
  year={2023}
}