ASG: Automated Synthetic-to-Real Generalization

Paper

Automated Synthetic-to-Real Generalization.
Wuyang Chen, Zhiding Yu, Zhangyang Wang, Anima Anandkumar.
In ICML 2020.

• Visda-17 to COCO

• train resnet101 with only proxy guidance
• train resnet101 with both proxy guidance and L2O policy
• evaluation

• GTA5 to Cityscapes

• train vgg16 with only proxy guidance
• train vgg16 with both proxy guidance and L2O policy
• evaluation

Usage

Visda-17

• Download Visda-17 Dataset

Evaluation

• Download pretrained ResNet101 on Visda17
• Put the checkpoint under ./ASG/pretrained/
• Put the code below in train.sh

python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--lwf 0.1 \
--resume pretrained/res101_vista17_best.pth.tar \
--evaluate

• Run CUDA_VISIBLE_DEVICES=0 bash train.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with SGD

• Put the code below in train.sh

python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--lwf 0.1

• Run CUDA_VISIBLE_DEVICES=0 bash train.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with L2O

• Download pretrained L2O Policy on Visda17
• Put the checkpoint under ./ASG/pretrained/
• Put the code below in l2o_train.sh

python l2o_train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--lwf 0.1 \
--agent_load_dir ./ASG/pretrained/policy_res101_vista17.pth

• Run CUDA_VISIBLE_DEVICES=0 bash l2o_train.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

GTA5 → Cityscapes

• Download GTA5 dataset.
• Download the leftImg8bit_trainvaltest.zip and gtFine_trainvaltest.zip from the Cityscapes.
• Prepare the annotations by using the createTrainIdLabelImgs.py.
• Put the file of image list into where you save the dataset.
• Remember to properly set the C.dataset_path in the config_seg.py to the path where datasets reside.

Evaluation

• Download pretrained Vgg16 on GTA5
• Put the checkpoint under ./ASG/pretrained/
• Put the code below in train_seg.sh

python train_seg.py \
--epochs 50 \
--batch-size 6 \
--lr 1e-3 \
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--lwf 75. \
--evaluate \
--resume ./pretrained/vgg16_segmentation_best.pth.tar

• Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with SGD

• Put the code below in train_seg.sh

python train_seg.py \
--epochs 50 \
--batch-size 6 \
--lr 1e-3 \
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--lwf 75. \

• Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with L2O

• Download pretrained L2O Policy on GTA5
• Put the checkpoint under ./ASG/pretrained/
• Put the code below in l2o_train_seg.sh

python l2o_train_seg.py \
--epochs 50 \
--batch-size 6 \
--lr 1e-3 \
--num-class 19 \
--gpus 0 \
--gamma 0 \
--early-stop 2 \
--lwf 75. \
--algo reinforce \
--agent_load_dir ./ASG/pretrained/policy_vgg16_segmentation.pth

• Run CUDA_VISIBLE_DEVICES=0 bash l2o_train_seg.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Citation

If you use this code for your research, please cite:

@inproceedings{chen2020automated,
 author = {Chen, Wuyang and Yu, Zhiding and Wang, Zhangyang and Anandkumar, Anima},
 booktitle = {Proceedings of Machine Learning and Systems 2020},
 pages = {8272--8282},
 title = {Automated Synthetic-to-Real Generalization},
 year = {2020}
}