T-SEA: Transfer-based Self-Ensemble Attack on Object Detection(CVPR 2023)

English | 中文

Paper | Hao Huang*, Ziyan Chen*, Huanran Chen*, Yongtao Wang, Kevin Zhang

(*Equal contribution)

An official implementation of T-SEA, and also a framework provided to achieve universal (cross model&instance) patch-based adversarial attack.

If T-SEA is helpful for your work, please help star this repo~ Thanks! :-D

Update

• 2023.02.02 - v1.2 An anchor-free detector, CenterNet, is now supported.
• 2022.11.22 - v1.1 Fix bugs for running train & eval scripts.
• 2022.11.18 - v1.0 This repo is created.

Install

Environment

conda create -n tsea python=3.7
conda activate tsea
pip install -r requirements.txt

Please refer to PyTorch Docs to install torch and torchvision for better compatibility.

Data

Data	Generated Labels	Source
CCTVPerson	GoogleDrive	Human Detection
COCOperson	GoogleDrive	HomePage
INRIAPerson	GoogleDrive	Paper

See more details in Docs.

Run

Patch Zoo - See more results in GoogleDrive | BaiduCloud.

Faster RCNN	SSD	Yolo V5	Yolo V3	CenterNet

Evaluation

The evaluation metrics of the Mean Average Precision(mAP) is provided.

# You can run the demo script directly:
bash ./scripts/eval.sh 0 # gpu id

# To run the full command in the root proj dir:
python evaluate.py \
-p ./results/v5-demo.png \
-cfg ./configs/eval/coco80.yaml \
-lp ./data/INRIAPerson/Test/labels \
-dr ./data/INRIAPerson/Test/pos \
-s ./data/test \
-e 0 # attack class id

# for torch-models(coco91): replace -cfg with ./configs/eval/coco91.yaml

# For detailed supports of the arguments:
python evaluate.py -h

Training

# You can run the demo script directly:
bash ./scripts/train.sh 0 -np
# args: 0 gpu-id, -np new tensorboard process

# Or run the full command:
python train_optim.py -np \
-cfg=demo.yaml \
-s=./results/demo \
-n=v5-combine-demo # patch name & tensorboard name

# For detailed supports of the arguments:
python train_optim.py -h

The default save path of tensorboard logs is runs/.

Modify the config .yaml files for custom settings, see details in README.

Framework Overview

We provide a main pipeline to craft a universal adversarial patch to achieve cross-model & cross-instance attack on detectors, and support evaluations on given data & models. See more details in README.

Acknowledgements

• AdvPatch - Paper | Source Code

Citation

@inproceedings{huang2023t,
  title={T-sea: Transfer-based self-ensemble attack on object detection},
  author={Huang, Hao and Chen, Ziyan and Chen, Huanran and Wang, Yongtao and Zhang, Kevin},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={20514--20523},
  year={2023}
}

Contact Us

If you have any problem about this work, please feel free to reach us out at [email protected].

License

The project is only free for academic research purposes, but needs authorization forcommerce. For commerce permission, please contact [email protected].