Source code for the following paper(arXiv link):
Improved Actor Relation Graph based Group Activity Recognition
Zijian Kuang, Xinran Tie
- Python
3.x
- install requirements using
pip install -r requirements.txt
- RoIAlign for Pytorch (Remeber, to rename folder "roi_align" to something else, such as "roi_align_1")
- Datasets: Collective and Augmented dataset
-
Download both collective and augmented dataset dataset file.
-
Unzip the dataset file into
data/collective
. -
The folder structure should looks like:
. └── data ├── collective ├── ActivityDataset ├── seq01 ├── 9374.jpg ├── 9375.jpg └── ... (3,125 files) ├── seq02 └── seq03
-
Stage1: Fine-tune the model on single frame without using GCN.
# collective dataset python scripts/train_collective_stage1.py
-
Stage2: Fix weights of the feature extraction part of network, and train the network with GCN.
# collective dataset update the stage1_model_path (path of the base model checkpoint) in config.py, and then run the script below: python scripts/train_collective_stage2.py
-
Test: Test the result using test video clips.
# collective dataset update the stage2_model_path (path of the GCN model checkpoint) in config.py, and then run the script below: python scripts/test_collective.py
-
You can specify the running arguments in the python files under
scripts/
directory. The meanings of arguments can be found inconfig.py
Based on our expirements, we suggest to use either NCC or SAD to calculate the pair-wise acotrs' appearance similarty, by setting the self.appearance_calc = "NCC" or "SAD":
To speed up the training/testing speed, we would also suggest to set the self.backbone='mobilenet':
-
Our expirements proved that using MobileNet as backbone in feature extraction will improve the training speed by 35%.
-
Our expirements proved that using NCC or SAD to calculate the pair-wise actors' appearance similarty and draw the actor relation graph can improve the group activity prediction accuracy.
The majority of our project is based on J. Wu et al.'s work, our original contributions (are commented as "Original code by Zijian and Xinran" in each .py file) include changes:
Paper Reference
@inproceedings{CVPR2019_ARG,
title = {Learning Actor Relation Graphs for Group Activity Recognition},
author = {Jianchao Wu and Limin Wang and Li Wang and Jie Guo and Gangshan Wu},
booktitle = {CVPR},
year = {2019}
}
Code Reference