How do you train this model?

[ThierryDeruyttere]

Could you maybe elaborate (i) which files you need to train a bottom-up attention model from scratch, i.e: do you need to convert VG to the COCO format and (ii) how do you train the model with said files.
Thanks!

[airsplay]

I converted from the original caffe weight here: https://github.com/peteanderson80/bottom-up-attention without training it. I think that the original repo has the instructions for training it.

[ThierryDeruyttere]

Thanks for the fast response! I was a bit confused because of https://github.com/airsplay/py-bottom-up-attention/blob/master/configs/VG-Detection/faster_rcnn_R_101_C4_attr_caffemaxpool.yaml if this code base allowed you to train a model from scratch or not! In any case, thanks a lot for this code base!

[airsplay]

Good question! I think that it is possible but need to change some code. E.g., the original detectron2 does not have the prediction branch for the attribute. However, these attribute classification is critical to get good results on vision-and-language tasks.

[yezhengli-Mr9]

I converted from the original caffe weight here: https://github.com/peteanderson80/bottom-up-attention without training it. I think that the original repo has the instructions for training it.

Is weights conversion easy? I personally trying to apply a smaller backbone (for example, resnet18 instead of resnet101). I also follow #6.