-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Load weights from original pretrained model #1
Comments
Thanks. I actually converted the Caffe weight to PyTorch and dump it. The converted Detectron2 weight is saved on my server here
Trying to load weights from CAFFE was a disaster... It took me some time to fix everything. Besides the name difference of weights, there are mainly three tricky parts:
All these tricky differences (It's really a hard time to locate them...) are carefully handled in the provided weight I am willing to provide the conversion code (it is badly written and organized...) if you are interested in. |
By the way, I also found a bug in the previous Caffe implementation which introduces some disagreement in the proposed objects. The Caffe anchor generator starting from here has shown the desired anchors generated by the script. However, if you printed out the anchors by running this python file. You would definitely be surprised that these anchors do not match the printed one! Whatever, this issue has also been fixed by this |
Ok, thanks for the hints, yes I also figured that your .pkl file would contain the converted weights, but is it derived from the pretrained model or from the alternative pretrained model from bottom-up-attention/#demo? Anyways I would also be interested in the conversion code. I continued my work on my adrelino/py-bottom-up-attention-extracted to compare the original 36 features from bottom up attention with newly generated ones. 1. ReferenceAs an example, I extracted the first line of
from bottom-up-attention/#pretrained-features 2. Generated:I run Base64To investigate further, I base64 decoded and numpy reshaped the fields
Do you have any idea what could be the reason? |
Thanks. If the boxes are the same, the model would predict the same feature as
|
@adrelino @airsplay We have converted the Caffe model (with K in [10,100]) and tested to extract the same visual features (with a tiny deviation <0.01) as the original Caffe version. Our repo: https://github.com/MILVLG/bottom-up-attention.pytorch |
Great to know that. I have added the link to your project to the Readme file. BTW, I noticed that the attribute head is closed here. It would be better if this part is made available. |
we have added this feature in our latest update : ) |
Cool :). |
Hi, in README, you said, "the features extracted from this repo is compatible with LXMERT code and pre-trained models here", that is say, if i want to directly use the extracted features, can i dowlnoad the features (MSCOCO) in this repro https://github.com/airsplay/lxmert ? |
Can your work achieve a fixed extraction of 36 regions? |
Can your work achieve a fixed extraction of 36 regions? |
Thanks for your effort to migrate the slightly modified Faster-RCNN from Caffe to PyTorch!
Your README.md states that
When going through your code and extracting the diff towards detectron2, I could see how you transferred the model from bottom-up-attention using the additional options:
see defaults.py and faster_rcnn_R_101_C4_caffemaxpool.yaml.
However, I cannot see in the code how the original weights are transferred.
Is it possible to load the weights from the original Caffe alternative pretrained model
resnet101_faster_rcnn_final.caffemodel
from bottom-up-attention#demo into your modified PyTorch/detectron2 model?The text was updated successfully, but these errors were encountered: