Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recall and Confusion matrix, also the confusion matrix, of the WBF #61

Open
moahaimen opened this issue Sep 26, 2023 · 5 comments
Open

Comments

@moahaimen
Copy link

I used the WBF for combining 3 yolov5 models, but now i need the final Precision and Recall and Confusion matrix, also the confusion matrix, of the WBF how can i do it,

@bshakya77
Copy link

@moahaimen, I am using YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, YOLOv8x models with WBF for object detection in VisDrone2019 dataset.

Did you able to perform the evaluation for the ensemble model predictions?

Your feedback is highly appreciated.

Regards,
Bijay

@KAKAROT12419
Copy link

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

@bshakya77
Copy link

can you please tell me what format did weighted box fusion is giving output. Like its xmin, ymin , xmax, ymax or anything else.

Yes, it gives output in xmin, ymin , xmax, ymax format. But, it is in de-normalized format.

Sample prediction result:

  • bounding box coordinates
  • conf scores
  • class label
{'0000006_01659_d_0000004': [[404, 364, 216, 94], [452, 609, 156, 72], [435, 298, 196, 85], [463, 575, 145, 56], [446, 672, 160, 66], [489, 525, 123, 55], [551, 503, 106, 54], [442, 721, 164, 43], [481, 257, 182, 72], [509, 209, 164, 61], [541, 168, 137, 54], [364, 425, 209, 103], [684, 119, 33, 55], [712, 124, 37, 51], [743, 124, 21, 50]], '0000006_05168_d_0000013': [[594, 285, 62, 72], [743, 278, 21, 73], [83, 248, 131, 58], [679, 282, 58, 70], [103, 277, 126, 60], [137, 200, 123, 51], [195, 179, 110, 47], [528, 286, 59, 64], [0, 230, 31, 60], [123, 45, 16, 25], [115, 216, 131, 58]], '0000011_06595_d_0000010': [[746, 191, 18, 55], [697, 151, 18, 45], [513, 324, 36, 62], [496, 344, 75, 47], [340, 394, 60, 75], [255, 103, 20, 45], [533, 125, 18, 46]]}
{'0000006_01659_d_0000004': [[0.86604], [0.84769], [0.81754], [0.81647], [0.81016], [0.80896], [0.76495], [0.73147], [0.6655], [0.63703], [0.62914], [0.45945], [0.44228], [0.42985], [0.40013]], '0000006_05168_d_0000013': [[0.84664], [0.81553], [0.76947], [0.7288], [0.72482], [0.64824], [0.56348], [0.44876], [0.37089], [0.22063], [0.20727]], '0000011_06595_d_0000010': [[0.68253], [0.67364], [0.54689], [0.37154], [0.35386], [0.27636], [0.24566]]}
{'0000006_01659_d_0000004': [5, 3, 5, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5], '0000006_05168_d_0000013': [3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 3], '0000011_06595_d_0000010': [6, 0, 1, 2, 2, 0, 0]}

I am trying to convert it in .csv format that is required for benchmark.
I am confused with the .csv annotation file used in benchmark file (ann). Is it annotation from test data for those images used in prediction?
Also, is it necessary that the prediction result (xmin, ymin , xmax, ymax) should be in normalized form?

Thank you,
Bijay

@KAKAROT12419
Copy link

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

@bshakya77
Copy link

Thanks for information Bijay. See i am not clear with benchmark file can you tell me something about benchmark file please.

I am referring to this file:
https://github.com/ZFTurbo/Weighted-Boxes-Fusion/blob/master/benchmark_oid/run_benchmark_oid.py

I see it takes the two .csv files: detection.csv and annotation.csv and calculates the mean average precision.

  • I wanted to know if the (xmin, ymin , xmax, ymax) should be in normalized form?
  • For annotation file, does the file contain the annotations of test image data from the dataset used (like those images used for predictions)?
  • Have you calculated any other evaluation metrics like recall and accuracy for ensemble prediction?

Thank you,
Bijay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants