Prediction accuracy in the nvgesture dataset is poor

[uzumaki-noharu]

Hi, @ahmetgunduz
I downloaded your pre-training model and used offline_ The test.py file tests its accuracy on the nvGesture data set, but the results show that its accuracy is very low, with errors such as

UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples Use `zero_ division` parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))

The following is the result after filtering the warning：

[1/8] Time 4.59765 (4.59765) prec@1 0.71875 prec@5 0.98438 precision 0.40512 (0.40512) recall 0.38369 (0.38369)
[2/8] Time 3.16794 (3.88279) prec@1 0.72656 prec@5 0.99219 precision 0.31801 (0.36157) recall 0.25735 (0.32052)
[3/8] Time 3.16407 (3.64322) prec@1 0.76042 prec@5 0.98958 precision 0.41215 (0.37843) recall 0.36193 (0.33433)
[4/8] Time 3.31298 (3.56066) prec@1 0.81250 prec@5 0.98438 precision 0.83333 (0.49216) recall 0.81458 (0.45439)
[5/8] Time 3.10335 (3.46920) prec@1 0.82500 prec@5 0.98438 precision 0.60476 (0.51468) recall 0.54767 (0.47305)
[6/8] Time 3.20778 (3.42563) prec@1 0.82292 prec@5 0.97917 precision 0.53571 (0.51818) recall 0.46732 (0.47209)
[7/8] Time 3.16362 (3.38820) prec@1 0.83036 prec@5 0.98214 precision 0.55556 (0.52352) recall 0.50741 (0.47714)
[8/8] Time 1.68080 (3.17477) prec@1 0.83402 prec@5 0.98133 precision 0.33333 (0.51011) recall 0.29630 (0.46438)
-----Evaluation is finished------
Overall Prec@1 0.83402% Prec@5 0.98133%

At the beginning of reading the nvgesture dataset, because the dataset does not have a sk_color_all folder, the code cannot read the data, so I processed the video into pictures and created this folder.It contains every frame of video.

But when I finished processing and successfully ran the code, the result was very unsatisfactory. I don't know whether my parameters are set incorrectly or my dataset is not processed properly. Can you help me to have a look?Thank you very much！

Here are my parameter settings:

python offline_test.py \
  --root_path ./ \
  --pretrain_path  models/models/nv_resnext_101_Depth_32.pth \
  --resume_path models/models/nv_resnext_101_Depth_32.pth \
  --video_path ../../dataset/nvgesture/ \
  --annotation_path annotation_nvGesture/nvall_but_None.json \
  --result_path results \
  --dataset nvgesture \
  --downsample 1 \
  --sample_duration 32 \
  --learning_rate 0.01 \
  --resnet_shortcut B \
  --model resnext \
  --pretrain_modality Depth \
  --model_depth 101 \
  --batch_size 64 \
  --n_classes 25 \
  --n_finetune_classes 25 \
  --modality Depth \
  --n_threads 0 \
  --train_crop random \
  --checkpoint 1 \
  --train_crop corner \
  --n_val_samples 1 \
  --test_subset val \
  --n_epochs 100 \

[JonathanWangs]

I got the same problem that Prec@1 0.07676% Prec@5 0.19502%. Do you figured it out?

[JonathanWangs]

I solved this problem that I replace the --pretrain_path to --resume_path, and it works. Honestly, I see this solution in another issue, but I have not solved this problem at the first time. I download the code again and use the parameters like these:
--root_path F:/RealTimeGD/Real-time-GesRec-master --video_path nvGesture --annotation_path annotation_nvGesture/nvall_but_None.json --result_path results --modality Depth --pretrain_modality Depth --dataset nvgesture --n_classes 25 --n_finetune_classes 25 --sample_duration 32 --downsample 1 --resume_path nv_resnext_101_Depth_32.pth --test_subset test --model resnext --model_depth 101 --downsample 1

I got this: Overall Prec@1 0.83402% Prec@5 0.98133%

[Prashant2717]

Hi @uzumaki-noharu,
Have you tried with Egogesture dataset? did that work? I'm having an issue to run the main.py.

[qyh-stbz]

Hi, @ahmetgunduz I downloaded your pre-training model and used offline_ The test.py file tests its accuracy on the nvGesture data set, but the results show that its accuracy is very low, with errors such as

UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples Use zero_ division parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))

The following is the result after filtering the warning：

[1/8] Time 4.59765 (4.59765) prec@1 0.71875 prec@5 0.98438 precision 0.40512 (0.40512) recall 0.38369 (0.38369)
[2/8] Time 3.16794 (3.88279) prec@1 0.72656 prec@5 0.99219 precision 0.31801 (0.36157) recall 0.25735 (0.32052)
[3/8] Time 3.16407 (3.64322) prec@1 0.76042 prec@5 0.98958 precision 0.41215 (0.37843) recall 0.36193 (0.33433)
[4/8] Time 3.31298 (3.56066) prec@1 0.81250 prec@5 0.98438 precision 0.83333 (0.49216) recall 0.81458 (0.45439)
[5/8] Time 3.10335 (3.46920) prec@1 0.82500 prec@5 0.98438 precision 0.60476 (0.51468) recall 0.54767 (0.47305)
[6/8] Time 3.20778 (3.42563) prec@1 0.82292 prec@5 0.97917 precision 0.53571 (0.51818) recall 0.46732 (0.47209)
[7/8] Time 3.16362 (3.38820) prec@1 0.83036 prec@5 0.98214 precision 0.55556 (0.52352) recall 0.50741 (0.47714)
[8/8] Time 1.68080 (3.17477) prec@1 0.83402 prec@5 0.98133 precision 0.33333 (0.51011) recall 0.29630 (0.46438)
-----Evaluation is finished------
Overall Prec@1 0.83402% Prec@5 0.98133%

At the beginning of reading the nvgesture dataset, because the dataset does not have a sk_color_all folder, the code cannot read the data, so I processed the video into pictures and created this folder.It contains every frame of video.

But when I finished processing and successfully ran the code, the result was very unsatisfactory. I don't know whether my parameters are set incorrectly or my dataset is not processed properly. Can you help me to have a look?Thank you very much！

Here are my parameter settings:

python offline_test.py \
  --root_path ./ \
  --pretrain_path  models/models/nv_resnext_101_Depth_32.pth \
  --resume_path models/models/nv_resnext_101_Depth_32.pth \
  --video_path ../../dataset/nvgesture/ \
  --annotation_path annotation_nvGesture/nvall_but_None.json \
  --result_path results \
  --dataset nvgesture \
  --downsample 1 \
  --sample_duration 32 \
  --learning_rate 0.01 \
  --resnet_shortcut B \
  --model resnext \
  --pretrain_modality Depth \
  --model_depth 101 \
  --batch_size 64 \
  --n_classes 25 \
  --n_finetune_classes 25 \
  --modality Depth \
  --n_threads 0 \
  --train_crop random \
  --checkpoint 1 \
  --train_crop corner \
  --n_val_samples 1 \
  --test_subset val \
  --n_epochs 100 \

so, could you please tell me what you put in sk_color_all folder?