the gpu output results are different from cpu when I used mxnet-cu100 in adversarial_action_recognition.ipynb

[AndssY]

When I started running this script(adversarial_action_recognition.ipynb) with GPU, the ART classifier outputs are right, but the adversarial attack outputs become:

which are different from the CPU version which in notebooks:

I'm a green hand, do not well in programming. I want to know, can I run this script with GPU to speed up? And if can, how to change it to run it correctly?

My changes are all listed below(except these, I didn't change anything else):

nd.array()    ->    nd.array(ctx=mx.gpu())    #in every nd.array()
...
model=get_model()    ->    model = get_model(ctx=mx.gpu())
....
classifier_art = MXClassifier()    ->    classifier_art = MXClassifier(ctx=mx.gpu())

my system information are:
OS: Windows10
Python version: 3.7.0
ART version: 1.9.1
MXNet version: mxnet-cu100==1.5.0

[beat-buesser]

Hi @AndssY Thank you for using ART! Theoretically, the results from CPU and GPU runs should be very similar. What is the maximal absolute difference between adv_sample and adv_sample_input after cell 10 for CPU and GPU runs?

[beat-buesser]

I have been able to reproduce the issue with 1.7 on Linux.

[beat-buesser]

It looks like the gradients returned by loss_gradient are nan if ctx=mx.gpu().

[beat-buesser]

One thing I noticed, the loss should be from logits: loss=gluon.loss.SoftmaxCrossEntropyLoss(from_logits=True),, but it didn't solve the issue.

Repeating calls loss_gradient leads occasionally to non-nan gradients, but mostly the behavior seems similar as described also here: awslabs/gluonts#1904

[AndssY]

I have been meet this situation(loss_gradient are nan) when I use this script to attack other example in UCF101.

after trying loss=gluon.loss.SoftmaxCrossEntropyLoss(from_logits=True), the result becomes:

So is this problem related to Mxnet and now we can't know what happen and it can't be fixed?
(why output are different, why NaN gradient, when use GPU but works fine for CPU)

Would it be better to replace the deep learning framework with PyTorch or TensorFlow in adversarial_action_recognition.ipynb?
or I want to know how to replace Mxnet to other framework(like PyTorch), is there some tutorials here? or what can I learn to attack a action recognition model in PyTorch? : ) Thanks again for help!

[beat-buesser]

Reproducing the notebook in another deep learning framework is definitely possible. The two main steps you would have to change are getting a model for action recognition in another framework and replace MXClassifier with PyTorchClassifier, for example.