Generating adversarial examples from a restored model causes examples generation to run endlessly

[Abhishek2271]

Hi,

This is not exactly a feature request but a question. Maybe it is already implemented.
Is there any way to debug or to see what is happening when adversarial examples are being generated. For instance,

attack = FastGradientMethod(estimator=classifier, eps=0.2)
x_test_adv = attack.generate(x=p_test)

In the above code to generate adversarial examples using FGSM, I restored a custom model trained to classify SVHN data and created an ART classifier from it. But when I run the attack. generate, the code runs endlessly. I reduced the p_test data to contain only 1 sample but it still did not work.
Is there any way I can see what is happening?

Again, I am sorry if this is not exactly a bug or feature request. I do not know where to ask it.

[beat-buesser]

Hi @Abhishek2271 Thank you very much for using ART! Most of the attacks have a verbose option to show progress during running generate, but not FastGradientMethod as it runs usually very fast. Is your CPU/GPU busy when you run your code? When you stop the run, what does the stack-trace show?

[Abhishek2271]

HI @beat-buesser, thank you for the reply. When I checked, the CPU usages does spike-up

Also, I checked with a simple classifier. predict:

predictions = classifier.predict(p_test)

here, classifier is an ART TensorFlowClassifier. And this also runs continuously.

what does the stack-trace show?

I think it is blank. How do I view the stack-trace? I meant that the output is blank with the code running.

[beat-buesser]

@Abhishek2271 Usually the terminal where you run the program will contain a stack-trace of where the execution was at the moment when you interrupted the running program, this could give an indication in which line your program is at that time or if it has started or not, etc.

I see you are using Windows. Could you please describe how you are running the Python script? Would you be able to share the complete script? It would be interesting to see the code before the attack.

[Abhishek2271]

Hi @beat-buesser,

Thank you for your interest and time on the issue. I am using Jupiter notebook to run python scripts; although, I also tried running via .py files. Doing this also did not show anything on the terminal.

The script is as below:

import tensorflow as tf
print(tf.__version__)
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

from tensorflow.keras.losses import categorical_crossentropy
from tensorflow.keras.utils import to_categorical

from sklearn.model_selection import train_test_split

import tensorflow.keras
import tensorflow.keras.layers as layers
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.image import ImageDataGenerator
#from tensorflow.keras.utils.np_utils import to_categorical
from tensorflow.keras.callbacks import TensorBoard
from tensorflow.keras.layers import Dense, Flatten, Conv2D, AveragePooling2D


import datetime
from time import time

import scipy.io
import os
import math
import tensorflow.contrib.eager as eager

#begin to import data
eager.enable_eager_execution()
def load_data(path):
    """ Helper function for loading a MAT-File"""
    data = scipy.io.loadmat(path)
    return data['X'], data['y']
x_test, y_test = load_data(r'c:\tmp\SVHN\test_32x32.mat')
x_test = x_test.astype('float32')
#normalize the input
x_test /= 255
#reshape labels to array
y_test.reshape((-1))
#Rearrang default dim of img from (HWCB) to (BHWC) 
x_test = np.transpose(x_test, (3, 0, 1, 2))
#select only 10 input for now
p_test = x_test[ :10]
#resize inp to 40:40 to match input signature
p_test = tf.image.resize(p_test, [40,40]).numpy()
#print(p_test.shape)
y_test[y_test == 10] = 0
tf.compat.v1.disable_eager_execution()
print(y_test.shape)
#select only 10 output labels for now
q_test= y_test[:10]
print(q_test.shape)

#train art classifier
from art.attacks.evasion import FastGradientMethod
from art.estimators.classification import TensorFlowClassifier
graph = tf.Graph()
with graph.as_default():
    with tf.Session(config=tf.ConfigProto(allow_soft_placement=True, log_device_placement=True)) as sess:
        global s
        s=sess
        sess.run(tf.global_variables_initializer())
        #_ = tf.Variable(initial_value='fake_variable')
        new_saver = tf.train.import_meta_graph(r"C:\Users\sab\Downloads\AI Testing\Source\Dorefanet\tensorpack\examples\DoReFa-Net\train_log\svhn-dorefa-1,2,4\graph-0707-160817.meta")
        new_saver.restore(s, tf.train.latest_checkpoint(r"C:\Users\sab\Downloads\AI Testing\Source\Dorefanet\tensorpack\examples\DoReFa-Net\train_log\svhn-dorefa-1,2,4"))    
        logits = tf.get_collection("logits")[0]   
        #x = graph.get_tensor_by_name("conv0/W:0")
        #var_23 = [v for v in tf.global_variables() if v.name == "conv0/W:0"][0]
        tf.initialize_all_variables().run()
        #print(x.eval())
        #print(logits) 
        input_ph = tf.placeholder(tf.float32, shape=[None, 40, 40, 3])
        labels_ph = tf.placeholder(tf.int32, shape=[None, 10])
        loss = tf.reduce_mean(tf.losses.softmax_cross_entropy(logits=logits, onehot_labels=labels_ph))
        optimizer = tf.train.AdamOptimizer(learning_rate=0.01)
        train= tf.train.exponential_decay(
                learning_rate=1e-3,
                global_step=2,
                decay_steps=4721 * 100,
                decay_rate=0.5, staircase=True, name='learning_rate')
        min_= 0.0
        max_ =  1.0
        classifier = TensorFlowClassifier(
            clip_values=(min_, max_),
            input_ph=input_ph,
            output=logits,
            labels_ph=labels_ph,
            train=train,
            loss=loss,
            learning=None,
            sess=sess,
            preprocessing_defences=[],
            )         
        classifier.fit(p_test, q_test, batch_size=1, nb_epochs=1)
        predictions_org = classifier.predict(p_test)

The model that I restored has the following graph:

def build_graph(self, image, label):
    fw, fa, fg = get_dorefa(BITW, BITA, BITG)

    # monkey-patch tf.get_variable to apply fw
    def binarize_weight(v):
        name = v.op.name
        # don't binarize first and last layer
        if not name.endswith('W') or 'conv0' in name or 'fc' in name:
            return v
        else:
            logger.info("Binarizing weight {}".format(v.op.name))
            return fw(v)

    def nonlin(x):
        if BITA == 32:
            return tf.nn.relu(x)
        return tf.clip_by_value(x, 0.0, 1.0)

    def activate(x):
        return fa(nonlin(x))

    image = image / 256.0

    with remap_variables(binarize_weight), \
            argscope(BatchNorm, momentum=0.9, epsilon=1e-4), \
            argscope(Conv2D, use_bias=False):
        logits = (LinearWrap(image)
                  .Conv2D('conv0', 48, 5, padding='VALID', use_bias=True)
                  .MaxPooling('pool0', 2, padding='SAME')
                  .apply(activate)
                  # 18
                  .Conv2D('conv1', 64, 3, padding='SAME')
                  .apply(fg)
                  .BatchNorm('bn1').apply(activate)

                  .Conv2D('conv2', 64, 3, padding='SAME')
                  .apply(fg)
                  .BatchNorm('bn2')
                  .MaxPooling('pool1', 2, padding='SAME')
                  .apply(activate)
                  # 9
                  .Conv2D('conv3', 128, 3, padding='VALID')
                  .apply(fg)
                  .BatchNorm('bn3').apply(activate)
                  # 7

                  .Conv2D('conv4', 128, 3, padding='SAME')
                  .apply(fg)
                  .BatchNorm('bn4').apply(activate)

                  .Conv2D('conv5', 128, 3, padding='VALID')
                  .apply(fg)
                  .BatchNorm('bn5').apply(activate)
                  # 5
                  .Dropout(rate=0.5 if self.training else 0.0)
                  .Conv2D('conv6', 512, 5, padding='VALID')
                  .apply(fg).BatchNorm('bn6')
                  .apply(nonlin)
                  .FullyConnected('fc1', 10)())
    tf.nn.softmax(logits, name='output')
    tf.add_to_collection("logits", logits)

I used the "Logits" here as an input to the classifier after restoring.

[Abhishek2271]

svhn-dorefa-1,2,4.zip

Attached is the model I used. The dataset I used is SVHN test data set from http://ufldl.stanford.edu/housenumbers/
dataset is: test_32x32.mat

[beat-buesser]

Hi @Abhishek2271 Thank you for the details, I will take a look if I can run it.

Btw, we started using the new Discussions feature (the tab next to Pull Requests) and we move general question over there until an issue arises.

[Abhishek2271]

svhn-dorefa-1,2,4.zip

[Abhishek2271]

@beat-buesser I think this should work.

[beat-buesser]

@Abhishek2271 Yes, now I can reproduce it.

I think this could be caused by this line input_ph = tf.placeholder(tf.float32, shape=[None, 40, 40, 3]). This placeholder for the inputs is not connected with the model graph. When loading a model from file the existing input placeholder in the graph has to be provided, similar as with the logits a few lines later.

[Abhishek2271]

@beat-buesser, thank you very much for the reply and your patience. I think you are right, this has to do with the input_ph and logits. I think that input_ph is connected with model graph through LinearWrap(image) in the model definition above, but the logits are not properly defined.
But now I am a bit confused about what "logits" would actually mean in the ART TensorflowClassifier. I simply passed tf.add_to_collection("logits", logits) and got the value from there. But now I think may be this is not correct way to do it.

I am sorry for a very basic question, but what does the logits in the ART TensorflowClassifier actually mean?

[beat-buesser]

I think your code definition of the logits looks ok. But I think there are two ways of running the script that need to be distinguished:

• running it for the first time when you create the graph with build_graph where the input placeholder gets connected to the graph through variable image and saved in the checkpoint, this seems to be working

and

• running the script with loading the model from the checkpoint: in this case we need to get the input placeholder from the loaded graph and not redefine it as in line input_ph = tf.placeholder(tf.float32, shape=[None, 40, 40, 3] because this line is now not connected to the graph

[Abhishek2271]

@beat-buesser thank you very much for help. I think I understand what you mean and it makes sense. I think I also save the input placeholder in the model itself and retrieve it but it still gives the same issue. Is below the correct way to restore the input placeholder:
pls notice that I replaced input_ph with:
x_inp = graph.get_tensor_by_name("input:0")

from art.attacks.evasion import FastGradientMethod
from art.estimators.classification import TensorFlowClassifier
graph = tf.Graph()
with graph.as_default():
    with tf.Session(config=tf.ConfigProto(allow_soft_placement=True, log_device_placement=True)) as sess:
        global s
        s=sess
        sess.run(tf.global_variables_initializer())
        #_ = tf.Variable(initial_value='fake_variable')
        new_saver = tf.train.import_meta_graph(r"C:\Users\sab\Downloads\AI Testing\Source\Dorefanet\tensorpack\examples\DoReFa-Net\train_log\svhn-dorefa-1,2,4\graph-0707-160817.meta")
        new_saver.restore(s, tf.train.latest_checkpoint(r"C:\Users\sab\Downloads\AI Testing\Source\Dorefanet\tensorpack\examples\DoReFa-Net\train_log\svhn-dorefa-1,2,4"))    
        logits = tf.get_collection("logits")[0]   
        x_inp = graph.get_tensor_by_name("input:0")
        y_out = graph.get_tensor_by_name("output:0")        
        tf.initialize_all_variables().run()    
        #input_ph = tf.placeholder(tf.float32, shape=[None, 40, 40, 3], name ='input')
        #labels_ph = tf.placeholder(tf.int32, shape=[None,10], name = 'label')
        loss = tf.reduce_mean(tf.losses.softmax_cross_entropy(logits=logits, onehot_labels=y_out))
        optimizer = tf.train.AdamOptimizer(learning_rate=0.01)
        train= tf.train.exponential_decay(
                learning_rate=1e-3,
                global_step=2,
                decay_steps=4721 * 100,
                decay_rate=0.5, staircase=True, name='learning_rate')
        min_= 0.0
        max_ =  1.0
        classifier = TensorFlowClassifier(
            clip_values=(min_, max_),
            input_ph=x_inp,
            output=logits,
            labels_ph=y_out,
            train=train,
            loss=loss,
            learning=None,
            sess=sess,
            preprocessing_defences=[],
            )   

        #x_test = x_test.astype('float32')
        #x_test /= 255
        #print(x_test)
        #x_test = np.transpose(x_test, (3, 0, 1, 2))
        #print(x_test.shape)
        #y_test = y_test.reshape((-1))
        #classf = classifier.predict(x_test)
        classifier.fit(p_test, q_test, batch_size=1, nb_epochs=1)
        #attack = FastGradientMethod(estimator=classifier, eps=0.2)
        #x_test_adv = attack.generate(x=p_test)

        predictions_org = classifier.predict(p_test)

        #accuracy = np.sum(np.argmax(predictions_org, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
        #print("Accuracy on benign test examples: {}%".format(accuracy * 100))

[beat-buesser]

I think you are getting closer but there is still some debugging necessary and it's not related to ART. I think the first goal should be to make a prediction with the model after loading from file before adding it to ART. A few items I have noticed:

• the label tensor for loss (and ART) should be labels_ph and not y_out
• train should be something like train = optimizer.minimize(loss)
• before starting with ART the model should be able to run a prediction with something like predictions = sess.run(logits, feed_dict=feed_dict)

[Abhishek2271]

Hi @beat-buesser, thank you for your patience and your time with the issue. I was able to make predictions with the model by manually restoring the model. Perhaps I try it for ART. As you said, I think it makes more sense to make the model first run predictions and try the same with ART.

Thanks again XD

[beat-buesser]

No problem! Let us know what you find out!

[Abhishek2271]

Hi @beat-buesser,
Thanks again for looking into this issue. To reiterate: I was having issues restoring a saved model and using it to craft adversarial examples.
I was able to make this work by creating a clean graph again and NOT using the metadata for restoring the graph. This was not ART related issue, it turns out that the quantization functions I was using in the model somehow resulted in meta graph not being able to restore the model properly. I verified this by removing all quantization-related functions from the model and restoring the model. ART successfully crafted examples in this case.
So, I ended up restoring the model without using meta graph and running ART using the restored model which worked.

Thank you again for your help with this issue. Your suggestions on this were what led me to solve it.

[beat-buesser]

Hi @Abhishek2271 Thank you very much for sharing the updates and I am very happy to hear that we have been able to help a little bit in solving the issue!