AUC Metric

[JoaoMoranguinho]

Hello!

I've been using your code and I would like to know which changes or additions must be done in order to obtain the AUC metric at least for the testing phase.

Best regards,
João Moura

[utayao]

For that purpose, the direct way is to write the auc metric calculation function in the metrics.py. Another solution is to modify the code to get model output (classification prediction) and then calculate the metric out of Keras API.

[JoaoMoranguinho]

Hi!

I ended up doing just that. Take a look at the implementation and tell me what you think.

class bag_auc(tf.keras.metrics.Metric):
    def __init__(self, name="bag_auc", **kwargs):
        super(bag_auc, self).__init__(name=name, **kwargs)
        self.true_labels = []
        self.pred_labels = []

    def update_state(self, y_true, y_pred, sample_weight=None):
        b_true = K.mean(y_true, axis=0, keepdims=False)
        self.true_labels.append(b_true.numpy())
        pred_max = K.max(y_pred, axis=0, keepdims=False)
        self.pred_labels.append(pred_max.numpy())

    def result(self):
        m = tf.keras.metrics.AUC()
        m.update_state(self.true_labels, self.pred_labels)
        return m.result().numpy()

Note: the model.compile() function must take the "run_eagerly=True" parameter