Empirical Robustness Metric Support for all Evasion attacks #1228
-
Hi, I can see that the attack methods currently supported for evaluating empirical robustness are Fast Gradient Sign Method and Hop Skip Jump. But in theory, it can be evaluated for all evasion attacks, right? Are there any plans to support more attack methods for evaluating empirical robustness? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
Hi @shreyakhandelwal07 Thank you very much for your question! I have moved your issue to our new Discussion forum. Yes, Would you be interested to update and extend |
Beta Was this translation helpful? Give feedback.
Hi @shreyakhandelwal07 Thank you very much for your question! I have moved your issue to our new Discussion forum.
Yes,
empirical_robustness
should be applicable to all evasion attacks. TheSUPPORTED_METHODS
is not exclusive and would need to be adapted by the user. A few drawbacks and reason for caution are that the implementation ofempirical_robustness
includes the default values for the attacks inSUPPORTED_METHODS
which might need adaptations to a specific problem, it does not adapt to models and defences, and it attacks the predictions of the model instead of the stronger attack against true labels. I think these are the main reasons why it has not been updated recently.Would you b…