You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#33 will soon be resolved with an experiment group that compares untransformed data against transformed data. With data drop combinations and with 3 seeds, this adds an extra 72 dataset pairs to the original 9 for a total of 81.
From the perspective of statistical power, if the correlation between our similarity metrics and transfer attack metrics exists but is small, this may not be enough to avoid a type 2 error. We might therefore wish to add additional experiment groups later on, potentially comparing transformed data against other transformed datasets (or on the more extreme end, adding a new dataset such as MNIST - although this could might a lot of work to make e.g. the configs backwards compatible for replication purposes).
The text was updated successfully, but these errors were encountered:
#33 will soon be resolved with an experiment group that compares untransformed data against transformed data. With data drop combinations and with 3 seeds, this adds an extra 72 dataset pairs to the original 9 for a total of 81.
From the perspective of statistical power, if the correlation between our similarity metrics and transfer attack metrics exists but is small, this may not be enough to avoid a type 2 error. We might therefore wish to add additional experiment groups later on, potentially comparing transformed data against other transformed datasets (or on the more extreme end, adding a new dataset such as MNIST - although this could might a lot of work to make e.g. the configs backwards compatible for replication purposes).
The text was updated successfully, but these errors were encountered: