You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
That's an interesting idea! When we think of a leaderboard, we think of rankings. Would this be sort of a "meta" leaderboard, where the models have a rank based on all the benchmarks, e.g. a borda count over all benchmarks? Or based on one of multilingual or English?
If it's the former, one probable scenario I'm not sure how to deal with is with missing values, i.e. if that model does not have values for a particular benchmark(s).
Otherwise, we could enable sorting models based on each benchmark available. And users can click into each benchmark's breakdown.
Hmm I'm not sure whether this is something we actually want to have. If people are agnostic and just want to get a reasonable overview, wouldn't they just choose the multilingual or the English benchmarks? Like why would anyone be equally interested in a somewhat arbitrary selection of benchmarks that are sometimes subsets of each other? (European, Scandinavian, French, German, etc.)
Instead of having either English or Multilingual as the default why not have "overview" as the default?
Something like:
(this does not need to be added for the first version of the benchmark)
The text was updated successfully, but these errors were encountered: