You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What problem or use case are you trying to solve?
Currently, there is no comprehensive leaderboard that effectively tracks the performance of open-sourced models within OpenHands. A clear leaderboard would facilitate comparison and improvement.
Describe the UX of the solution you'd like
An online sheet that continuously updates and tracks performance metrics for various models, possibly with the ability to click on links for deeper insights specific to each model.
Do you have thoughts on the technical implementation?
The leaderboard could utilize existing performance data stored in a central database, automatically populated and refreshed at intervals, ensuring real-time updates.
Describe alternatives you've considered
Using a static document or manual tracking methods, but these lack the dynamic features and timeliness that a real leaderboard would provide.
Additional context
This initiative was discussed in a recent thread on Slack with users expressing interest in better tracking mechanisms for performance.
Issue Created By: Graham Neubig on Slack
The text was updated successfully, but these errors were encountered:
What problem or use case are you trying to solve?
Currently, there is no comprehensive leaderboard that effectively tracks the performance of open-sourced models within OpenHands. A clear leaderboard would facilitate comparison and improvement.
Describe the UX of the solution you'd like
An online sheet that continuously updates and tracks performance metrics for various models, possibly with the ability to click on links for deeper insights specific to each model.
Do you have thoughts on the technical implementation?
The leaderboard could utilize existing performance data stored in a central database, automatically populated and refreshed at intervals, ensuring real-time updates.
Describe alternatives you've considered
Using a static document or manual tracking methods, but these lack the dynamic features and timeliness that a real leaderboard would provide.
Additional context
This initiative was discussed in a recent thread on Slack with users expressing interest in better tracking mechanisms for performance.
Issue Created By: Graham Neubig on Slack
The text was updated successfully, but these errors were encountered: