You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Seems like we currently need to manually update the MTEB leaderboard (@orionw I believe you fixed the old one. I believe we really only need to restart the space daily)
The text was updated successfully, but these errors were encountered:
Yes, definitely! I'm trying to figure out how the leaderboard gets its data, seems like there's a function in MTEB now that provides the results, is that right? (also cc'ing @x-tabdeveloping). The way I restarted it daily before was by updating the results file in the repo daily and then the results repo would re-start.
If I understand correctly, would the best way to do this be to auto-restart the leaderboard on MTEB merges? Or perhaps MTEB results github merges? I don't see the connection to the results github, but I haven't traced it all the way back yet.
So the way it currently happens is that I download the results and cache them everytime the Docker container is rebuilt.
The results live in the specific Docker image, so the app needs a rebuild every time we update.
I know this isn't optimal, but we can look into ways of making this more seamless
Seems like we currently need to manually update the MTEB leaderboard (@orionw I believe you fixed the old one. I believe we really only need to restart the space daily)
The text was updated successfully, but these errors were encountered: