Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

105 Open 451 Closed
105 Open 451 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Abnormally low values for NanoBEIR benchmark
#1627 opened Dec 24, 2024 by minsik-ai
1
Adding enhanced cross-lingual retrieval benchmark by merging retrieval pools from different languages
#1625 opened Dec 24, 2024 by yjoonjang
Integrate ModernBERT new-model Questions related to adding a new model to the benchmark
#1624 opened Dec 23, 2024 by Samoed
3
Leaderboard: Add ONNX and GGUF to frameworks to allow us to filter based on these leaderboard issues related to the leaderboard
#1622 opened Dec 22, 2024 by KennethEnevoldsen
1
Normalize image encoders interface to the text encoder interface
#1606 opened Dec 17, 2024 by KennethEnevoldsen
leaderboard v2 - add autoupdate leaderboard issues related to the leaderboard
#1605 opened Dec 16, 2024 by KennethEnevoldsen
3
Investigate performance discrepancies in gte-Qwen and NV-embed models
#1600 opened Dec 16, 2024 by isaac-chung
5
Add Support for Index-Based Retrieval for ColBERT enhancement New feature or request
#1593 opened Dec 14, 2024 by sam-hey
Integrate UniBench mieb The image extension of MTEB new-benchmark
#1590 opened Dec 14, 2024 by Samoed
Integrate ChemTEB new-benchmark
#1585 opened Dec 13, 2024 by Muennighoff
5
[MIEB] Update VLM2Vec model mieb The image extension of MTEB
#1582 opened Dec 12, 2024 by XMHZZ2018
1
Unable to reproduce the metrics of BAAI/bge-reranker-large on the T2Reranking dataset.
#1581 opened Dec 12, 2024 by HuDi2018
4
Leaderboard 2.0: Missing results leaderboard issues related to the leaderboard
#1571 opened Dec 9, 2024 by x-tabdeveloping 100+
11
Track contamination in new leaderboard leaderboard issues related to the leaderboard
#1556 opened Dec 5, 2024 by Muennighoff
1 of 4 tasks
7
New Italian tasks
#1553 opened Dec 4, 2024 by MattiaSangermano
5
MIRACLRetrieval results are missing for most models. leaderboard issues related to the leaderboard
#1550 opened Dec 4, 2024 by Samoed
17
Add bge-m3 dense, sparse, multi-vector
#1529 opened Nov 29, 2024 by yjoonjang
2
[MIEB] Add new multimodal retrieval tasks mieb The image extension of MTEB
#1523 opened Nov 28, 2024 by izhx
8
leaderboard 2.0: Performance on subtasks (Bright retrieval) leaderboard issues related to the leaderboard
#1517 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard v2.0: disagreement in scores between v2.0 and v1.0 leaderboard issues related to the leaderboard
#1516 opened Nov 27, 2024 by KennethEnevoldsen
leaderboard 2.0: Add missing models leaderboard issues related to the leaderboard
#1515 opened Nov 27, 2024 by KennethEnevoldsen
15
leaderboard 2.0: Some task <> benchmark associations seem wrong leaderboard issues related to the leaderboard
#1514 opened Nov 27, 2024 by KennethEnevoldsen
Remove nan scores on leaderboard leaderboard issues related to the leaderboard
#1509 opened Nov 27, 2024 by KennethEnevoldsen
Add mixedbread-ai/mxbai-embed-xsmall-v1 and mixedbread-ai/mxbai-embed-2d-large-v1
#1502 opened Nov 26, 2024 by KennethEnevoldsen
Add voyage-3 and voyage-3-lite new-model Questions related to adding a new model to the benchmark
#1501 opened Nov 26, 2024 by KennethEnevoldsen
ProTip! Exclude everything labeled bug with -label:bug.