Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

108 Open 382 Closed
108 Open 382 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[mieb] add support for LLM2CLIP
#1458 opened Nov 14, 2024 by SaitejaUtpala
[mieb] TIGER-Lab/VLM2Vec-Full torch.cat(): expected a non-empty list of Tensors
#1457 opened Nov 14, 2024 by Muennighoff
[mieb] TIGER-Lab/VLM2Vec-LoRA fails with dtype issue
#1456 opened Nov 14, 2024 by Muennighoff
[mieb] royokong/e5-v on EDIST2ITRetrieval 'NoneType' object has no attribute 'dtype'
#1455 opened Nov 14, 2024 by Muennighoff
[mieb] add support for voyage-multimodal-3:
#1446 opened Nov 14, 2024 by Muennighoff
"KeyError: 'document' not found and no similar keys were found.
#1445 opened Nov 13, 2024 by LeMoussel
5
Leaderboard: Add radar chart
#1443 opened Nov 13, 2024 by KennethEnevoldsen
Model metas: E5 instruct not listed as sbert compatible
#1442 opened Nov 12, 2024 by x-tabdeveloping
Leaderboard: Add query parameters to link leaderboard issues related to the leaderboard
#1440 opened Nov 12, 2024 by x-tabdeveloping
[MIEB] Flickr30K not matching openAI ViT B32 results mieb The image extension of MTEB
#1434 opened Nov 11, 2024 by isaac-chung
2
Leaderboard: Add an Overview leaderboard leaderboard issues related to the leaderboard
#1432 opened Nov 11, 2024 by KennethEnevoldsen
2
Enable Code Carbon by default in evaluation runs
#1427 opened Nov 10, 2024 by isaac-chung
1
6
Add new benchmark MAIR
#1426 opened Nov 10, 2024 by sunnweiwei
1
Evaluating LLM2Vec fails
#1422 opened Nov 9, 2024 by Muennighoff
@sivareddyg @vaibhavad
1
Evaluating nomic on BornholmBitextMining fails
#1421 opened Nov 9, 2024 by Muennighoff
[mieb] google/siglip-large-patch16-384 fails on ImageNet10Clustering mieb The image extension of MTEB
#1417 opened Nov 9, 2024 by Muennighoff
1
[mieb] 'VLM2VecWrapper' object has no attribute 'calculate_probs' mieb The image extension of MTEB
#1416 opened Nov 9, 2024 by Muennighoff
@isaac-chung
1
[mieb] Salesforce/blip-image-captioning-large buggy mieb The image extension of MTEB
#1415 opened Nov 8, 2024 by Muennighoff
1
descriptive stats missing for multiple datasets
#1412 opened Nov 8, 2024 by KennethEnevoldsen
1
Running models before leaderboard release leaderboard issues related to the leaderboard
#1411 opened Nov 8, 2024 by x-tabdeveloping
12 tasks
6
Flexible post-evaluation filtering
#1410 opened Nov 8, 2024 by gnatesan
2
[mieb] EVA on CVBenchCount fails mieb The image extension of MTEB
#1409 opened Nov 7, 2024 by Muennighoff
1
@isaac-chung
1
Integrate ArabicMTEB new-dataset
#1408 opened Nov 7, 2024 by KennethEnevoldsen
Error when creating task (SICK-BR-STS, XStance)
#1407 opened Nov 7, 2024 by AlexeyVatolin
Overview issue: Leaderboard 2.0 release leaderboard issues related to the leaderboard
#1405 opened Nov 7, 2024 by KennethEnevoldsen
3 of 8 tasks
leaderboard2.0
1
ProTip! What’s not been updated in a month: updated:<2024-10-14.