Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MIEB] Update VLM2Vec model #1582

Open
XMHZZ2018 opened this issue Dec 12, 2024 · 1 comment
Open

[MIEB] Update VLM2Vec model #1582

XMHZZ2018 opened this issue Dec 12, 2024 · 1 comment
Labels
mieb The image extension of MTEB

Comments

@XMHZZ2018
Copy link

Thanks for the great work on MTEB and MIEB, as well as for including our VLM2Vec!

We have released some new version of VLM2Vec. Could you please let me know how to update it here? I’d be happy to work on this if you could provide me some guidance.

It seems I need to update this script. Please let me know if my understanding is correct: https://github.com/embeddings-benchmark/mteb/blob/mieb/mteb/models/vlm2vec_models.py

Also, it seems the current instruction is "<|image_1|> Represent the given image.", I'm wondering whether MIEB would support task-specific instruction. Thanks!

@KennethEnevoldsen
Copy link
Contributor

Thanks for the interest @XMHZZ2018, we are working on reformulating the MIEB interface (#1551). This reformulation should allow for prompts, multiple images pr. document etc.

cc @gowitheflow-1998

@isaac-chung isaac-chung added the mieb The image extension of MTEB label Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mieb The image extension of MTEB
Projects
None yet
Development

No branches or pull requests

3 participants