Add pharia data english #75
Conversation
|
Can you add implementation of your model to mteb repo? |
|
For now we would keep it with trust_remote_code in the checkpoint repo if possible. |
There was a problem hiding this comment.
A few minor things to clarify
For now we would keep it with trust_remote_code in the checkpoint repo if possible.
We need to unify codebases from CausalLM model and put it on HF too. I don't think that will happen until EOY so ideally I would like to merge this branch before.
This is perfectly fine. However a PR to mteb can simply be the metadata without the implementation (or the implementation using an API).
| "open_weights": true, | ||
| "public_training_data": null, | ||
| "public_training_code": true, | ||
| "framework": [], |
There was a problem hiding this comment.
| "framework": [], | |
| "framework": ["API"], |
Assuming?
| "public_training_data": null, | ||
| "public_training_code": true, | ||
| "framework": [], | ||
| "reference": null, |
| "public_training_code": true, | ||
| "framework": [], | ||
| "reference": null, | ||
| "similarity_fn_name": null, |
There was a problem hiding this comment.
| "similarity_fn_name": null, | |
| "similarity_fn_name": "cosine", |
| "dataset_revision": "e8379541af4e31359cca9fbcf4b00f2671dba205", | ||
| "evaluation_time": 25.99530529975891, | ||
| "kg_co2_emissions": null, | ||
| "mteb_version": "1.14.21", |
There was a problem hiding this comment.
Is there a reason why the latest version is not used?
Checklist
make test.make pre-push.