You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current implementation of ColBERT only utilizes the MaxSim operator for retrieval and does not leverage an index. While this works for smaller datasets, it is not scalable for large datasets, as the resource consumption becomes impractical.
To enable efficient retrieval for large-scale datasets, it is essential to integrate an indexing mechanism. This will ensure that resource consumption remains manageable while maintaining retrieval performance.
PyLate plans to introduce the PLAID Index to address this limitation. This index is designed to optimize retrieval tasks for larger datasets. For details on the current progress, refer to the PyLate Issue.
The text was updated successfully, but these errors were encountered:
KennethEnevoldsen
changed the title
Enhancement: Add Support for Index-Based Retrieval for ColBERT
Add Support for Index-Based Retrieval for ColBERT
Dec 14, 2024
The current implementation of ColBERT only utilizes the MaxSim operator for retrieval and does not leverage an index. While this works for smaller datasets, it is not scalable for large datasets, as the resource consumption becomes impractical.
To enable efficient retrieval for large-scale datasets, it is essential to integrate an indexing mechanism. This will ensure that resource consumption remains manageable while maintaining retrieval performance.
PyLate plans to introduce the PLAID Index to address this limitation. This index is designed to optimize retrieval tasks for larger datasets. For details on the current progress, refer to the PyLate Issue.
The text was updated successfully, but these errors were encountered: