Fix IndexIVFFastScan reconstruct_from_offset method #4095
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves issue #4089 - IndexIVFPQFastScan crashes with certain nlist values
The
reconstruct_from_offset
method inIndexIVFFastScan
was incorrectly reconstructing vectors, causing crashes when thenlist
parameter was not byte-aligned (e.g. 100 instead of 256).The root cause was that the
list_no
(Voronoi cell number) was not being properly encoded into thecode
vector before passing it to thesa_decode
function. This resulted in invalidlist_no
values being read insa_decode
, triggering the assertion failure'list_no >= 0 && list_no < nlist'
whennlist
in some cases.This PR fixes the issue with the following changes to
reconstruct_from_offset
:list_no
into the beginning of thecode
vector using the existingencode_listno
methodBitstringWriter
after the coarse code portion ofcode
(shifted bycoarse_code_size()
bytes)sa_decode
After these changes:
nlist
valueIndexIVFPQ
Fixes #4089
Please review and let me know if any changes are needed. Thanks!