Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always replicate vectors metadata #2495

Merged
merged 1 commit into from
Sep 25, 2024
Merged

Conversation

javitonino
Copy link
Contributor

Vectors metadata was not being replicated when no segments changed. This is wrong because of two things:

  • Deletions are only kept in metadata, so deleting a resource wouldn't be replicated.
  • Empty indexes / empty KBs won't replicate anything and fail to open

The second case used to work before #2492 because we created a dummy shard which included some empty metadata files for the vectors index.

Copy link

codecov bot commented Sep 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.19%. Comparing base (4b324d2) to head (cf2cf24).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2495      +/-   ##
==========================================
- Coverage   86.23%   86.19%   -0.05%     
==========================================
  Files         375      375              
  Lines       23395    23395              
==========================================
- Hits        20175    20165      -10     
- Misses       3220     3230      +10     
Flag Coverage Δ
nucliadb 73.97% <ø> (-0.02%) ⬇️
nucliadb-ingest 29.08% <ø> (ø)
nucliadb-reader 25.01% <ø> (ø)
nucliadb-search 37.24% <ø> (ø)
nucliadb-standalone 47.00% <ø> (ø)
nucliadb-train 45.62% <ø> (ø)
nucliadb-writer 39.96% <ø> (ø)
nucliadb_dataset 55.31% <ø> (ø)
nucliadb_models 85.08% <ø> (ø)
nucliadb_sdk 80.11% <ø> (ø)
nucliadb_sidecar 87.24% <ø> (-1.79%) ⬇️
nucliadb_telemetry 86.55% <ø> (ø)
nucliadb_utils 84.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@javitonino javitonino merged commit 56fa47d into main Sep 25, 2024
60 checks passed
@javitonino javitonino deleted the replicate_vectors_metadata branch September 25, 2024 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants