Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vectorstores: add vertex vector search datapoints deletion #559

Merged
merged 1 commit into from
Oct 29, 2024

Conversation

Pablito2020
Copy link
Contributor

This PR is very similar to #331, but fully implemented and with metadata filtering support.

Now, you can delete documents from VectorVertexAiSearch with:

Delete datapoints by id

ids = [] # in case you know the datapoints id's you can add them here!
vector_store.delete(ids)

Delete datapoints via metadata values.

The more powerful feature is the ability to delete datapoints based on metadata. For example, to delete all datapoints originating from a specific source file:

file_to_delete = "testing.pdf"
# This will delete all the datapoints that have indexes from this source file!
vector_store.delete(metadata={"source": file_to_delete})

You can filter by any metadata key-value pair, not just the source file. For instance, to delete all the documents 15th page:

# Delete all the 15th pages
vector_store.delete(metadata={"page": 15}) 

@lkuligin
Copy link
Collaborator

@Pablito2020 thanks a lot for your contribution! Could you finalize it and fix lint errors, please?

@Pablito2020
Copy link
Contributor Author

@Pablito2020 thanks a lot for your contribution! Could you finalize it and fix lint errors,

The feature is finalized and works as expected, I've just fixed the linter errors.

The CI is complaining about some LLM tests (which I didn't touch) and they don't test the vector search vector store implementation.

@lkuligin lkuligin merged commit 6aa6686 into langchain-ai:main Oct 29, 2024
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants