Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Qdrant support vector delete by key word #989

Open
1 of 2 tasks
Wendong-Fan opened this issue Sep 25, 2024 · 4 comments · May be fixed by #1049
Open
1 of 2 tasks

[Feature Request] Qdrant support vector delete by key word #989

Wendong-Fan opened this issue Sep 25, 2024 · 4 comments · May be fixed by #1049
Assignees
Labels
enhancement New feature or request P0 Task with high level priority
Milestone

Comments

@Wendong-Fan
Copy link
Member

Required prerequisites

Motivation

currently we only support deleting by using id, it's hard to track, better support with deleting by key word

Solution

No response

Alternatives

No response

Additional context

No response

@Wendong-Fan Wendong-Fan added enhancement New feature or request P0 Task with high level priority labels Sep 25, 2024
@Wendong-Fan Wendong-Fan added this to the Sprint 14 milestone Oct 7, 2024
@koch3092
Copy link
Collaborator

koch3092 commented Oct 8, 2024

  • When initializing the Qdrant client, a global Client can be initialized to handle CRUD operations for collections.
  • Filter vectors using metadata.
  • Delete relevant collections by performing a fuzzy search on collection names (in conjunction with Redis).
  • Check if a collection exists by performing a fuzzy search on collection names (in conjunction with Redis) and matching the full name.

For the above requirements, consider the distinction and unification between local storage and self-hosted cloud deployment.

@Wendong-Fan
Copy link
Member Author

  • When initializing the Qdrant client, a global Client can be initialized to handle CRUD operations for collections.
  • Filter vectors using metadata.
  • Delete relevant collections by performing a fuzzy search on collection names (in conjunction with Redis).
  • Check if a collection exists by performing a fuzzy search on collection names (in conjunction with Redis) and matching the full name.

For the above requirements, consider the distinction and unification between local storage and self-hosted cloud deployment.

all doable

@koch3092
Copy link
Collaborator

thanks @Wendong-Fan , and I will take the task and develop

@koch3092 koch3092 linked a pull request Oct 13, 2024 that will close this issue
13 tasks
@koch3092 koch3092 linked a pull request Oct 13, 2024 that will close this issue
13 tasks
@koch3092
Copy link
Collaborator

  • When initializing the Qdrant client, a global Client can be initialized to handle CRUD operations for collections.
  • Filter vectors using metadata.
  • Delete relevant collections by performing a fuzzy search on collection names (in conjunction with Redis).
  • Check if a collection exists by performing a fuzzy search on collection names (in conjunction with Redis) and matching the full name.

For the above requirements, consider the distinction and unification between local storage and self-hosted cloud deployment.

I have completed the development and have a few clarifications regarding the requirements:

  • Collections cannot be filtered by metadata, and it's not recommended to integrate Redis for this purpose. Qdrant tools should maintain loose coupling.
  • Collections should not be deleted based solely on fuzzy search for collection types, as this approach is too risky. It’s necessary to ensure that the full collection_name is used to delete a collection.
  • If there are many collections with similar names, I think the vectors should be placed in the same collection, and classification should be handled using payload.
  • Self-hosted mode actually doesn't support concurrent searches. In the original design, this was addressed by managing through an in-memory list, which I believe is sufficient for local testing. In server mode, this issue doesn't need to be considered. Therefore, I believe the functionality between local storage and self-hosted Qdrant has been unified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request P0 Task with high level priority
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

2 participants