Is this of because stuck in some local minima where no neighbors are improving distance and the search queue is empty ?

Also is this somehow related to data dimensionality as well ? for example for a fixed M and search ef , ef_construction param , the chances of running into these kind of errors are more probable large high dimensional data or similar scale data but lower dimension

Hi @prateekpatelsc,

That might be connected to duplicates in the data. The duplicates are useless for search as each component can be substituted with a single element decreasing the index size and complexity (and one can control them on the client side doing a search before the insertion of a batch) and incremental graph-based approaches can suffer from them due to loss of connectivity in the graph if the size of a maximum duplicate component is much larger than the number of links M (and it is hard to check for duplicates for inner product inside the algorithm).

Can you check if it has large components of duplicates?