Is it good approach if i create indexes of every Category?

[Talhaz]

Let Suppose I have a fashion clothing store, and to get accurate results should I make indexes of each category like [Tops, Blouse, jeans] ?
Because I tested it, and I created a folder include every 200 images of different categories but it did not show good results to me
So I thought I should make indexes of every category and after classification by CNN model then I will pass the image to clip backend
But in your case, the OpenAI clip has the same model that gives almost 99.9% accurate similar items, Can u Explain to me how it works guide me about it?

[rom1504]

can you explain what feature you want to implement?

[Talhaz]

Clip model is a general similarity model that works for any category as you can see in the demo https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fsplunk.vra.ro&index=laion_400m_128G
It provides a good similarity estimation between an image and a text.

Are you trying to implement a text search on a dataset of image ? If so then yes using a single image index can work.

What kind of query and results did you have exactly ?

Creating several indices can make sense if you know in advance the kind of items you want to retrieve but if you are working with a very small dataset of size 200, it doesn't make a lot of sense.

I am working on a project which task is to first detect objects in an image and then search for similar images for that object. yeah Detector already classify object return a class from where it belongs, that's why it would be easier and accurate if I create indices of every category
Yeah, images would be approximately for one category could be 3K I know it's not good .
So My question is is it a good approach if I create different indices for every image category ??

[rom1504]

Clip is probably not the best model for image-image similarity. You could try other models for example any classification trained model (see https://github.com/rom1504/image_embeddings ) or something like https://github.com/facebookresearch/dino

About creating one index per category you can do that yes. Another way is to post filter the results after the knn search

[Talhaz]

Thanks, I tried this one "https://github.com/rom1504/image_embeddings " In this, I didn't understand how to get similarities from a new input image because it runs only on its generated image embeddings, can u improve this functionality?

I will look into this too "https://github.com/facebookresearch/dino", thanks for ur help

[Talhaz]

rom1504/image_embeddings#25 this is my question in that repo

[rom1504]

There, you could use efficientnet to generate the embedding of one image (https://github.com/qubvel/efficientnet)

You didn't explain yet clearly the problem you're having.