how to create ids_vectors.p and index file? about faiss-web-service HOT 2 CLOSED

Hey,

So this project is based on Facebook's faiss.

I would advice you to have a look at their getting started guide. That should help you understand how to build the index file.

The ids_vectors.p is one implementation, you don't actually need it. What you need is your own config file. Two examples can be found in faiss_web_service_config.

The reason behind the ids_vectors.p is that faiss currently only has query by vector. If you wish to query by id, you need a way to convert an id to a vector. I used a simple dictionary that I pickled, but feel free to use any storage solution like a file (csv, json, ...), or even a database (sqlite, mysql, ...).

Based on your project's description, you need a way to convert your images to vectors. You are going to have to do that on your own as it really depends what you want to extract from your image. Once that is done, you should have an array of ids and vectors. That should be used to create your faiss index, and populate your storage solution.

If your index is periodically updated, you should look at faiss_index_urllib. GET_FAISS_RESOURCES should be set to download the newest index, GET_FAISS_ID_TO_VECTOR should be changed based on the storage solution you have picked, and lastly, UPDATE_FAISS_AFTER_SECONDS should be set to your refresh period.

I hope that answered all your questions. If you have more, don't hesitate to ask.

from faiss-web-service.

tks!

from faiss-web-service.