How to impl a custom store? about tantivy HOT 7 CLOSED

That's not possible currently. What's your use case?

from tantivy.

I want every doc fully stored, so I dont need to query database using doc_id, and it's easy for the client to query data if we have a database as a store like tikv or mongodb etc., and it will also decouple the index storage from data storage.

from tantivy.

You can fully store the documents in tantivy, so you don't need to query a database using doc_id. doc_id or DocAddress is something that's internal to tantivy and should not be used external. It's possible to compress with zstd in the docstore for better compression.

from tantivy.

Thanks! Normally, how much data it can store without performance downgrade?

from tantivy.

You could use a fast field or a stored field to store a primary id from an external DB.

from tantivy.

Thanks! Normally, how much data it can store without performance downgrade?

Performance downgrade on indexing? It depends on your doc store settings (blocksize, compression algorithm, compression level). We use a separate thread to compress the doc store, so you may not notice any difference until doc store compression gets slower than regular search indexing.

Search performance is not really impacted by this.

from tantivy.

Thanks, very helpful!

from tantivy.