Search Implementation - Query Time Sorting about dragonfly HOT 4 OPEN

Hi. Thanks for trying out our new features!

We'll try to add this feature in the nearest feature, probably even in the next version. As you already pointed out, the performance will be much worse, because the values need to be fetched from the entries. But as long as your queries don't match thousands of items, it should be acceptable

from dragonfly.

@dranikpg

Not to dictate how you build this feature
But my suggestion would be that I would declare these sort fields as their own mini index, so that filtering happens on the main index and the sorting is done on the subsidiary index?

Atleast then I wouldn't see such a massive hit to the performance of the query.

Either way, you're my hero

from dragonfly.

@claridgicus , we are going to implement sort during the query time, i.e. according to your original suggestion:

Ideally I would like to implement a sort on a property of my documents at query time, ideally, after the search module has actually retrieved the search results dragonfly would sort the limited subset based on my field (which in my case is a property of an object on my document which is an integer)

if we build a precomputed index (secondary or not) , it will take space, which will bring us to the original issue you had with 1000 indices taking too much space.

from dragonfly.

@dranikpg I think you've got exactly my issue.

Right now so you understand my usecase

I have a multitennanted redis cluster (in 4 regions)

For each Tennant

I load in JSON format

• Every Collection the Tennant has
• Every Product the Tennant has
• A view of every product (for some application logic where I implode many references into a composite)

I build

• A "Search" index which is what I use for my Full Text Search requirements
• Many "Collection" indexes (this is my problem) which reference their own sort orders

I have many, many, many prefixes

I want an index to partition my tenants data at the boundary of their prefix so I'm not indexing everyone's stuff everywhere.

from dragonfly.