Best Practice: ordering records about aerospike-client-go HOT 8 CLOSED

Have you considered using an array and putting the objects in it? How big is your array?

g1 := map[interface{}]interface{}{1: "My first group"}
g2 := map[interface{}]interface{}{2: "My second group"}
g3 := map[interface{}]interface{}{3: "My third group"}

bin := NewBin("groups", []interface{}{g1, g2, g3})
client.PutBins(wpolicy, key, bin)

rec, _ := client.Get(policy, key)
fmt.Println(rec.Bins["groups"])

from aerospike-client-go.

Thank you for your response. I have considered using an array and putting the objects in it, but that defeats the whole purpose of using a database in this scenario. The actual scenario is more advanced than the basic scenario (Group also contains a parent/child relation with other Groups). This means it'd be desirable to be able to select the Group using its PK.

But if I understand correctly, the "best practice solution" would be to store the sequence as an array in a bin?

The arrays consist of max. 20 items.

from aerospike-client-go.

Unless I'm missing something, the best practice solution would be sorting in client-side. 20 Items do not represent a size or performance risk.

from aerospike-client-go.

The thing to keep in mind with regards to Aerospike is that it is a distributed hash table, where the keys are not ordered.

Sounds like you should have a record per Group. You can create a secondary index on the bin containing the parent Group's id, so you can perform queries on all children of a given parent. You can also have the inverse relationship in the parent, and do a Batch Read of the child Group ids. With a small dataset, meaning thousands or less, then sorting on the application side will be quite trivial.

from aerospike-client-go.

To expand on Chris's answer, in a distributed database, data may come from several different nodes. As a result, even if each data set from an individual node is sorted, another entity with access to the combined data streams from all nodes is needed to sort the merged data (complications ensue regarding space, ...)

In your case and with constraints you mentioned, the best candidate to do the sort ends up being in your own application.

from aerospike-client-go.

Thank you for the suggestion to do the sorting application side.
I decided to save the index in a bin (after one-time sorting), making it possible to read everything coming through via a channel, and put it in the correct index in an array. That way, when looking at the array, everything is sorted.
As for the child-elements, I am using a Query/Filter to search for the parent-key, and using the saved index for the sorting.

I wonder what the performance difference is between regular queries / scans versus saving those results in bins and fetching those specficially.
i.e.: would it be better to Query for "parent = 5", or would it be better to select "key 4, key 6, key 8"? Perhaps if using a batch, the latter would be faster..?

from aerospike-client-go.

I'm dropping the last question I asked ... I am curious, but I should stop worrying about performance this much. The average latency for reading a record is close to nothing (I wrote a golang Benchmark for the first time of my life - turns out, fetching the entire tree structure from the database costs about 1ms), so I shouldn't worry that much unless I'm doing massive reads per second.
Thank you very much for your answers.

from aerospike-client-go.

No worries, we are here to help.

The standard first and quick answer is: Do a quick and dirty benchmark.

That aside, there is certain overhead when you start expanding the number of keys in batch requests. In your use case with ~20 elements, I think the cost will be relatively negligible, but you may have to find another solution if that number increases.

You're still sending more than 20 * 20 bytes per request, not considering the headers. But don't rush to optimize, let it show up first in your profiling charts first.

from aerospike-client-go.