Embedded - Scaling out? about tiedot HOT 3 CLOSED

Sorry - I'm afraid that tiedot does not support it at the moment. There won't be any problem opening multiple database instances, but once schema is changed, the read-only instances don't realize it.

There is an easy workaround though, we can send signals to the read-only instances to notify reloading of schema.

BTW - please use the "nextgen" branch for evaluation, it has important fixes to some concurrency related issues currently present in master branch.

from tiedot.

A reset for schema changes isn't too bad... I could live with that.

You mention read-only instances. What I was proposing is each instance would have read/write access. Hence, scaling out to say 2 or 3 nodes for "redundancy" reasons, as well as high load reasons.

from tiedot.

Oh I see.

Take a look at nextgen commit history, you'll see that I made an attempt at partitioning document data and indexes over multiple data structure servers. Scalability came out, however the overall performance was not satisfactory - the use of message passing over network communication is too significant. In the end, the scalable implementation turns out to be 12x slower than the current one.

So I dropped the idea (for now) because majority of tiedot users don't have over 100GB of data.

I could use some more scalable programming exercises =)

from tiedot.