allow for connecting to multiple cassandra cluster nodes about marina HOT 8 CLOSED

Hi,
I finally got around implementing a proper pooling strategy for marina. It will now first connect to a bootstrap node to lookup the system.peers table. From that table we gather all peers and open connections to all of them. You can filter nodes by providing filter_dc. Currently, the only mode supported is random picking, I plan to eventually add token-aware picking.

PR: #30

Cheers,
LP

from marina.

@lpgauth noticed you had token aware load balancing on the roadmap. Is that going to be open sourced in the future?

In your production setup right now, how are you currently handling spreading the requests to multiple nodes in production to avoid overloading a single coordinator node?

DNS or some other load balancing software?

thanks in advance!

from marina.

@lpgauth do you have any update on this?

from marina.

@unbalancedparentheses hey, haven't had much time to play with this as we're moving from C* to ScyllaDB and I didn't want to change too many variables at once. Last time I checked the only remaining issue was to fix the prepared statement cache.

from marina.

@lpgauth so if we don't use prepared statements it should be working fine, right? we are moving away from cqerl because it keeps on trying to connect to dead cassandra nodes of a cluster, it doesn't backlist the dead cassandra nodes until they get online.

from marina.

@unbalancedparentheses yes, it should, but it's only been testing on one node in production for maybe a week so there might be other bugs.

from marina.

This finally came back to the top of my list. Still WIP, but coming soon.
#34

from marina.

This has finally been merged 🎉

from marina.