Automate cluster joining about docker-ejabberd HOT 18 CLOSED

I don't know my way around bash scripting, so it's very hard for me to implement this. Would you be able to implement this, @rroemhild ? Even just part 1 would be a very useful feature because now it seems to be completely impossible to join other nodes, because they cannot connect to each other based on their hostnames.

I cannot update /etc/hosts during docker build phase, since docker will overwrite this file when the build phase completes.

Since I'm running the containers via tutum (http://tutum.co), I can also not use the --add-host parameter when doing 'docker run ...'.

The solution I propose under topic 1 seems to be the only easy implementation to do this, would you agree?

from docker-ejabberd.

I just pushed my cluster test to the cluster branch but run into another problem. I published it so you can take a look.

For topic 1 I think this could be solved with tools like kubernetes or the Docker Orchestrating tools. They should handle the container linking, like docker-compose does.

Another solution for this could be etcd from the CoreOS people. Etcd running in an extra container and every docker-ejabberd container announce itself to etcd and look out for a master node in etcd. I think to manual announce hosts, etcd is a solid solution for topic 1. This could also be achieved with an redis instance or something else, if orchestrating is not an option.

I will think about automating clustering the next days, it is a new topic for me.

from docker-ejabberd.

I'm getting closer 😄

With some work around the issues processone/ejabberd#498 and processone/ejabberd#497 I get an two node cluster up and running with docker-compose. See cluster branch.

from docker-ejabberd.

Cool!

Is this 'multiple master' or 'master-slave'? And does this work across physical hosts (e.g. multiple different physical machines, probably in the same network private though)?

from docker-ejabberd.

This is an 'multiple master' cluster, but only with two hosts because docker-compose can't cross-link container. I'm thinking and experimenting with an elegant host/node auto discover solution.

If the physical machines can talk to each other on the required ports it should work in the same network or over multiple locations.

I've setup a tutum.co and aws account to test the cluster feature in a real world scenario.

from docker-ejabberd.

Awesome, I'm looking forward to your findings!

from docker-ejabberd.

Finally I've add an example how this image could be used to setup a multi-node cluster based on different tools.

On tutum i.e. you can't set dnsname and nameservers, but you can start with the scripts in this example or write your own start-scripts and use the tools provided by tutum.

from docker-ejabberd.

Do you have a working example on how cluster ejabberd containers auto?
I would be interested in seeing you accomplished that.

from docker-ejabberd.

You can start with the compose-cluster example.

from docker-ejabberd.

I know this is old thread but is this solution is good for multi hosts?
host1 - ejabberd@node1
host2 - ejabberd@node2

from docker-ejabberd.

I've deployed ejabberd cluster on k8s by taking examples from the scripts used by rroemhild. but i observed that after 10 requests container was failing. After lot of tweeks i found when i used containers with host-network, it worked fine. although it is working fine as of now. I'm not sure what was the actual cause of cluster not accepting more requests. I've tried using 3 and 5 node cluster.

Below was the error i was getting for more than 10 requests without using host-network for containers:
INFO: Starting with 1 request processors
Jun 19, 2018 6:36:23 PM org.jivesoftware.smack.AbstractXMPPConnection callConnectionClosedOnErrorListener
WARNING: Connection XMPPBOSHConnection[not-authenticated] (4) closed with error
org.igniterealtime.jbosh.BOSHException: Could not obtain response
at org.igniterealtime.jbosh.ApacheHTTPResponse.awaitResponse(ApacheHTTPResponse.java:246)
at org.igniterealtime.jbosh.ApacheHTTPResponse.getBody(ApacheHTTPResponse.java:187)
at org.igniterealtime.jbosh.BOSHClient.processExchange(BOSHClient.java:1114)
at org.igniterealtime.jbosh.BOSHClient.processMessages(BOSHClient.java:990)
at org.igniterealtime.jbosh.BOSHClient.access$300(BOSHClient.java:100)
at org.igniterealtime.jbosh.BOSHClient$RequestProcessor.run(BOSHClient.java:1719)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.NoHttpResponseException: 1

from docker-ejabberd.

@Joginder22 Any working example of cluster on k8s?

from docker-ejabberd.

from docker-ejabberd.

@Joginder22 How did you solve the ejabberd nodes joining cluster in k8s cluster?
I've done that using pod DNS settings, but I believe there is a more elegant way to do that

from docker-ejabberd.

from docker-ejabberd.

Hi @Joginder22
thanks for your reply. it's very similar to my configuration. I'm using pod dns setting which is available on k8s starting 1.10+ in beta version but how you are making new nodes to autojoin cluster? I did it this way:

1. added root ejabberd node as separate pod
2. created replicaset which takes control of this node and creates new nodes based on predefined params
3. used script inside container to connect to root node once ejabberd started
   This allows to scale replica set up and down and new nodes are automatically joining the existing cluster aka ejabberd root node.

from docker-ejabberd.

@Joginder22 unfortunately I have some issues, when new node joins the cluster, it hangs without any error or warning log until ejabberd restarted. But the thing is that when I run 'ejabberdctl list_cluster' command the new node is there even in freezed state. I want to resolve this issue. any idea why this can happen?

from docker-ejabberd.

Hi @Hayyk @Joginder22

Let me ask you guys! have you solved the issues with the cluster nodes joining automatically the cluster?

I was looking for an implementation on kubernetes for ejabberd. unfortunatelly there ins't much documented. After some blogs and also the discussion you guys had here, I'm now able to deploy a cluster of ejabberd nodes that join automatically. there some improvements that can be made. Like join the cluster if the connection with the cluster has been lost, but over all the implementation is easy to replicate.

I am going to suggest to @rroemhild that implementation documentation to be available here. I think that would be a great to people on the community who use ejabberd and are migrating to a Kubernetes env.

from docker-ejabberd.