Giter VIP home page Giter VIP logo

neo4j-kubernetes's Introduction

Neo4j on Kubernetes

This is a first attempt at getting Neo4j 3.1 to run on Kubernetes.

You can either setup Kubernetes locally using minikube or on one of the cloud providers e.g. Google Compute Engine

Once you’ve got that setup you can run the following command to see where Kubernetes is running. e.g. on my machine

$ kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443
KubeDNS is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Ok. Now we’re ready to install Neo4j.

Core Servers

First let’s install the Core servers. We’d typically have a small number of Core servers in a cluster and they can handle both read and write traffic.

Run the following commands to create the Core servers:

$ ./neo4j.sh
...
persistentvolume "pv0" created
persistentvolumeclaim "datadir-neo4j-core-0" created
persistentvolume "pv1" created
persistentvolumeclaim "datadir-neo4j-core-1" created
persistentvolume "pv2" created
persistentvolumeclaim "datadir-neo4j-core-2" created
$ kubectl create -f neo4j_svc.yaml
service "neo4j" created
$ kubectl create -f neo4j_core.yaml
statefulset "neo4j-core" created

We can check that those servers have spun up by running:

$ kubectl get pods
NAME           READY     STATUS    RESTARTS   AGE
neo4j-core-0   1/1       Running   0          23s
neo4j-core-1   1/1       Running   0          19s
neo4j-core-2   1/1       Running   0          16s

And we can check that Neo4j is up and running by running:

$ kubectl logs neo4j-core-2
Starting Neo4j.
2016-12-07 11:23:23.133+0000 INFO  Starting...
2016-12-07 11:23:24.551+0000 INFO  Bolt enabled on 0.0.0.0:7687.
2016-12-07 11:23:24.566+0000 INFO  Initiating metrics...
2016-12-07 11:23:24.820+0000 INFO  Waiting for other members to join cluster before continuing...
2016-12-07 11:23:45.546+0000 INFO  Started.
2016-12-07 11:23:45.700+0000 INFO  Mounted REST API at: /db/manage
2016-12-07 11:23:46.683+0000 INFO  Remote interface available at http://neo4j-core-2.neo4j.default.svc.cluster.local:7474/

Neo4j also exposes the cluster topology via a procedure:

$ kubectl exec neo4j-core-0 -- bin/cypher-shell "CALL dbms.cluster.overview()"
id, addresses, role
"484178c4-e7ae-46c9-a7d5-aaf6250efc7f", ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"
"0acdb8dd-3bb1-4c76-bbe7-213530af1d23", ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474"], "LEADER"
"ca9b954c-c245-418f-9803-7790d36efdd9", ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"

Bye!

Read Replicas

Now that we’ve got the Core Servers up and running let’s add some Read Replicas. Read Replicas are used to scale out reads and they don’t accept any write operations.

Run the following command to create a single Read Replica:

$ kubectl create -f neo4j_rr.yaml
replicationcontroller "neo4j-replica" created

If we query for the list of pods now we’ll see that it’s been added to the list:

$ kubectl get pods
NAME                  READY     STATUS    RESTARTS   AGE
neo4j-core-0          1/1       Running   0          4m
neo4j-core-1          1/1       Running   0          4m
neo4j-core-2          1/1       Running   0          4m
neo4j-replica-rdwsh   1/1       Running   0          9s

Let’s check if the replica has joined the cluster and is ready to go:

$ kubectl logs neo4j-replica-rdwsh
Starting Neo4j.
2016-12-07 11:27:31.388+0000 INFO  Starting...
2016-12-07 11:27:33.072+0000 INFO  Bolt enabled on 0.0.0.0:7687.
2016-12-07 11:27:33.094+0000 INFO  Initiating metrics...
2016-12-07 11:27:39.693+0000 INFO  Started.
2016-12-07 11:27:39.878+0000 INFO  Mounted REST API at: /db/manage
2016-12-07 11:27:41.172+0000 INFO  Remote interface available at http://neo4j-replica-rdwsh:7474/

Yep, looks good!

Let’s check that Neo4j knows about our new server:

$ kubectl exec neo4j-core-0 -- bin/cypher-shell "CALL dbms.cluster.overview()"
id, addresses, role
"484178c4-e7ae-46c9-a7d5-aaf6250efc7f", ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"
"0acdb8dd-3bb1-4c76-bbe7-213530af1d23", ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474"], "LEADER"
"ca9b954c-c245-418f-9803-7790d36efdd9", ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"
"00000000-0000-0000-0000-000000000000", ["bolt://neo4j-replica-rdwsh:7687", "http://neo4j-replica-rdwsh:7474"], "READ_REPLICA"

Bye!

It does indeed.

Now let’s scale up to 3 read replicas:

$ kubectl scale rc neo4j-replica --replicas=3
replicationcontroller "neo4j-replica" scaled

And give it a few seconds and Neo4j will know about those servers as well:

$ kubectl exec neo4j-core-0 -- bin/cypher-shell "CALL dbms.cluster.overview()"
id, addresses, role
"484178c4-e7ae-46c9-a7d5-aaf6250efc7f", ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"
"0acdb8dd-3bb1-4c76-bbe7-213530af1d23", ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474"], "LEADER"
"ca9b954c-c245-418f-9803-7790d36efdd9", ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474"], "FOLLOWER"
"00000000-0000-0000-0000-000000000000", ["bolt://neo4j-replica-48r7z:7687", "http://neo4j-replica-48r7z:7474"], "READ_REPLICA"
"00000000-0000-0000-0000-000000000000", ["bolt://neo4j-replica-j28g0:7687", "http://neo4j-replica-j28g0:7474"], "READ_REPLICA"
"00000000-0000-0000-0000-000000000000", ["bolt://neo4j-replica-rdwsh:7687", "http://neo4j-replica-rdwsh:7474"], "READ_REPLICA"

Bye!

Bit of explanation

For the core servers we’re using a Kubernetes beta feature introduced in 1.5.0 called StatefulSets. This is helpful because Neo4j Core Servers need to discover each other so that they can take participate in a consensus commit algorithm. We therefore need to know the hostnames of these servers so that we can specify it in the config file.

For the read replicas we don’t have this constraint so we can use the standard replication controller approach. Read Replicas list the hostnames of Core Servers in their config file.

neo4j-kubernetes's People

Watchers

avatar avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.