This is a project inspired by Carlos Santana's knative-kind, created with ❤️.
With the help of KIND(K8s IN Docker), KGonK helps us provision a dedicated ephemeral K8S cluster with all dependencies inside Docker, including:
- A dynamic storageClass provider
- All third party dependencies of nebula-operator
- Nebula-Operator Pods in namespace:
nebula-operator-system
- The Nebula Graph Cluster Pods in default namespace
- A nodePort service exposing
0.0.0.0:30000
of the host mapped tographd:9669
Just call the following one liner from your Linux Machine:
curl -sL nebula-kind.siwei.io/install.sh | bash
Then you may see something like this:
Download the basketball player dataset and import it to the nebula graph:
wget https://docs.nebula-graph.io/2.0/basketballplayer-2.X.ngql
#### Modify vid format to numeric due to GraphX's limitation.
sed -i 's/player1/11/' basketballplayer-2.X.ngql
sed -i 's/team2/22/' basketballplayer-2.X.ngql
~/.nebula-kind/bin/console -u root -p password --address=127.0.0.1 --port=30000 -f basketballplayer-2.X.ngql
Access nebula graph console with this command:
~/.nebula-kind/bin/console -u root -p password --address=127.0.0.1 --port=30000
You could learn more about Nebula-Operator:
Items | URL |
---|---|
Repo | https://github.com/vesoft-inc/nebula-operator |
Install Guide | https://github.com/vesoft-inc/nebula-operator/blob/master/doc/user/install_guide.md |
Sample Nebula Cluster CRD | https://github.com/vesoft-inc/nebula-operator/blob/master/config/samples/apps_v1alpha1_nebulacluster.yaml |
Access Nebula Cluster created by Nebula Operator | https://github.com/vesoft-inc/nebula-operator/blob/master/doc/user/client_service.md |
Docs of Nebula Graph | English: https://docs.nebula-graph.io Chinese: https://docs.nebula-graph.com.cn |
To quickly try Nebula Algorithm, we could create a spark pod in same namespace of the nebulaCluster CRD:
kubectl create -f http://nebula-kind.siwei.io/deployment/spark.yaml
kubectl wait pod --timeout=-1s --for=condition=Ready -l '!job-name'
After the spark pod is ready, we could access to the spark container:
kubectl exec -it deploy/spark-deployment -- bash
Then, we could download the nebula-algorithm, i.e. in version 3.0.0
, please refer to https://github.com/vesoft-inc/nebula-algorithm/ for more.
Note:
- Official released packages can be found: https://repo1.maven.org/maven2/com/vesoft/nebula-algorithm/
wget https://repo1.maven.org/maven2/com/vesoft/nebula-algorithm/3.0.0/nebula-algorithm-3.0.0.jar
wget https://github.com/vesoft-inc/nebula-algorithm/raw/master/nebula-algorithm/src/main/resources/application.conf
Then we could change the config file of nebula-algorithm on meta and graph addresses:
sed -i '/^ metaAddress/c\ metaAddress: \"nebula-metad-0.nebula-metad-headless.default.svc.cluster.local:9559\"' application.conf
sed -i '/^ graphAddress/c\ graphAddress: \"nebula-graphd-0.nebula-graphd-svc.default.svc.cluster.local:9669\"' application.conf
##### change space
sed -i '/^ space/c\ space: basketballplayer' application.conf
##### read data from nebula graph
sed -i '/^ source/c\ source: nebula' application.conf
##### execute algorithm: labelpropagation
sed -i '/^ executeAlgo/c\ executeAlgo: labelpropagation' application.conf
Run LPA Algorithm on the basketballplayer graph space:
/spark/bin/spark-submit --master "local" --conf spark.rpc.askTimeout=6000s \
--class com.vesoft.nebula.algorithm.Main \
nebula-algorithm-3.0.0.jar \
-p application.conf
The result was configured in csv file:
bash-5.0# ls /tmp/count/
_SUCCESS part-00000-5475f9f4-66b9-426b-b0c2-704f946e54d3-c000.csv
bash-5.0# head /tmp/count/part-00000-5475f9f4-66b9-426b-b0c2-704f946e54d3-c000.csv
_id,lpa
1100,1104
2200,2200
2201,2201
1101,1104
2202,2202
You may encounter this error in case docker was not installed before our installation, you could just follow instructions below to run newgrp docker
and then rerun the installation, it will pass in next go.
ℹ️ Ensuring Linux Docker Permission
❌ Ensuring docker Permission Failed, please try:
option 0: execute this command and retry:
$ newgrp docker
option 1: relogin current shell session and retry install.sh
ℹ️ Waiting for <foo bar> pods to be ready...
There could be different causes:
You could check the reason from another terminal:
kubectl get pods --all-namespaces
kubectl describe pods <pod_name> -n <namespace_name>
- Docker hub pull limit hit In this case, you can refer to https://medium.com/rossum/how-to-overcome-docker-hub-pull-limits-in-a-kubernetes-cluster-382f317accc1.
If you are in China, this may help.
- CPU resource is not enough Please assign more CPU/RAM to your docker or host machine.
Also, there are some extended script for different purposes:
Bootstrap a nebula clsuter on existing KubeSphere All-in-one K8s env:
curl -sL nebula-kind.siwei.io/install-ks-1.sh | bash
Bootstrap a nebula clsuter on existing K8s single node env:
curl -sL nebula-kind.siwei.io/install-on-k8s.sh | bash