This project aims to provide you single dependency, single node Kubernetes clusters for local testing, experimenting and development purposes.
Do you ever heard from Nix, the functional package manager? Don’t worry if not, all you need to know is that it provides all the third party dependencies for this project, pinned to a dedicated and reproducible version.
KuberNix itself is the Rusty helper program, which takes care of bootstrapping the Kubernetes cluster, passing the right configuration parameters around and keeping track of the running processes.
The following technology stack is currently being used:
Application | Purpose | Version |
---|---|---|
Kubernetes | Cluster Orchestration | v1.15.4 |
CRI-O | Container Runtime | v1.15.2 |
runc | Container Runtime | v1.0.0-rc8 |
cri-tools | CRI Manipulation Tool | v1.15.0 |
CNI Plugins | Container Networking | v0.8.2 |
etcd | Database Backend | v3.3.13 |
CoreDNS | Kubernetes DNS Support | v1.6.4 |
Some other tools are not explicitly mentioned here, like CFSSL for the certificate generation.
As already mentioned, there is only one single dependency needed to run this project: Nix. To setup Nix, simply run:
$ curl https://nixos.org/nix/install | sh
Please make sure to follow the instructions output by the script.
To bootstrap your first cluster, download one of the latest release binaries or build the application via:
$ make build-release
The binary should now be available in the target/release/kubernix
directory of
the project. Alternatively, install the application via cargo install kubernix
.
After the successful binary retrieval, start KuberNix by running it as root
:
$ sudo kubernix
KuberNix will now take care that the Nix environment gets correctly setup,
downloads the needed binaries and starts the cluster. Per default it will create
a directory called kubernix
in the current path which contains all necessary
data for the cluster.
If everything went fine, you should be dropped into a new bash-shell session, like this:
[INFO kubernix] Everything is up and running
[INFO kubernix] Spawning interactive shell
[INFO kubernix] Please be aware that the cluster gets destroyed if you exit the shell
>
Now you can access your cluster via tools like kubectl
:
> kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-85d84dd694-xz997 1/1 Running 0 102s
All configuration files have been written to the target directory, which is now the current one:
> ls -1
apiserver/
controllermanager/
coredns/
crio/
encryptionconfig/
etcd/
kubeconfig/
kubelet/
kubernix.env
kubernix.toml
log/
nix/
pki/
proxy/
scheduler/
For example, the log files for the different running components are now
available within the log
directory:
> ls -1 log
crio.log
etcd.log
kube-apiserver.log
kube-controller-manager.log
kubelet.log
kube-proxy.log
kube-scheduler.log
If you want to spawn an additional shell session, simply run kubernix shell
in
the same directory as the initial bootstrap.
$ sudo kubernix shell
[INFO kubernix] Spawning new kubernix shell in 'kubernix-run'
> kubectl run --generator=run-pod/v1 --image=alpine -it alpine sh
If you don't see a command prompt, try pressing enter.
/ #
This means that you can spawn as many shells as you want to.
The whole cluster gets automatically destroyed if you exit the bash session from the initial process:
> exit
[INFO kubernix] Cleaning up
Please note that the directory where all the data is stored is not being removed after the exit of KuberNix. This means that you’re still able to access the log and configuration files for further processing.
KuberNix has some configuration possibilities, which are currently:
CLI argument | Description | Default | Environment Variable |
---|---|---|---|
-r, --root |
Path where all the runtime data is stored | kubernix-run |
KUBERNIX_ROOT |
-l, --log-level |
Logging verbosity | info |
KUBERNIX_LOG_LEVEL |
-c, --cidr |
CIDR used for the cluster network | 10.10.0.0/16 |
KUBERNIX_CIDR |
-o, --overlay |
Nix package overlay to be used | KUBERNIX_OVERLAY |
|
-p, --packages |
Additional Nix dependencies to be added to the environment | KUBERNIX_PACKAGES |
|
-i, --impure |
Do not clear the current env during bootstrap | false |
Please ensure that the CIDR is not overlapping with existing local networks and that your setup has access to the internet. The CIDR will be automatically split up over the necessary cluster components.
Overlays provide a method to extend and change Nix derivations. This means, that
we're able to change dependencies during the cluster bootstrapping process. For
example, we can exchange the used CRI-O version to use a local checkout by
writing this simple overlay.nix
:
self: super: {
cri-o = super.cri-o.overrideAttrs(old: {
src = ../path/to/go/src/github.com/cri-o/cri-o;
});
}
Now we can run KuberNix with the --overlay, -o
command line argument:
$ sudo kubernix --overlay overlay.nix
[INFO kubernix] Nix environment not found, bootstrapping one
[INFO kubernix] Using custom overlay 'overlay.nix'
these derivations will be built:
/nix/store/9jb43i2mqjc94mbx30d9nrx529w6lngw-cri-o-1.15.2.drv
building '/nix/store/9jb43i2mqjc94mbx30d9nrx529w6lngw-cri-o-1.15.2.drv'...
Using this technique makes it easy for daily development of Kubernetes components, by simply changing it to local paths or trying out new versions.
It is also possible to add additional packages to the KuberNix environment by
specifying them via the --packages, -p
command line parameter. This way you
can easily utilize additional tools in a reproducible way. For example, when to
comes to using always the same Helm version, you could simply run:
$ sudo kubernix -p kubernetes-helm
[INFO kubernix] Nix environment not found, bootstrapping one
[INFO kubernix] Bootstrapping cluster inside nix environment
...
> helm init
> helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
All available packages are listed in the official Nix website.
If you still want to access some system packages inside the interactive shell,
then it is possible to run KuberNix in non-pure mode via the command line flag
--impure, -i
. This is not recommended and can have negative impact on the
overall cluster bootstrapping process.
You want to contribute to this project? Wow, thanks! So please just fork it and send me a pull request.