kvaps / bridget Goto Github PK

View Code? Open in Web Editor NEW

43.0 3.0 11.0 79 KB

Simple bridge network for kubernetes

License: Apache License 2.0

Shell 98.45% Dockerfile 1.55%

cni kubernetes bridge-network bridget pod-network pipework vlan

bridget's Introduction

bridget

Simple bridge network for kubernetes

How it works

bridget is a short shell script that helps you to organise simple bridged network for Kubernetes. There are no overlays, no policies. Just a flat L2-network across all your hosts and pods.

In addition bridget can automatically configure VLAN and bridge interfaces for that. See the picture:

bridget automatically retrieves node cidr from your pod-network and configures cni to use it.

Parameters

All parameters are passed as environment variables:

BRIDGE (example: cbr0) - Bridge name. Mandatory option.
VLAN (example: 100) - VLAN id. If set, a new vlan-interface under IFACE will be created and added to BRIDGE.
IFACE (example: eth0) - Physical interface to connect bridge to. Mandatory if VLAN is set, but can also be used alone.
MTU (default: 1500) - MTU value for cni config
CHECK_SLAVES (default: 1) - Make bridget configure slave interfaces if the bridge already exists.
POD_NETWORK (default: 10.244.0.0/16) - Your pod network.
DEBUG (default: 0) - Enable verbose output.

Quick start

Instantiate your kubernetes with --pod-network-cidr=10.244.0.0/16 flag.
Download yaml file:

curl -O https://raw.githubusercontent.com/kvaps/bridget/master/bridget.yaml

Edit desired parameters:

vim bridget.yaml

By default bridget uses cbr0 bridge that isn't connected anywhere, so you need to either set IFACE and VLAN parameters or configure your host system to connect the physical interface to this bridge manually.

Please make sure that you have no IP address on the bridge because it will be configured automatically.

Run daemonset:

kubectl create -f bridget.yaml

Update

Check your bridget.yaml for changes.
Run:

kubectl delete -f bridget.yaml
kubectl create -f bridget.yaml

Alternatives

There aren't a lot of alternatives if you want to use flat L2-network with kubernetes.

Even with most of the existing solutions like flannel's or romana's L2 modes it's still quite difficult to use your own rules for NATing and routing. So you gain flexible policies and some other things, but lose simplicity and productivity of a simple L2-network.

Bridget was created under pipework's inspiration. pipework allows you to add single interfaces to your containers, but with additional manual actions, and Kubernetes doesn't know anything about your manual changes.

Unlike pipework, bridget uses CNI to configure pod interfaces. As a result all configuration occurs automatically and kubernetes gets right IP-addresses.

Another alternative is to сreate your own CNI configuration with bridge or macvlan plugin for each of your hosts.

Contact

Author: kvaps
Bugs: issues

Contributing

Use Pull Requests to contribute bugfixes or new features. It is assumed that your code and documentation are contributed under the Apache License 2.0.

Reporting bugs

Please use github issue-tracker to submit bugs

License

bridget is distributed under the Apache 2.0 license. See the LICENSE file for details.

bridget's People

Contributors

Stargazers

Watchers

Forkers

raceli devlikeops vitalif morningsong franklinharry cloudcafetech sfxworks chongdershubhayu xumia johnyin123 mestic-al

bridget's Issues

Bridget doesn't play nicely with CentOS' firewall

Bridget doesn't play nicely with CentOS (and probably RHEL) firewall by default as it sets the policy on the FORWARD chain to DROP. This I believe can be fixed by including the 'firewall' CNI plugin in the CNI script.

I think you need to add something like:

"plugins": [
      {
        "type": "firewall"
      }
    ]

Problem trying to install bridget addon

It looks like something is wrong with bridget.yaml or between bridget.yaml and my kubernetes version or environment.

I initialize the control node in this way:

ubuntu@kube-master:~$ sudo kubeadm init --pod-network-cidr=10.221.0.0/16

This is the kubernetes version:

ububuntu@kube-master:~$ sudo kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T16:51:36Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

I downloaded bridget.yaml and customized it to my needs:

ubuntu@kube-master:~$ curl -O https://raw.githubusercontent.com/kvaps/bridget/master/bridget.yaml
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1723 100 1723 0 0 6153 0 --:--:-- --:--:-- --:--:-- 6153

ubuntu@kube-master:~$ sed -i -e 's/value: "cbr0"/value: "br0"/' -e 's/#- name: IFACE/- name: IFACE/' -e 's/# value: "eth0"/ value: "enp1s0"/' -e 's/#- name: MTU/- name: MTU/' -e 's|# value: "1500"| value: "1400"\n - name: POD_NETWORK\n value: "10.221.0.0/16"|' bridget.yaml

And when I try to installt it:

ubuntu@kube-master:~$ sudo kubectl create -f bridget.yaml
clusterrole.rbac.authorization.k8s.io/bridget created clusterrolebinding.rbac.authorization.k8s.io/bridget created serviceaccount/bridget created error: unable to recognize "bridget.yaml": no matches for kind "DaemonSet" in version "extensions/v1beta1"

Do you have any idea about what might be happening?

Thanks

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

And apply bridget manifest by simply downloading the configuration,

curl -O https://raw.githubusercontent.com/kvaps/bridget/master/bridget.yaml

Bridget pods are running,

 kubectl -n kube-system get pod -l app=bridget -o wide
NAME            READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
bridget-49l64   1/1     Running   0          4m37s   10.20.1.62    ip-10-20-1-62    <none>           <none>
bridget-4prxh   1/1     Running   0          4m37s   10.20.1.172   ip-10-20-1-172   <none>           <none>
bridget-gvz57   1/1     Running   0          4m37s   10.20.1.95    ip-10-20-1-95    <none>           <none>
bridget-qz59n   1/1     Running   0          4m37s   10.20.1.157   ip-10-20-1-157   <none>           <none>
bridget-z5j2d   1/1     Running   0          4m37s   10.20.1.15    ip-10-20-1-15    <none>           <none>

But coredns pods are not passing healthcheck,

coredns-66bff467f8-fmc4f                 0/1     Running   0          5m10s   10.244.1.2    ip-10-20-1-95    <none>           <none>
coredns-66bff467f8-jn4xj                 0/1     Running   0          9m39s   10.244.2.2    ip-10-20-1-172   <none>           <none>

Looks like its having problem while trying to reach k8s endpoint (service object)

kubectl logs coredns-66bff467f8-fmc4f  -n kube-system

Output:

I0625 22:08:45.801746       1 trace.go:116] Trace[60780408]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105 (started: 2020-06-25 22:08:15.801233016 +0000 UTC m=+310.027901299) (total time: 30.000480721s):
Trace[60780408]: [30.000480721s] [30.000480721s] END
E0625 22:08:45.801770       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0625 22:08:45.802757       1 trace.go:116] Trace[340007387]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105 (started: 2020-06-25 22:08:15.802351838 +0000 UTC m=+310.029020177) (total time: 30.000388737s):
Trace[340007387]: [30.000388737s] [30.000388737s] END
E0625 22:08:45.802772       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout
I0625 22:08:45.803735       1 trace.go:116] Trace[1304066831]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105 (started: 2020-06-25 22:08:15.803378498 +0000 UTC m=+310.030046784) (total time: 30.000337496s):
Trace[1304066831]: [30.000337496s] [30.000337496s] END
E0625 22:08:45.803753       1 reflector.go:153] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: i/o timeout

kubectl version

kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:41:22Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:33:59Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

ifconfig on one of the node

ifconfig

       cbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.244.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether d6:60:82:58:3b:0c  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3  bytes 270 (270.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

describe coredns pod

kubectl describe po coredns-66bff467f8-fmc4f  -n kube-system 
Name:                 coredns-66bff467f8-fmc4f
Namespace:            kube-system
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 ip-10-20-1-95/10.20.1.95
Start Time:           Thu, 25 Jun 2020 22:03:02 +0000
Labels:               k8s-app=kube-dns
                      pod-template-hash=66bff467f8
Annotations:          <none>
Status:               Running
IP:                   10.244.1.2
IPs:
  IP:           10.244.1.2
Controlled By:  ReplicaSet/coredns-66bff467f8
Containers:
  coredns:
    Container ID:  docker://9b763857bff24d1820d23d4e31c42944d93ab4059c647ef4ea4d02a767a586f3
    Image:         k8s.gcr.io/coredns:1.6.7
    Image ID:      docker-pullable://k8s.gcr.io/coredns@sha256:2c8d61c46f484d881db43b34d13ca47a269336e576c81cf007ca740fa9ec0800
    Ports:         53/UDP, 53/TCP, 9153/TCP
    Host Ports:    0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    State:          Running
      Started:      Thu, 25 Jun 2020 22:03:05 +0000
    Ready:          False
    Restart Count:  0
    Limits:
      memory:  170Mi
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from coredns-token-88rpd (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
  coredns-token-88rpd:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  coredns-token-88rpd
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  kubernetes.io/os=linux
Tolerations:     CriticalAddonsOnly
                 node-role.kubernetes.io/master:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                    Message
  ----     ------     ----                    ----                    -------
  Normal   Scheduled  8m35s                   default-scheduler       Successfully assigned kube-system/coredns-66bff467f8-fmc4f to ip-10-20-1-95
  Normal   Pulling    8m34s                   kubelet, ip-10-20-1-95  Pulling image "k8s.gcr.io/coredns:1.6.7"
  Normal   Pulled     8m32s                   kubelet, ip-10-20-1-95  Successfully pulled image "k8s.gcr.io/coredns:1.6.7"
  Normal   Created    8m32s                   kubelet, ip-10-20-1-95  Created container coredns
  Normal   Started    8m32s                   kubelet, ip-10-20-1-95  Started container coredns
  Warning  Unhealthy  3m28s (x31 over 8m28s)  kubelet, ip-10-20-1-95  Readiness probe failed: HTTP probe failed with statuscode: 503

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.