Comments (7)
Thanks for the explanation, I was able to run a DaemonSet using the logic mentioned however with some changes. As you said we would have to keep a check on the distro and its version of the worker nodes. Please feel free to close the issue if you believe that the package cannot be/will never be made part of the worker nodes. Thanks.
I am posting the yaml definition for anyone who would like to use.
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: gfs-client
namespace: kube-system
labels:
app: gfs-client
spec:
selector:
matchLabels:
name: gfs-client
template:
metadata:
labels:
name: gfs-client
spec:
hostNetwork: true
containers:
- name: ubuntu
securityContext:
privileged: true
image: ubuntu:bionic
args:
- /bin/bash
- "-c"
- |
chroot /host /bin/bash -u <<'EOF'
export GLUSTERFS_PACKAGE=glusterfs-client
apt list --installed | grep ${GLUSTERFS_PACKAGE} > /dev/null
case $? in
0)
echo "nothing to do: glusterfs package is already installed"
;;
1)
echo "installing glusterfs package"
apt update && apt install -y ${GLUSTERFS_PACKAGE}
;;
*)
echo "failed to check if glusterfs is installed" >&2
exit 1
;;
esac
sleep infinity
EOF
volumeMounts:
- name: host
mountPath: /host
- name: docker
mountPath: /var/run/docker.sock
priorityClassName: system-cluster-critical
volumes:
- name: host
hostPath:
path: /
- name: docker
hostPath:
path: /var/run/docker.sock
type: Socket
from doks.
Hey @m-usmanayub 👋
for cases like this, we usually recommend running a Daemonset that installs the binary onto the host system. One key advantage of this approach is that you do not depend on the cloud provider and can update the binary per your own preferences.
How do you feel about this approach?
from doks.
(Sorry for the accidental close and half-finished comment, I fat-fingered that one.)
from doks.
Hey @m-usmanayub 👋
for cases like this, we usually recommend running a Daemonset that installs the binary onto the host system. One key advantage of this approach is that you do not depend on the cloud provider and can update the binary per your own preferences.
How do you feel about this approach?
Hi @timoreimann
The approach seems realistic. Do you have any example of this?
Also in such a case we need to know the OS flavor (Ubuntu, CentOS etc) running on the worker nodes and install the relevant OS Package (deb/rpm) by creating a customized docker image. am I right?
from doks.
@m-usmanayub I'd try to create a Daemonset that mounts the root host-filesystem, chroot
s into it, checks if the package is already installed, and if not updates the repository and installs the package. We currently run Debian, so the logic could look something like this (untested):
#!/bin/bash
chroot /host /bin/bash -u <<'EOF'
export GLUSTERFS_PACKAGE=glusterfs-client
dpkg -l ${GLUSTERFS_PACKAGE} > /dev/null
case $? in
0)
echo "nothing to do: glusterfs package is already installed"
;;
1)
echo "installing glusterfs package"
apt-get update && apt-get install -y ${GLUSTERFS_PACKAGE}
;;
*)
echo "failed to check if glusterfs is installed" >&2
exit 1
;;
esac
sleep infinity
EOF
You might not even need a custom container image if you choose to inline the script as arguments to a container that comes with bash (e.g., a Debian-based image).
For mounting the host file-system, you'll need your Daemonset to specify volume and volume mount fields accordingly. A small blog post on the topic is available here, and our own doks-debug Daemonset does it as well. (Adjust the manifest to your needs, of course.)
Hope this gets you started. Note that while we do not have plans to do so, we could change the distro and/or distro version in the future. Such a change would likely be bundled with a Kubernetes minor release, however, so if you keep an eye on the release notes you should be able to respond to any updates we might do that affect how your Daemonset works.
from doks.
A very nice solution! In the spirit of sharing, I've updated the installation script above ☝️ to install a more recent version of glusterfs-client
. At the time of writing, the master nodes are based on Debian Buster which are shipped with version 5.5-3. Due to some strange and persistent issues with missing files / non-empty directories showing as empty on the gluster volumes, I had to update both the server and the client to 8.3-1 from upstream. Here's my code:
image: ubuntu:groovy
args:
- /bin/bash
- "-c"
- |
chroot /host /bin/bash -u <<'EOF'
export GLUSTERFS_PACKAGE=glusterfs-client
apt show ${GLUSTERFS_PACKAGE}
apt list --installed | grep ${GLUSTERFS_PACKAGE} > /dev/null
case $? in
0)
echo "nothing to do: glusterfs package is already installed"
;;
1)
echo "installing glusterfs package"
wget -O - https://download.gluster.org/pub/gluster/glusterfs/7/rsa.pub | apt-key add -
DEBID=$(grep 'VERSION_ID=' /etc/os-release | cut -d '=' -f 2 | tr -d '"')
DEBVER=$(grep 'VERSION=' /etc/os-release | grep -Eo '[a-z]+')
DEBARCH=$(dpkg --print-architecture)
echo deb https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/${DEBID}/${DEBARCH}/apt ${DEBVER} main > /etc/apt/sources.list.d/gluster.list
apt update && apt install -y ${GLUSTERFS_PACKAGE}
;;
*)
echo "failed to check if glusterfs is installed" >&2
exit 1
;;
esac
sleep infinity
EOF
from doks.
I'll be closing out this ticket given a workaround is possible. I could imagine a future feature where customers can choose from a limited list of packages / node properties to enable on bootstrap. This needs broader discussion, however, and consideration for how DOKS worker node images may possibly evolve. We'll continue to track this possibility internally.
from doks.
Related Issues (20)
- Wrong timezone? HOT 5
- Maintain HA on single-node cluster during updates HOT 6
- Automatic minor version k8s upgrades HOT 4
- Scale node pool to zero throws HTTP 500 HOT 2
- Support pod security policies HOT 2
- cert-manager and metrics-server broken in 1.16 upgrade? HOT 8
- dont cap grace period at 0 for soft evictions HOT 2
- support dynamic kubelet config HOT 1
- Support metrics in the integrated dashboard HOT 15
- Integrate DOKS plus managed resources into projects HOT 1
- No metrics for cluster in version 1.20.2-do.0 HOT 6
- kube-state-metrics serviceaccount installed to kube-system namespace by default HOT 3
- Create load balancer for service in same project as cluster HOT 2
- Resizing root partition of a DOKS Worker Node Droplet
- Feature Request: UI Integration to Apply Taints automatically to Node Pools HOT 5
- Linux Kernel 5.x on DOKS Nodes HOT 4
- Feature request: UDP support in Load Balancers HOT 1
- Question about node taints with regard to doks-managed 'coredns' deployment HOT 1
- Cronjob TimeZone in >=1.27, TZ not found HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from doks.