Comments (8)
Hi @raguilera82 ,
After some experimentation with LXC and zfs I got microk8s working. Here is what I had to do:
- Create the following profile:
$ cat microk8s.profile
name: microk8s
config:
boot.autostart: "true"
linux.kernel_modules: ip_vs,ip_vs_rr,ip_vs_wrr,ip_vs_sh,nf_conntrack_ipv4,ip_tables,ip6_tables,netlink_diag,nf_nat,overlay
raw.lxc: |
lxc.apparmor.profile=unconfined
lxc.mount.auto=proc:rw sys:rw
lxc.cgroup.devices.allow=a
lxc.cap.drop=
security.nesting: "true"
security.privileged: "true"
description: ""
devices:
aadisable:
path: /sys/module/nf_conntrack/parameters/hashsize
source: /sys/module/nf_conntrack/parameters/hashsize
type: disk
aadisable1:
path: /sys/module/apparmor/parameters/enabled
source: /dev/null
type: disk
aadisable2:
path: /dev/zfs
source: /dev/zfs
type: disk
$ lxc profile copy default microk8s
$ cat microk8s.profile | lxc profile edit microk8s
- Launch a container:
$ lxc launch -p default -p microk8s ubuntu:18.04
- Install
zfsutils-linux
in the container. This package should be part of microk8s so I also opened a new issue.
# apt install zfsutils-linux
- And finally install microk8s
# snap install microk8s --edge --classic
Most of the work is on the lxc profile so there is not really much to do for microk8s. Here is an ascii cast of the above steps: https://asciinema.org/a/198964
Thanks
from microk8s.
Hi @raguilera82 ,
What is your lxc profile look like?
I had partial success in running microk8s with this one:
name: microk8s-profile
config:
boot.autostart: "true"
linux.kernel_modules: ip_tables,ip6_tables,netlink_diag,nf_nat,overlay
raw.lxc: |
lxc.apparmor.profile=unconfined
lxc.mount.auto=proc:rw sys:rw
lxc.cap.drop=
security.nesting: "true"
security.privileged: "true"
description: ""
devices:
aadisable:
path: /sys/module/nf_conntrack/parameters/hashsize
source: /dev/null
type: disk
aadisable1:
path: /sys/module/apparmor/parameters/enabled
source: /dev/null
type: disk
from microk8s.
@raguilera82 , seems docker inside lxc containers does not play well with ZFS https://github.com/lxc/lxd/issues/2305 could you use dir instead?
from microk8s.
Hi @ktsakalozos
Thanks for your answers. My experience with LXC containers (version 3.2) is that they work properly with Docker and ZFS (this is the recommend storage) if you enabled nesting and privileged properties of security. For example, I have a LXC container with an instance of GitLab and I can work with Docker and docker-compose without problems into the container. The issue that you have mentioned is quite old.
Now, I can not change the system storage, maybe for that reason I will get back to Minikube until this issue will be fixed.
Thank you very much for your time.
from microk8s.
Thank you for giving microk8s a try
from microk8s.
Thank you @ktsakalozos
Your steps work perfectly! Also, I have tried with ubuntu:16.04 with your microk8s profile and all works like a charm.
Thank you very much for your time!
from microk8s.
@ktsakalozos Thanks for the info above! I have a question, I see we're removing a lot of the security constraints from lxc .. Can you comment on how contained is that lxc container now? Is it still a little hard to break out of a lxc container with that configuration, or does it basically have full host control without much effort? Basically can I let fairly hostile guests in that lxc with some level of guarantee it's not too easy to break out
Thanks!
from microk8s.
Hi @kim0 ,
I am not entirely sure what you are asking me. I would say it is a little hard to reach the host but not hard enough.
MicroK8s, can get out of the lxc confinement and can access the host machine and this is the purpose of the profile we apply. MicroK8s is expected to run on the host and you are expected to trust it. What you might not trust is the user workload that runs inside Kubernetes and that is constrained by docker containers. You should be aware that MicroK8s is not supposed to be deployed in production or in untrusted environments. Its purpose is mainly to assist in application development locally. If security is of concern you should look into production grade Kubernetes solutions.
Having said that here are a few ways to harden MicroK8s.
- Refine the lxc profile
- Use MicroK8s inside a VM
- Set AppArmor and firewall rules that apply to your usecase
- Test and contribute to this PR #88
Thanks
from microk8s.
Related Issues (20)
- `microk8s join ... --worker` works, `microk8s join ... ` does not HOT 2
- microk8s wont install with MacOs M1 macmini, Homebrew HOT 6
- microk8s add-node --format=short outputs the join command twice HOT 1
- Microk8s snap breaks on custom Ubuntu Core image on AirGapped environments HOT 6
- snap.microk8s.daemon-kubelite.service: Main process exited, code=exited, status=1/FAILURE HOT 1
- Calico-node pods being recreated every few seconds HOT 2
- please add *.svc as no_proxy in install guide behind a proxy HOT 1
- docs : please add *.svc as no_proxy in install guide behind a proxy HOT 2
- docs: follow up issue for #4540 HOT 2
- POD resource limits not respected
- no kind "Deployment" is registered for version "apps\x10v1" in scheme HOT 4
- containerd is not able to authenticate and pull image from GCP Artifact Registry HOT 4
- Update Helm to the latest version HOT 1
- Microk8s kubelite restarting infinitely after power failure
- AMD GPU with ROCm
- Fatal Error When Using Remote Directories in kustomization.yaml with microk8s on Ubuntu 24.04 LTS HOT 3
- Odd dmesg content in `sudo microk8s.inspect` tarball
- Configure containerd configuration template via launch configuration
- Configure nodes during join
- RBAC rolebindings not working
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from microk8s.