automaticserver / lxe Goto Github PK

If we run the following command to create the deployment, we can view that the pods are up with lxc containers.
kubetcl apply -f Deployment.yaml
If we run the following command to remove the deployment, the deployment is cleaned up.
kubectl delete deployment
But if we run the following command to view the existing containers inside the node, we can see that the container(which was created at the time of deployment) is in STOPPED state. So it is not cleaned up completely during kubectl delete deployment step.
lxc list
Another issue that I have faced is that though I am using remote lxc image server to create the container still it is downloading the image inside the node and storing it.

I have successfully deployed lxc container in k3s cluster. Now I want to configure lxc container to act as nfs client with rw permission in nfs share. The nfs share mount is configured with rw permission but actually it gets read permission. Using docker the requirement is tested successfully. Is there any way that I can access rw permission in a common location from all lxc container?

I have to run two pods with lxc container and needs to communicate between those how to do that?

I have lxe, lxc and lxd installed on my system. I have followed the instructions here. But when i ran the command it gave me an error
that failed to pull image like kube api server and all.

ERROR  [10-17|14:51:41.838] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-apiserver%v1.28.2"
ERROR  [10-17|14:51:41.854] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-apiserver%v1.28.2"
ERROR  [10-17|14:51:41.872] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-apiserver%v1.28.2"
ERROR  [10-17|14:51:41.888] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-apiserver%v1.28.2"
ERROR  [10-17|14:51:41.903] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-apiserver%v1.28.2"
ERROR  [10-17|14:51:41.936] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-controller-manager%v1.28.2"
ERROR  [10-17|14:51:41.952] PullImage: failed to pull image               error="The remote \"registry.k8s.io\" doesn't exist" image="registry.k8s.io:kube-controller-manager%v1.28.2"

Am i missing something?

Hello Team ,

Greetings.
Please find below our stack set-up / our problem statement and provide a suitable guidance for the same.

Stack Overview
Kubernetes master with Docker Runtime in one node and LXC/LXD worker in a separate node and it is attached to the master node with the help of LXE.

Problem Statement :
we could not able to successfully establish Flannel based pod network across this Master and Worker node stack. We presume the issue is related to differences in yaml template which varies between docker and LXC runtimes.

What we are trying to do ?
Our ultimate intention is to have LXC container's run on top of CRIO runtime. And for that we are trying to come past small milestones as like above.

Kindly guide us for proceeding further in our path.

Kindly guide us through this issue. Thanks.

Hello,
is it possible to run this in coexistence with docker as CRI (probably on the same nodes)?

https://stgraber.org/2023/12/12/lxd-now-re-licensed-and-under-a-cla/

Impact to downstreams (consumer of LXD Go packages)
Up until now, all the Go packages of LXD were under the Apache 2.0 license, that was fitting quite well in the Go ecosystem where the Apache 2.0, BSDs and MIT licenses are very popular.

Now with this change, you need to realize that you may start to include/bundle AGPLv3 code within your own project. This a copyleft license and so may require re-licensing of your own project to comply with it.

Again, this is quite the can of worms, with my usual recommendation being “stay away”, but if you must use any of LXD’s Go packages, I’d strongly recommend talking to a lawyer to fully understand your exposure to that new license.

LXE is using the go-client package to interact with lxd and thus is impacted by this change.

I followed https://github.com/automaticserver/lxe/blob/update_deps/doc/examples/multi-node-cluster-with-k3s.md to setup multinode kubernetes cluster. I created my own public lxc image repository to use for deployment in kubernetes cluster. But failed to pull the image at the time of deployment. From images/ubuntu public image repository the deployment is working.

LXD is installed using snap on a ubuntu1804 system. encountered error when building LXE. Is my current software version compatible with LXE? how to fix the build error?

$ lxd --version
3.15
$ go version
go version go1.12.6 linux/amd64

Error when build LXE code:
$ make build
0.1.5
/bin/sh: 1: gofmt: not found
Makefile:79: recipe for target 'version' failed
make: *** [version] Error 127

When building on Centos 7 I get the following error:

0.0.0-f6e0603-dirty
<standard input>:1:12: illegal character U+005C '\'

the fix is to add -e to the version line in make file:

@echo -e "package cri\n// Version of LXE\nconst Version = \"$(VERSION)\"" | gofmt > cri/version.go

Hi,

I was trying to run Cilium for networking and LXE as container images in a k8s cluster, however it seems that the kubelet is only able to run one type of image at the same time, therefore when runing LXE the cilium docker containers are failing to start (cannot pull image)

Are there any workarounds I am not aware of to be able to run/specify the container type? Would it be able to emulate a docker container image? by using lxc with a docker inside for those types of images? Any ideas?

Thank you

Using kubespray I have formed k8s cluster. The cluster works fine with containerd as container runtime endpoint. But if I change the endpoint to use lxe, all the pods of kube-system are going down and claiming that the images are missing. If I try to create a new pod it says the image alias is missing. I want all the containers of kube-system namespace use containerd image and my created pod uses lxe image. Is it possible?

lxd-remote-config is using the following config.yml

default-remote: local
remotes:
images:
ubuntu:
addr: Ubuntu Cloud Images
public: true
protocol: simplestreams
…

My pod creation config content:
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: default
spec:
tolerations:

• key: node-role.kubernetes.io/master
  operator: Exists
  containers:

name: nginx
image: “ubuntu/xenial/amd64”
…

I followed https://github.com/automaticserver/lxe/blob/update_deps/doc/examples/multi-node-cluster-with-k3s.md to setup multi node kubernetes cluster. I created 2 nodes with container runtime as lxe. I created pods in both nodes. But I am not able to ping from one container to another container which is residing in separate node. I am able to ping between containers of same node. Default bridge network is being used.

Hello,
I like the idea to manage stateful lxc-containers with kubernetes. However, after reading the drawbacks and workarounds, I wonder if this is the "best" way to achieve this.

A while ago, I have created a small project that runs LXC within a container (https://github.com/micw/docker-lxc). This is just a PoC with many glitches but it works out of the box with an unmodified kubernetes and it runs stable (I run my a daily used archlinux as remote desktop on it for >1 year). One drawback is that the lxc-container dies if the docker container is restarted.

But I have thought if it could be possible to combine both approaches and I came to the following ideas:

• instead of implementig this as CRI, a service could just listen to k8s events and start/stop lxd containers
• to start an lxc-container, a special k8s docker container could be started. it's just a dummy but the event listening service could be triggered by creation/deletion/changes of such containers and update the lxc containers accordingly
• the dummy container could also be used to proxy logs and console between lxc and kubernetes: it should contain an entrypoint that tailf the lxc container logs and it should contains a modified /bin/sh that connects to the container

What do you think about this?

This approach would also allow to run thet event listening service as well as the lxd as docker containers, removing all special reqirements for the deployment.

Hi there

When we have K8s create the container, we get the following error:

Warning Failed 0s kubelet, ip-172-31-35-162 Error: Failed to run: /snap/lxd/current/bin/lxd forkstart ubuntu /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/ubuntu/lxc.conf:

Here is the yaml file I am using:

apiVersion: v1
kind: Pod
metadata:
name: os
labels:
app: os
spec:
containers:
- name: ubuntu
image: ubuntu/14.04

If I check lxc, I can see it successfully downloaded the image and tried to create the container, but failed on starting it

Only thing I see in the lxe.log file is the following:

t=2019-07-27T19:54:29+0000 lvl=eror msg="StartContainer: ContainerID usl24jpixjqd7rpi trying to start container: Failed to run: /snap/lxd/current/bin/lxd forkstart usl24jpixjqd7rpi /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/usl24jpixjqd7rpi/lxc.conf: "

Any ideas?

Thanks!