Comments (14)
janhaa
commented on July 21, 2024
2
After running sudo restorecon -v /usr/local/bin/k3s on all machines deployment works!
from terraform-hcloud-kube-hetzner.
mysticaltech
commented on July 21, 2024
2
@janhaa @CroutonDigital This is fixed in v2.11.4, please upgrade to it with terraform init -upgrade.
from terraform-hcloud-kube-hetzner.
janhaa
commented on July 21, 2024
1
Some digging with the help of almighty ChatGPT revealed an issue related to SELinux.
k3s-control-plane-1-myr:~ # sudo ausearch -m AVC -ts recent | grep k3s
type=AVC msg=audit(1704401173.178:542): avc: denied { execute } for pid=2234 comm="(k3s)" name="k3s" dev="sda3" ino=279 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
type=AVC msg=audit(1704401178.471:544): avc: denied { execute } for pid=2251 comm="(k3s)" name="k3s" dev="sda3" ino=279 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
type=AVC msg=audit(1704401183.721:546): avc: denied { execute } for pid=2264 comm="(k3s)" name="k3s" dev="sda3" ino=279 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=file permissive=0
...
Running sudo restorecon -v /usr/local/bin/k3s allowed me to get past the issue on this control plane...
from terraform-hcloud-kube-hetzner.
Silvest89
commented on July 21, 2024
1
See also for a possible workaround: #1145 (comment)
@mysticaltech
What do you think of this issue and the work around?
from terraform-hcloud-kube-hetzner.
CroutonDigital
commented on July 21, 2024
1
Thank you! All worked fine
from terraform-hcloud-kube-hetzner.
Taronyuu
commented on July 21, 2024
1
@mysticaltech I just ran into this issue while updating my cluster, remembered this issue and upgraded right away. All solved now. Just wanted to thank you for your effort 🙏🏻
from terraform-hcloud-kube-hetzner.
Wayneoween
commented on July 21, 2024
I'm observing the same issue. Fixing this once might be fine but I presume the issue will come up if there is an automated upgrade of a node?
from terraform-hcloud-kube-hetzner.
CroutonDigital
commented on July 21, 2024
Today 2 k3s nodes got status not Ready, reboot not helped.
I made rollback system snaphot to 1 day ago use snapper rollback. After start k3s node comeback to status Ready.
rebuild Suse MicroOs and try add new k3s node, but not success with same errors:
module.kube-hetzner.null_resource.agents["2-2-bots-large"]: Still creating... [2m10s elapsed]
module.kube-hetzner.null_resource.agents["2-2-bots-large"] (remote-exec): Waiting for the k3s agent to start...
module.kube-hetzner.null_resource.agents["2-2-bots-large"] (remote-exec): Waiting for the k3s agent to start...
module.kube-hetzner.null_resource.agents["2-2-bots-large"]: Still creating... [2m20s elapsed]
╷
│ Error: remote-exec provisioner error
│
│ with module.kube-hetzner.null_resource.agents["2-2-bots-large"],
│ on .terraform/modules/kube-hetzner/agents.tf line 107, in resource "null_resource" "agents":
│ 107: provisioner "remote-exec" {
│
│ error executing "/tmp/terraform_1588448047.sh": Process exited with status 124
How add new additional node to k3s?
from terraform-hcloud-kube-hetzner.
CroutonDigital
commented on July 21, 2024
When I connect to VM:
h-k3s-test-bots-large-wto:~ # journalctl -xeu k3s-agent
░░ The error number returned by this process is ERRNO.
Jan 05 07:49:17 h-k3s-test-bots-large-wto (k3s)[3475]: k3s-agent.service: Failed at step EXEC spawning /usr/local/bin/k3s: Permission denied
░░ Subject: Process /usr/local/bin/k3s could not be executed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The process /usr/local/bin/k3s could not be executed and failed.
░░
░░ The error number returned by this process is ERRNO.
Jan 05 07:49:17 h-k3s-test-bots-large-wto systemd[1]: k3s-agent.service: Main process exited, code=exited, status=203/EXEC
PS: Autoscaller create new 6 VMs and I don't see on k3s )))))
from terraform-hcloud-kube-hetzner.
CroutonDigital
commented on July 21, 2024
restorecon -v /usr/local/bin/k3s helped, too
from terraform-hcloud-kube-hetzner.
janhaa
commented on July 21, 2024
See also for a possible workaround: #1145 (comment)
from terraform-hcloud-kube-hetzner.
mysticaltech
commented on July 21, 2024
@Silvest89 I think the work around is safe to do just after setup. I will introduce it right away. And will also update the k3s selinux package.
from terraform-hcloud-kube-hetzner.
jimping
commented on July 21, 2024
I am getting the same error.
Newest Version, Mac, Fresh Install unchanged config (except hcloud token)
from terraform-hcloud-kube-hetzner.
mysticaltech
commented on July 21, 2024
@jimping Please open a new issue with all the details to reproduce.
from terraform-hcloud-kube-hetzner.
Related Issues (20)
- [Bug]: ImagePullBackoff of system-upgrade controller HOT 1
- Not able to upgrade Traefik HOT 1
- [Bug]: Sudden drop of public internet connectivity for some nodes of arm64 cluster HOT 10
- [Bug]: zram_size not passed on HOT 4
- [Bug]: Terraform Validate fails agent_nodepools HOT 1
- [Bug]: Waiting for load-balancer to get an IP... Hangs HOT 2
- Disable the default load balancer HOT 7
- [Bug]: nginx stuck deploying when not scheduling on control-plane
- Upgrading a clean cluster 1.27 to 1.28 - one of the nodes stuck in emergency mode HOT 1
- Update `cluster-autoscaler` version HOT 4
- Restore hangs waiting for load balancer ip HOT 2
- Allow specifying an existing Floating IP HOT 3
- [Bug]: Disabling SELINUX option is not working HOT 3
- Solution for multiple networks for nodes
- [Bug]: image pull backoff error with latest: hetznercloud/hcloud-csi-driver:v2.7.0 HOT 2
- On GitLab, waiting for MicroOS to become available HOT 5
- [Bug]: /etc/cloud/rename_interface.sh: No such file or directory HOT 7
- [Bug]: helm releases keep installing after disabling them in kube.tf HOT 3
- [Bug]: Terraform does not deploy well HOT 2
- Allow patching default Helm values HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terraform-hcloud-kube-hetzner.