Comments (5)
That annotation is/was used as there was a 🐔 and 🥚 problem with CAPA provider. When we let it to create all the resources, which includes ELB for the CP, it also populates the controlPlaneEndpoint
field. And as that is marked as immutable, k0smotron was/is not able to override it with the hosted controlplane address (LB service).
More so with the annotation removed all works just fine.
This is surprising, to me at least. We did quite extensive testing with AWS in the early days and always hit the above mentioned problem. Maybe they've changed somthing in CAPA provider side. Did you test with both hosted controlplanes and CPs using Machine
s?
from k0smotron.
Thanks for the answer!
Yes, now I see what you're talking about.
CAPA controller tries to create LB and fails to do so, because it wants to update LB field which was already filled by k0smotron
E0809 21:30:35.544488 1 controller.go:329] "Reconciler error" err="failed to patch AWSCluster default/kk-1: admission webhook \"validation.awscluster.infrastructure.cluster.x-k8s.io\" denied the request: A
WSCluster.infrastructure.cluster.x-k8s.io \"kk-1\" is invalid: spec.controlPlaneEndpoint: Invalid value: kk-1-apiserver-92222222.us-west-1.elb.amazonaws.com:6443: field is immutable" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="default/kk-1" namespace="default" name="kk-1" reconcileID="19e1f45c-a166-45b4-a558-dabd58a9f81e"
And then just marks cluster as defunct somehow
E0809 21:59:12.328636 1 controller.go:329] "Reconciler error" err="no loadbalancer exists for the AWSCluster kk-1, the cluster has become unrecoverable and should be deleted manually: no classic load balancer found with name: \"kk-1-apiserver\"" controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="default/kk-1" namespace="default" name="kk-1" reconcileID="c8d8257c-4a1b-404c-85b6-faaca324cad3"
But machines are actually created in my case and I don't see any extra LBs, so it looks like it all works from the first glance.
I overlooked the fact that AWS provider don't like what's going on there. And it looks more like CAPA related problem here.
Did you research a possibility for proper fix? In CAPA perhaps?
Because right now status patch workaround breaks declarative deployment flow.
from k0smotron.
Also I forgot to add:
Since we don't have any finalizers on AWSCluster
resource when it is deployed with the annotation it get deleted instantly.
This is a big problem, since AWSMachines
get stuck in this case, because awsmachine_controller
expects that AWSCluster
is present.
Thus to create cluster we need manually patch status and to delete it we must manually handle all AWSMachine
finalizers.
from k0smotron.
Did you research a possibility for proper fix? In CAPA perhaps?
I've had a quick look but unfortunately have not had time to go deep enough to provide an actual fix there. 😢
from k0smotron.
@jnummelin I recently tested Azure provider (CAPZ) and we have similar situation with it as well.
The main difference between the two is that CAPZ is not misbehaving trying to update AzureCluster
object. It just creates extra LB and all connected resources and then continuing work as usual.
Given the fact that CAPZ don't have any option to disable LB creation (as well as CAPA) will you consider handling it on k0smotron side?
from k0smotron.
Related Issues (20)
- Add service IP and FQDNs to the k0s config sans section for HCP
- Persistent volume resize
- RemoteMachine Setup: K0smotronControlPlane Etcd stuck restarting due to Rampant Reconciles HOT 4
- how to use clusterapi provider to deploy multiple clusters HOT 5
- kosmotron + capd doesn't work at all HOT 1
- Config files for apiserver
- K0smotron CAPI BYOCNI HOT 1
- ControlPlane manager throwing segfaults (control plane does come up) HOT 1
- Integration with Kapitan and Commodore HOT 1
- Document k0s persistence
- k0smotron operator falls into CrashLoopBackOff HOT 1
- Seeking Advice on Setting Up a High Availability K0s and K0smotron Lab HOT 2
- Worker node is not registered in kubernetes HOT 1
- Unable to upgrade standalone k0s cluster on AWS HOT 4
- Not Able To Create More Than 2 Clusters HOT 4
- Add an ability to delete hosted cluster state HOT 1
- K0smotron controller freezes for some time before continue reconciling
- Expose k0s cluster ID in kosmotron API
- Add ContentFrom option for Files configuration in K0sConfigSpec and K0sWorkerConfigSpec
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k0smotron.