Use container IP address in server field in kubeconfig for kind cluster about kind HOT 12 CLOSED

Each cluster has a different port for the API server so it should still be possible for them to talk to each other (?), the reason for localhost is that windows / mac do not support anything else as far as I know, because the container network is actually in a light VM.

On Linux, I don't think it's possible while using localhost because 127.0.0.1 is not routable from one container to another i.e. across the network bridge.

Can federation support keying off of the address + port instead of just the address instead?

We key off of the entire server field so we include the port. It's just that using localhost:<port_for_other_kind_cluster_api_server> returns a connection refused error because the local container (referred to by using localhost) is not listening on that port since it's part of the other cluster's container network. So the pod running inside the kind-1 cluster needs to have a routable IP address to the API server running in the kind-2 cluster.

from kind.

--raw makes sense, BUT, we need the random port outside the container to handle docker for mac / windows and I'd rather minimize code paths because CI should match local usage for reproducibility as much as possible..

from kind.

kind get kubeconfig --internal > internal-kubeconfig after #478

from kind.

I think that as a workaround this will work
docker exec -it $CLUSTER_NAME-control-plane cat /etc/kubernetes/admin.conf

from kind.

On OSX at least, it's not possible to access the docker container network 😞

If we want to make this configurable, there's 3 options I can think of:

• Add a boolean/toggle to the kind Config type
• Add a flag to kind create cluster to toggle the behaviour
• Add a new kind get kubeconfig to grab a kubeconfig for an existing cluster, and add a flag to that

Not sure what's most desirable here? 😄

from kind.

Each cluster has a different port for the API server so it should still be possible for them to talk to each other (?), the reason for localhost is that windows / mac do not support anything else as far as I know, because the container network is actually in a light VM.

from kind.

It's also possible that using a single network bridge will be problematic in the future, once we get multiple nodes. Using localhost and a random port makes things pretty portable.

Can federation support keying off of the address + port instead of just the address instead?

It would be quite easy to create a kubeconfig with the container IP address when on linux, but so far I've preferred keeping the environment as consistent as possible. Ideally users should be able to replicate CI very closely on their local machines, which is a major goal for kind.

from kind.

Ah, right. Thanks. Will think about this some more, while you can trivially get the container IP and rewrite it currently, it would be ideal to make this work portably.

from kind.

@BenTheElder @font
Currently I'm working on multi node and I want to be sure my effort is compliant with federation test requirements

What I'm proposing for multi-node is to

• always use IP:6443 as API server address for "infra-docker" communication. This allows multiple cluster but gets out of the way random port for node join/API server load balancing
• only when copying the kubeconfig on the host machine, use localhost + random port

What described above is a small variation of current state, and if I got this thread right, it neither fix nor prevent fixing the above issue for federation test. is that right or do you see problems in what I'm proposing?

Instead, if I can give my two cents on how to address federation requirement, I think that a possible solution is to add an option for retrieving the raw kubeconfig (kind get kubeconfig --raw, that will give you exactly the kubeconfig that exist in docker/with IP:6443), that can be reused by federation controller - assuming it will be run in docker too -. Wdyt?

from kind.

i think we should make the random port a UX knob.

from kind.

for random port see #178, we should allow setting the address as well, we'll set a random port if you don't specify a port.

from kind.

neglected to update here: the random port is a config knob now, and it is always 6443 inside the network, we still need to add a command to export the kubeconfig, with a --raw option or similar.

from kind.