To use existing AWS VPC subnets created outside of pulumi with EKS, the user must manually tag the desired subnets on AWS with the required Kubernetes k/v pair.
If they are not manually tagged, then Kubernetes will not be able to discover them when needing to create Public or Private Load Balancers on AWS in those subnets - unless those subnets were already in use by running instances of the Workers.
Specifically, the VPC and subnets that the Workers are running in are automatically tagged for us in AWS by the EKS service with the key: kubernetes.io/cluster/${clusterName}, and the value: shared, where ${clusterName} is the name of the new EKS cluster. The cluster name is not known to the user or Pulumi until after the cluster has been created and auto-named.
However, this tagging is not done for any other public or private subnets that the Workers aren't already running in, as they are 1) not occupied by running Workers and 2) subsequently, are not automatically tagged by the EKS service.
The manual tagging on these other subnets is a required work around to enable a couple of use cases, such as when a user wants to:
- Create a Public LoadBalancer in public subnets across AZs, when the cluster is configured to have its Workers run in private subnets.
- Create a Private LoadBalancer in an other private subnets across AZs, that the Worker instances are not already running in.
See this gist for a repro of use case # 1, where Workers are in private subnets, and a Public LoadBalancer Service never comes up if you don't have public subnets appropriately tagged in AWS for Kubernetes discovery. Once you properly tag a public subnet in the VPC for the repro, only then does the Public LoadBalancer get provisioned for the cluster.
After my attempt above, I tried going down the path of retrieving the existing public subnet object using a couple of ways listed below, to modify its .tags props, but this does not seem possible:
- Tried retrieving an existing subnet object using
aws.ec2.Subnet.get(...)
and
- Tried retrieving an existing Vpc object using
awsx.Network.fromVpc(...)
- Tried using
awsx.ec2.Subnet(...) constructor, with Vpc object returned from above as a param.
However, none of my attempts allowed me to modify the .tags prop on the existing subnets as needed.
The Vpc/awsx.Network object returned from awsx.Network.fromVpc(...) captures all private and public subnets already, so defining and leveraging this object IMO feels like its part of the right approach to ultimately: retrieve the existing subnet(s) in question, and tag them as needed after the cluster has been created, and its pulumi auto-generated name is known. I'm certainly open to hear other alternatives if I'm misunderstanding the use of the packages, or how to best integrate this work around into the right package(s).
It doesn't seem like this is necessarily an issue with pulumi/pulumi-eks, but more about better understanding how to leverage and/or improve @pulumi/pulumi, @pulumi/pulumi-aws and @pulumi/pulumi-awsx to auto-tag any existing subnets in AWS needed by the user.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent