[request]: Security feature request to be able to run"proxy/init" container outside the "App mesh Injector" about aws-app-mesh-controller-for-k8s HOT 6 OPEN

Related issue in Kubernetes kubernetes/kubernetes#69882. This will make it simple to use CNI without running a separate controller.

from aws-app-mesh-controller-for-k8s.

kubernetes/kubernetes#69882 is still open.

tldr: CNI in Kubernetes that needs to read Pod annotations to perform its actions currently needs to read pod-info from API server (watch pods). This is logic is not performed in amazon-vpc-cni-plugins.

At this point best bet is to contribute appmesh support under https://github.com/aws/amazon-vpc-cni-k8s. Asking @mogren and @ofiliz for guidance.

from aws-app-mesh-controller-for-k8s.

@kiranmeduri are you recommending that we essentially subsume the functionality of aws-appmesh-proxy-route-manager into the VPC CNI routed-eni plugin? We would look for pod annotations and create the iptables rules as appropriate (basically, what aws-appmesh-proxy-route-manager currently does in-process in the sidecar)?

from aws-app-mesh-controller-for-k8s.

@kiranmeduri The CNI has a watcher already, scoped to pods on the current node:

https://github.com/aws/amazon-vpc-cni-k8s/blob/5283eaebe94dc6bc02d53c5106011e25133134a3/pkg/k8sapi/discovery.go#L126-L130

What is it that aws-appmesh-proxy-route-manager does?

from aws-app-mesh-controller-for-k8s.

I don't think we should replicate the logic in aws-appmesh CNI plugin. I assume you meant that the IPAMd daemon (not the vpc-cni-k8s CNI plugin itself) can watch the pods. In that case, instead of replicating the logic, IPAMd could either invoke the aws-appmesh binary itself, or use another mux CNI plugin to do so. However that work would be incompatible with CNI2 and vpc-resource-controller.

A much better solution is to do a full working backwards. Until then, use the init container instead of the CNI plugin.

from aws-app-mesh-controller-for-k8s.

Adding this logic to existing CNI plugin would be limiting for multiple reasons 1) what ofiliz@ mentioned 2) we don't want to have a tight coupling/hard dependency between AWS CNI and App Mesh use-uses. There might be clusters which run a different CNI plugin so a CNI with init container only logic chained with other CNI plugins would be better 3) complexity in the VPC CNI (it doing more than one thing), bigger testing matrix, release dependencies across these use-cases

from aws-app-mesh-controller-for-k8s.