add kubepod driver about buildx HOT 5 CLOSED

kube-pod seems consistent with docker-container

from buildx.

Bikeshedding: kubepod? kube-pod? k8s-pod? (I prefer kube-pod)

from buildx.

Maybe we should prioritize adding Deployment driver.

• docker buildx create --driver kube --driver-opt replicas=N will createDeployment with N replicas.
  We should not use StatefulSet because creating StatefulSet pod replicas is slow due to the pod creation order constraint.
• docker buildx rm will remove the Deployment
• docker buildx stop should be NOP? Or resize the Deployment to 0 replica?
• docker buildx build will connect to a Pod in Deployment using kubectl exec -it POD buildctl dial-stdio. An alternative way is to set up mTLS (which is mandatory to protect the daemon from the executor containers) and Service, but it is not straightforward to manage the lifecycle of the certificates.
• To choose a Pod in Deployment, we can implement both random and consistenthash. As we are not going to use StatefulSet, consistenthash will break on recreation of ReplicaSet (on rolling update of Deployment). Probably this is not a huge deal.
• docker buildx inspect will show the name of the Deployment.
• The user will be able to do rolling update of the buildkitd image using kubectl set image,
• The user will be able to change the number of the replicas using kubectl scale.

from buildx.

@AkihiroSuda SGTM, except the k8s objects should probably be created on build/bootstrap, not on create.

from buildx.

PR #167

from buildx.