open-cluster-management-io / api Goto Github PK

Currently, I followed quick start of registration to register a cluster, and the cluster registered succeeded, but I found the docs of cluster join process is outdated. For example:

• clusterrolebinding's name is not correct, I did not find any clusterrolebing named system:open-cluster-management:clusterName:agentName in my cluster, I think it should be open-cluster-management:managedcluster:clusterName(open-cluster-management-io/registration#55 gave me some clue)

5. hub-controller creates a clusterrolebinding on the hub with the identity of system:open-cluster-management:clusterName:agentName

The cluster join process doc is very important for newcomers to understand the whole cluster registration process, very nice if it could be updated(very pleased to try to do it by myself when I figure out the whole process). Thanks a lot.

currently when ManifestWork is deleted the resources created/managed by ManifestWork will also be deleted.

I propose that we add a "preserveOnDelete" to change this behavior.

when the preserveOnDelete is set to true ManifestWork controller will not delete the resources that it created

Pod Security Admission now requires labels on namespaces hosting privileged workloads; see https://kubernetes.io/docs/concepts/security/pod-security-admission/#pod-security-admission-labels-for-namespaces for details.

Privileged workloads need those labels to be defined on their namespaces; ultimately, they will need them before they can be deployed (which means they won’t be able to add them). It should therefore be possible to request them in the ManagedClusterAddon spec (of course, if there’s a better solution, that would be great too).

Below was the current docs this repo have.

% tree docs
docs
├── clusterhealthcheck.md
├── clusterjoinprocess.md
├── clusterset.md
├── design.md
├── development.md
└── manifestwork.md

0 directories, 6 files

@qiujian16 do we have any docs to describe the clusterclaim

After run make , it would return error msg:

vendor/github.com/openshift/build-machinery-go/make/targets/golang/../../lib/golang.mk:22: *** `go` is required with minimal version "1.15.2", detected version "1.14.15". You can override this check by using `make GO_REQUIRED_MIN_VERSION:=`.  Stop.
make: *** [generate-with-container] Error 2

It seems we should update the fedora:32 -> fedora:34 to satisfy the requirement.

The klusterlet CRD generated for kubernetes cluster which's version is <=1.16 is invalid

If we apply this CRD to a 1.16.3 kubernetes cluster, will get:

# kubectl apply -f operator/v1/0001_00_operator.open-cluster-management.io_klusterlets.crd.yaml
The CustomResourceDefinition "klusterlets.operator.open-cluster-management.io" is invalid:
* spec.validation.openAPIV3Schema.properties[spec].properties[registrationConfiguration].properties[featureGates].items.properties[mode].default: Forbidden: must not be set
* spec.validation.openAPIV3Schema.properties[spec].properties[workConfiguration].properties[featureGates].items.properties[mode].default: Forbidden: must not be set

Another validation result from GitHub action

multiple apis are put in one single types.go file which make it hard to read. It is better that we can separate them to multiple files.

if the project is outside of GOPATH directory and with the env var unset, running make update will generate the whole package path open-cluster-management/api/client/... in the project. the expected output should be omitting the extra parent paths i.e. client/....

one way to fix this is to switch the code-generator to controller-gen which runs perfect in go mod mode.

In some cases, the server in the Hub cluster kubeconfig file is a domain name. For example, as shown in the example below.

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: https://xxx.yyy.com

The contents of this kubeconfig file will be stored in the namespace(open-cluster-manager-agent) in the secret(bootstrap-hub-kubeconfig) as the startup configuration when the managed cluster is registered.

The registration-agent will use this bootstrap-hub-kubeconfig to create CRD(managedCluster) and CSR, but the following error message will be encountered in the logs.

E0623 03:02:35.268397       1 reflector.go:138] k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1.CertificateSigningRequest: failed to list *v1.CertificateSigningRequest: Get "https://xxx.yyy.com/apis/certificates.k8s.io/v1/certificatesigningrequests?limit=500&resourceVersion=0": dial tcp: lookup xxx.yyy.com on 10.xx.yy.zz:53: no such host

This is because the domain name of the Hub Cluster API Server is not available in the managed cluster's DNS servers. The solution to this problem is simply to write the host alias to the registration-agent pod. This is shown below.

spec:
hostAliases:
- ip: "xx.xx.xx.xx"
  hostnames:
  -  "xxx.yyy.com"

For the above problem and solution, the API needs to be modified, and the specific design and implementation are as follows.

Add a new field named HubApiServerHostAlias to the KlusterletSpec.

type KlusterletSpec struct {

	// ...

	// HubApiServerHostAlias contains the host alias for hub api server
	// +optional
	HubApiServerHostAlias *v1.HostAlias `json:"hub_api_server_host_alias,omitempty"`
}

Registration-operator Repo

Add a new rendering logic to manifests/klusterlet/management/klusterlet-registration-deployment.yaml template.

spec:
{{if .HostAlias }}
hostAliases:
- ip: {{ .HostAlias.Ip }}
  hostnames:
  {{range .HostAlias.Hostnames }}
  - {{ . }}
  {{end}}
{{end}}

Add helper function in pkg/helpers/helpers.go to get the value to be rendered from the Klusterlet configuration file.

func HubApiServerHostAlias(klusterlet *operatorapiv1.Klusterlet) *corev1.HostAlias {
	if len(klusterlet.Spec.HubApiServerHostAlias) == 0 {
		return nil
	}

	return &corev1.HostAlias{
		IP: klusterlet.Spec.HubApiServerHostAlias.IP,
		Hostnames: klusterlet.Spec.HubApiServerHostAlias.Hostnames,
	}
}

It would be good if we add integration test in api repo (maybe with kube builder). So we can at least check

1. whether the crd format is correct
2. whether the field validation is expected.

/kind feature