kubernetes-retired / cluster-registry Goto Github PK
View Code? Open in Web Editor NEW[EOL] Cluster Registry API
Home Page: https://kubernetes.github.io/cluster-registry/
License: Apache License 2.0
[EOL] Cluster Registry API
Home Page: https://kubernetes.github.io/cluster-registry/
License: Apache License 2.0
The proposed authentication configuration revolves around a map[string]string
// Config is a map of values that contains the information necessary for a
// client to authenticate to a Kubernetes API server.
// +optional
Config map[string]string `json:"config,omitempty" protobuf:"bytes,2,rep,name=config"
My understanding is that in general we are now avoiding untyped maps for anything intended to be system-meaningful. They are hard to version & update, and we risk repeating the same problems we hit when using labels/annotations where we probably should have been using fields.
Should we consider a pattern more akin to health checks or volumes, with an effective union of strongly-typed options-types?
/sig multicluster
This is already at least partially done, but could be improved.
Some ideas:
Feel free to add more ideas!
I can't open the Google doc in the readme. Please update permissions so anyone with the link can read it.
@perotinus has repurposed kubefed
(now called crinit
in this repository) to simplify cluster registry installation in a host Kubernetes cluster. Add support to crinit so that it could be used to register clusters to or deregister clusters from a cluster registry.
This can be done before we actually have the tests; the main objective here is to get out of the habit of hitting the merge button by hand.
cc @perotinus
Hey Jonathan,
We looked at this yesterday. Do you want to connect the cluster-registry up to Reviewable?
"Please connect this repo at https://reviewable.io/repositories so that all PRs get linked to reviews. Thanks!"
Cloned from kubernetes/kubernetes#43123.
Today crinit waits infinitely for some of its operations to complete. This is suboptimal. We should introduce a hidden flag to allow timeouts.
Once #19 goes in, there will be an incomplete but runnable suite of bazel tests that should be run before PR submission.
This involves adding some configuration to the Prow config (https://github.com/kubernetes/test-infra/blob/master/prow/config.yaml) and the job config (https://github.com/kubernetes/test-infra/blob/master/jobs/config.json).
Since it is currently necessary to run ./update-codegen.sh
and bazel run //:gazelle
before running the bazel tests, I expect that we will have to use a custom execution scenario (https://github.com/kubernetes/test-infra/blob/master/scenarios/execute.py) and call the bazel scenario directly after running the codegen commands.
As part of the release process, we will need to make sure that licensing requirements for the compiled binaries and distributed source tarballs (if provided) are met.
/sig multicluster
The cluster registry needs a way to authorize requests. Currently, it allows authenticated callers full access. It should at least support r/w vs readonly usage.
The canonical Kubernetes way to handle this would be RBAC, but we don't currently support creating RBAC policies in the cluster registry, and it's not clear that the cluster registry should have a parallel set of RBAC rules to the main Kubernetes API server (and it probably should defer to another API server if it is an aggregated API server).
cf this comment in the cluster registry API doc.
I'm trying to understand the use case for the CloudProvider field.
What is the cloud provider in a multi-repo, multi-cloudprovider-implementation world? If there are two AWS cloudprovider implementations, should they have the same name or a different name? How are the CloudProvider names registered?
What if a single cluster spans multiple clouds? Should this be of []CloudProvider
?
The cluster registry should have a way to handle authorization when it is not being run as a delegated API server. It doesn't seem ideal to implement the RBAC APIs in the cluster registry, given the potential for confusion and mismatches in implementing an API for a k8s object. One potential approach here is to use an authenticating/authorizing proxy.
Split from #8, which is tracking work for delegated authorization.
/sig multicluster
This requires some API design work: the Status
object is currently empty. Someone will need to drive building consensus around an initial design for this object. Note that there may be several controllers interacting with the cluster registry as an expected use case, and the status object needs to take this into account.
Cloned from kubernetes/kubernetes#50540
(Not sure if this is relevant for crinit
.)
crinit
does API version discovery and then uses the preferred API group version that server supports. This breaks when server supports a newer version that the generated clientset that crinit
uses does not know about.
crinit
should choose the version that both server and it knows about.
From #65 (comment), we should consider how the crinit
tool can be named and how its interface should be organized.
crinit
to kubecr
(credit @font for the name) or something elsekubecr <aggregated|standalone> <init|update|delete>
.I think this is something we'd want to do on a beta timeframe, since at that point we will be expecting the tool's interface not to change significantly.
/sig multicluster
The API should be stable enough at this point to support replacing the Federation cluster API with the cluster registry. The details of this integration will need some design: it's not clear whether the Federation API server will aggregate the cluster registry, or include the API types directly, or something else entirely.
It would be extremely useful to have a helm chart to deploy the cluster-registry and the apiservice resource to add it into the aggregator.
@pmorie raised a few issues in #16 about the cluster registry API that were not addressed:
Besides this, there are a few issues from the cluster registry API design that also need to be addressed:
api.Endpoints
from k8s, for reasons described here, but after pushback more discussion is neededCABundle
in KubeAPIServer
And an issue from #29:
CloudProvider
fieldFrom a comment on #21, there might be reason to check in the generated artifacts to enable people who do not use bazel to depend on this repository; and perhaps to make the test infrastructure easier to wrangle. I am quite opposed to checking in artifacts unless it really is blocking a workflow, so this issue is tracking whether enough need to outweigh the pain of checked-in generated code.
If this is causing you problems, please bump this issue!
Generate OpenAPI and/or Swagger definitions for the cluster API.
/sig multicluster
The cluster registry should provide a way to guarantee that a cluster that has been added has not been changed out for another cluster at the same IP address. This could be via a certificate stored in the cluster that a controller validates against the registry, and that can be provided to the user to validate against the cluster in a defined way.
Now that the cluster registry prototype is done, we should write a design based on this knowledge and circulate it more broadly.
Cloned from kubernetes/kubernetes#50746.
crinit init
should continue if related resources already exist in the host cluster; or it should provide a --force
flag.
Kubernetes API reference doc has been an invaluable resource not just for Kubernetes users, but also for the development team. We need similar a doc for the cluster registry API both for our users and the development team.
/sig multicluster
We should have a draft of user documentation for how to bootstrap/set up/use a cluster registry. This should probably include some examples as well.
Migrated from a question previously in the cluster API doc:
Are the auth configuration objects conceptually ordered? Is the semantic meaning of the ordering defined by the cluster registry itself, or are users free to define their own semantic meaning to the ordering of auth configurations?
Migrated from the Cluster API doc:
How can the API expose information about the transport necessary to reach a cluster? Can/should the cluster registry provide info about proxies, etc. as a field? Is there enough need to make it a top-level field rather than an application-specific annotation?
Once the cluster registry is in a stable enough state, it should have a users guide distinct from the developers guide.
This may make sense to host here, or perhaps on a shared Kubernetes page.
cc @pmorie
Cluster registry users need reference docs for both API server and crinit binaries. The help text accessible through --help
command line flag serves its purpose, but it is also extremely valuable to have it on the web so that it is indexed and made quickly accessible through search engines. It saves users from downloading the binaries and running --help
every time they want to look up something really quick.
Update cluster-registry rules_go to 0.6.0. This should help enable bazel debug builds enabled by bazelbuild/rules_go#743.
Migrated from the Cluster API doc:
How should the Status field work? How do users intend to use the status field? How does the API indicate that the status is being updated by a tool vs a user vs no-one? Is that useful information? For one particular example, CloudProvider, which can be provided by a user or provided by a tool, does it belong in status?
Generic API servers can store state anywhere. The sample API server (the project I believe this project is based off of) happens only supports etcd, but since the cluster-registry expects to store very small amounts of data (~hundred of objects max?) custom resources don't create a crazy amount of overhead.
I'd like to propose that the clusterregistry should support storing state in custom resources. This would dramatically reduce the amount of overhead it takes to deploy it.
The cluster registry currently doesn't support namespaces, which are the main way that Kubernetes APIs support multi-tenancy. It's not clear whether we can use RBAC as-is to support multi tenancy in the cluster registry, or whether we will have to build some alternative support or modify the cluster registry in order to work more fluently with RBAC.
This is one reason that we would implement an authorizer other than "AlwaysAllow": in the simple single-tenant case, we can probably assume that authn==authz (that is, anyone who can authenticate is authorized), since there is only one resource type, the cluster,
Not a major issue now.
From #8 (comment):
How do we want to handle the different personalities that clusterregistry will assume when running as a delegated vs standalone apiserver?
Some things that will need to be addressed:
Some possible options:
/sig multicluster
We need to test version skew for the cluster registry, the hosting clusters and kubectl
. Ideally, we will have upgrade tests and e2e tests that can verify many of these things, but we do need to decide what is worth testing and then write the tests/frameworks necessary.
As a principle, since the cluster registry itself is so simple and foundational, the value of having these sorts of tests is very high, since it allows users to upgrade with more confidence and because the tests themselves are not that complicated to write.
/sig multicluster
All the [API] requirements under https://github.com/kubernetes/community/blob/master/contributors/design-proposals/multicluster/cluster-registry/project-design-and-plan.md#alpha need to be met.
/sig multicluster
This is mainly to support authorization on groups of clusters. Namespaces are the obvious mechanism for this, but they are not the only one and we need to do some design work here to find the right approach.
/sig multicluster
Close criteria:
GoDoc seems to require doc.go
files in packages that it scans. Add them where appropriate, and make sure that there are no other things to do to integrate with GoDoc.
/sig multicluster
This is a bit loosely defined, and will probably require a mini-design (or at least a comment with some ideas in this issue).
Cloned from kubernetes/kubernetes#50817.
"It would be great to allow crinit init
to accept a file just like what we do for kubectl apply -f <filename>
, so that I can put all parameters in a file which is easy to maintain."
A testing plan will define what needs to be covered.
/sig multicluster
Cloned from kubernetes/kubernetes#50734.
crinit init
will install etcd, but the etcd is not secured, we should enable secure mode for etcd.
Details about etcd TLS
This tracks the work to modify crinit
to deploy the cluster registry using API aggregation. See https://kubernetes.io/docs/tasks/access-kubernetes-api/setup-extension-api-server/ for required steps.
Based on @ericchiang's comment (#58 (comment)).
dep
is currently not quite ready for use with all of Kubernetes, and using it will potentially expose us to incompatibilities; however, there is activity in other k8s repos to improve dep
support. Since updating vendored deps is not a common activity, it's not critical to have it be a completely smooth experience, but it should also not be so arcane and messy as to be undoable. We need to evaluate further.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.