helm / helm Goto Github PK
View Code? Open in Web Editor NEWThe Kubernetes Package Manager
Home Page: https://helm.sh
License: Apache License 2.0
The Kubernetes Package Manager
Home Page: https://helm.sh
License: Apache License 2.0
The vocabulary currently used by DM is confusing:
templates
.types
.configurable resource types
. There are no other uses of this term, unlike in DM for GCP.On top of this confusion, the term type
is opaque. Users looking for simple templates might not think to look for types. Everyone knows what a template is. It's a piece of content that contains substitution points. However, a type could be just about anything.
We need to clean up the terminology, and focus more on making it intuitive to the Kubernetes community than on following the precedents set by DM for GCP.
Automated tests should assert invariants about the chart version associated with a pull request in kubernetes/charts
.
It's in the wrong place in types/replicatedservice/v1/replicatedservice.py
Because users confuse it with metadata.name. Please use app: instead.
I want to be able to deploy this like I do other systems in Kubernetes via declarative config in kubectl create (e.g., without requiring local daemons/binaries running in a specialized local environment).
Currently, the template registry is embedded in this repository. It should be its own repository for several reasons:
When you try to delete a deployment but it cannot find certain resources the deployment is not deleted.
To reproduce:
Deploy the Guestbook example:
dm deploy guestbook.yaml
Manually delete a replication controller:
kubectl delete rc frontend-rc
Now try to delete the Guestbook deployment
dm delete guestbook.yaml
This returns an error:
cannot delete configuration: kubetcl failed for resource: frontend-rc: exit status 1: Error from server: error when stopping "STDIN": replicationControllers "frontend-rc" not found
And deployment manager still contains the guestbook.yaml deployment:
dm list
includes guestbook.yaml
A deployment needs to have information about the resources created:
Just a simple jinja or python syntax error yields:
$ dm --service=http://localhost:8080 deploy types/redis/v1/redis.yaml
2015/11/11 15:40:27 cannot deploy template named redis.yaml: status code: 400 status: 400 Bad Request : cannot expand template named redis.yaml (template expansion:expander service response:&{400 Bad Request 400 HTTP/1.1 1 1 map[Date:[Wed, 11 Nov 2015 23:40:27 GMT] Content-Length:[642] Content-Type:[text/plain; charset=utf-8]] 0xc8204c20c0 642 [] false map[] 0xc82040a0e0 }
):
imports:
resources:
GCP DM has the gcloud CLI which will, for an imported template, also look for and import its schema file and recursively pull in all imports defined within. e.g.,
imports: [foo.py]
may end up passing foo.py, foo.py.schema, and foo-helper.py to the API.
The DM client should also support this.
When a deployment fails, we should store error messages along-side the status inside the deployment
Currently, expandybird and resourcifier are called synchronously by manager. These calls should be asynchronous, so that we can parallelize execution where permitted by dependencies.
Make expansion run in a sandbox to guarantee that it's well behaved.
$ dm --service=http://localhost:8080 deploy types/redis/v1/redis.yaml
2015/11/11 15:37:20 cannot deploy template named redis.yaml: status code: 400 status: 400 Bad Request : cannot create configuration (deployer service response:
&{400 Bad Request 400 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff] Date:[Wed, 11 Nov 2015 23:37:20 GMT] Content-Length:[618]] 0xc82000e360 618 [] false map[] 0xc82040a000 }
)
$ cat log/manager.log
2015/11/11 15:37:14 Version: 0.0.1
2015/11/11 15:37:14 Listening on port 8080...
2015/11/11 15:37:16 manager: create deployment: handling request:POST /deployments
2015/11/11 15:37:16 manager: create deployment: handling request:POST /deployments
2015/11/11 15:37:16 Creating deployment: redis.yaml
2015/11/11 15:37:17 created deployment: &{redis.yaml 1 2015-11-11 15:37:17.613537833 -0800 PST 2015-11-11 15:37:17.613538146 -0800 PST 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC Creat t/r/v/redis.yaml d/dm.go X
ed map[]}
2015/11/11 15:37:17 Added manifest manifest-1447285037613578429 to deployment: redis.yaml
2015/11/11 15:37:20 deleted deployment: &{redis.yaml 1 2015-11-11 15:37:17.613537833 -0800 PST 2015-11-11 15:37:17.613538146 -0800 PST 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC Created map[manifest-1447285037613578429:0xc82049ab00]}
2015/11/11 15:37:20 manager: create deployment: returning response: status code:400, status:cannot create configuration (deployer service response:
&{400 Bad Request 400 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff] Date:[Wed, 11 Nov 2015 23:37:20 GMT] Content-Length:[618]] 0xc82000e360 618 [] false map[] 0xc82040a000 }
)
::1 - - [11/Nov/2015:15:37:16 -0800] "POST /deployments HTTP/1.1" 400 282 "" "Go-http-client/1.1"
$ cat log/resourcifier.log
2015/11/11 15:37:14 Version: 0.0.1
2015/11/11 15:37:14 Listening on port 8082...
2015/11/11 15:37:17 resourcifier: create configuration: handling request:POST /configurations
2015/11/11 15:37:18 kubetcl failed for resource: redis-master: exit status 1: Error from server: error when creating "STDIN": service "redis-master" already exists
2015/11/11 15:37:18 kubetcl failed for resource: redis-master-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-master-rc" already exists
2015/11/11 15:37:19 kubetcl failed for resource: redis-slave: exit status 1: Error from server: error when creating "STDIN": service "redis-slave" already exists
2015/11/11 15:37:20 kubetcl failed for resource: redis-slave-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-slave-rc" already exists
2015/11/11 15:37:20 resourcifier: create configuration: returning response: status code:400, status:kubetcl failed for resource: redis-master: exit status 1: Error from server: error when creating "STDIN": service "redis-master" already exists
kubetcl failed for resource: redis-master-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-master-rc" already exists
kubetcl failed for resource: redis-slave: exit status 1: Error from server: error when creating "STDIN": service "redis-slave" already exists
kubetcl failed for resource: redis-slave-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-slave-rc" already exists
::1 - - [11/Nov/2015:15:37:17 -0800] "POST /configurations HTTP/1.1" 400 618 "" "Go-http-client/1.1"
Given that we want members of the community to contribute type definitions, we should provide a guide that tells them how, instead of leaving it for them to try and figure out on their own.
Currently, resourcifier relies on kubectl to instantiate resources. However, it should be able to run arbitrary command line processes in a container, based on configuration metadata.
godeps or new tool go builder (http://dave.cheney.net/2015/06/09/gb-a-project-based-build-tool-for-the-go-programming-language) might be more appropriate as suggested by bburns@.
Currently, our CLI has the generic name client
and takes its action from a flag. We could make it friendlier by changing the name to dm and taking the action as the first argument. With those changes, the user experience would change from this:
client --action=deploy <template_files>
to something like this:
dm deploy <template_files>
GCP DM supports things like:
imports:
The client should also understand how to do this.
The type registry in this repository is useful for pedagogical purposes, but it doesn't serve well as a destination for publishers. We need a standalone type registry, also owned by CNCF, as described by this comment.
Currently, we pull type definitions from URLs. We then use the URL as the type name in DM, which makes for awkward REST queries, since the URL must be encoded.
A simpler approach would be to do what the go command does, and let the user specify well known registries by just their prefix (i.e., github.com, bitbucket.org, code.google.com, etc.). See go help importpath for details.
We need to do a run through and get some more consistent error messaging throughout the codebase.
When using an external template, the manifest is not populated with the contents of the template, or its schema or sub-imports.
There are some useful examples for DM on GCP. We should migrate them to the templates registry for Kubernetes.
It loads expander locally with incorrect paths.
We may want to remove it and reimplement it as a service call to an expand API in the manager that proxies to expandybird.
As a Kubernetes user, I can write DM templates that bind output parameters from a resource into input parameters in dependent resources.
Could start by copying the command line from the main README.
$ dm deploy redis:v1
2015/11/11 11:25:34 cannot deploy template named : status code: 400 status: 400 Bad Request : cannot expand template named (template expansion:expander service response:&{400 Bad Request 400 HTTP/1.1 1 1 map[Date:[Wed, 11 Nov 2015 19:25:34 GMT] Content-Length:[622] Content-Type:[text/plain; charset=utf-8]] 0xc20824e300 622 [] false map[] 0xc208033ad0 }
):
resources:
$ kubectl logs expandybird-rc-dj2di
2015/11/11 19:22:27 expandybird: expand: handling request:POST /expand
2015/11/11 19:22:28 Expansion process: pid: 14 SysTime: 20ms UserTime: 84ms
2015/11/11 19:22:28 expandybird: expand: returning response: status code:400, status:error (error expanding template : Traceback (most recent call last):
File "./expansion/expansion.py", line 372, in
main()
File "./expansion/expansion.py", line 369, in main
print Expand(template, imports, env=env, validate_schema=validate_schema)
File "./expansion/expansion.py", line 57, in Expand
raise ExpansionError('config', str(e))
main.ExpansionError: 'properties' is undefined Resource: config
) expanding template:
&{ resources:
We delete deployments in the system when they fail, so any k8s resources that may have been deployed are left in the system.
We should not delete the deployment but leave it failed, so that user can issue a delete or update to correct the issue.
There is useful documentation for DM on GCP that we don't yet have for Kubernetes. We should migrate it to this project.
As a Kubernetes user, I see that the configuration artifacts I create using open source DM are persisted in the cluster.
See gingko.
Currently, DM knows how to pull types from URLs. It also understands type schemas that contain metadata describing the type, as well as JSON schema describing the format of the input parameters. We should build on these mechanisms to further formalize the notion of a type registry by:
Resource definitions all have things like:
This should really be filled in from the type.
I need to be able to specify a deployment name when deploying a type or config.
This should probably be dm deploy <type | template> so deployment name is in the same position as get, etc.
Currently, there's no quality assurance for types pushed to the registry, other than review by the registry owners. We've talked about putting a test framework in place, to check for things like:
Tests should run when a type is created or updated. There needs to be a way to add type specific test suites to the type/version directory.
Currently, we require an exact match of the version to resolve chart reference. Instead, we should use semver semantics, meaning that:
Use https://github.com/kubernetes/kubernetes/blob/master/.travis.yml as a starting point. @vaikas-google: allowed travis access to this repo on 10/22. Need to find out if travis is still the recommended solution for build and test automation.
As a Kubernetes user, I can rely on DM to instantiate Kubernetes resources in an order that satisfies declared dependencies between resources.
Rather than just deploying to default
, consider deploying to a unique namespace, e.g. deployment-manager
, so that DM resources don't clutter the default namespace.
googlebot, k8s-bot, ... Need to find out which bots are required, and how to set them up.
It should be possible to bundle one or more versions of a type, along with their schemas, in an archive with a metadata file that describes its contents - i.e., something like a JAR. DM should be able to resolve type references that point to such archives.
It should be possible to indicate a version for a type, other than through its name, and a template that uses a type should be able to indicate the version range it will accept, and/or we should support semver based version alignment.
Currently, Kubernetes users have to install DM onto their clusters before they can use it. Make DM a cluster add-on, so that they can enable it with a command line switch.
Assuming #22, the CLI should be able to:
We do not need a way to fetch a type definition or schema, a place to put type definitions or schemas on the localhost, or a way to upload changes to a type definition or schema, since git provides all of that functionality for us, with many supporting features, such as issues, pull requests, etc.
This approach is consistent with our philosophy that config is code, and should be treated as such.
Several users have stumbled trying to bootstrap DM. Typically, they run afoul of getting python configured correctly. However, bootstrapping is unnecessary. It's a novelty and fundamentally, we don't need to modify the user's machine to deploy DM. We built the bootstrap initially to get a feeling for how things would work, but it's just as valid, much faster and much less error prone to create the resources for DM in the cluster using a flat kubectl configuration file. So, instead of subjecting first time users to python package installs and running processes locally, we should provide a configuration file to streamline getting started. We should also preserve the bootstrap file and the accompanying instructions as an example.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.