kapicorp / kapitan Goto Github PK
View Code? Open in Web Editor NEWGeneric templated configuration management for Kubernetes, Terraform and other things
Home Page: https://kapitan.dev
License: Apache License 2.0
Generic templated configuration management for Kubernetes, Terraform and other things
Home Page: https://kapitan.dev
License: Apache License 2.0
kapitan init
command to create an empty/example project skeleton.
$ pip install --user kapitan
Collecting kapitan
Downloading https://files.pythonhosted.org/packages/ac/01/613694f1f2eb99f73a6ea9ecc4e4d04492dc918048b438410c2faec88461/kapitan-0.21.0.tar.gz (79kB)
100% |████████████████████████████████| 81kB 617kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-exgfpmrk/kapitan/setup.py", line 55, in <module>
pkgs, new_links = install_deps()
File "/tmp/pip-install-exgfpmrk/kapitan/setup.py", line 41, in install_deps
with open('requirements.txt', 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
The inventory only support YAML right now, but it would be cool to add support for TOML, JSON and XML (would have to serialize back to YAML to feed into reclass).
Maybe check the keys containing
*key* *pass* *token* *secret* *pin* *security* *crypto*
and check the value entropy.
We should ignore gpg references.
Can be integrated as part of kapitan lint
It may be useful to do a simpler example for kaptian, something like a helloworld/hostname example that builds for multiple namespaces with a cluster.
For example, kapitan searchvar parameters.elasticsearch.replicas
will also match parameters.mysql.replicas
, as it only seems to look at the latest value.
e.g. kapitan inventory -t minikube-es -p parameters.secrets
would only show parameters.secrets variable struct.
Currently if something is undefined, the error message is an unhelpful Python stack trace with no indication of where the error actually was. Ideally it would be something more human readable.
Most compilation errors don't print the target name, making it harder to debug what the issue is.
Have kapitan use the jsonschema lib (if you enable it in your inventory).
jsonschema lib will validate the compiled file output structure for k8s/terraform/whatever other yaml resources.
links:
https://json-schema.org/implementations.html
https://pypi.org/project/jsonschema/
Feature request as discussed in #kapitan.
We have multiple teams (dev/ops/devops) and one infrastructure team. All of them have dedicated/own Git repositories and CI/CD pipelines to rollout applications on Kubernetes with manifests compile by kapitan. To prevent redundant data in classes we (infrastructure team) want to provide a central CMDB repository that contains shared (reclass) (inventory) classes and secrets.
Git-subtree is used to maintain a local copy of the global/shared CMDB Git repository.
This leads to classes/secrets that are distributed in multiple directories:
$ tree
.
`-- deploy/
|-- cmdb/
| |-- classes/
| | `-- kubernetes/
| | |-- clusters.yml
| | `-- namespaces.yml
| `-- secrets/
| |-- deploy-key
| `-- registry-readonly-key
`-- kapitan/
|-- inventory/
| |-- classes/
| | `-- abc.yml
| `-- targets/
| `-- mytarget.yml
`-- secrets/
`-- eu-central-1/
`-- monitoring/
`-- oidc_key
We need to tell kapitan to search for multiple classes
directories and secrets
directories.
Example:
$ cd deploy/kapitan/
$ kapitan compile \
--class-search-path "classes/:../cmdb/classes/" \
--secrets-search-path "secrets/:../cmdb/secrets/"
This might obsoletes/collides --search-path (?) and --secrets-path.
We already allow for jsonnet
to access the full inventory (cross target), and we should allow jinja
templates to access that as well.
This is helpful to create docs and canned scripts that need to have a global view across all targets.
I would like to create a INVENTORY.md to allow for looking at the inventory without running kapitan inventory
, so that users can understand what values are set to without running kapitan.
The INVENTORY.md doc contains the following
{{inventory.classes | pprint}}
{{inventory.parameters | pprint }}
This would look much better if there was a proper toyaml jinja filter.
Calling reclass and generating the inventory is an expensive process (~2 seconds). We do save the result using the memoise
decorator, however we still run it once for each thread.
We should load the inventory once before we create multiple threads and pass the result to each thread.
Secrets with GPG and especially automatic generation of secrets is very powerful.
However for some low-security setups (i.e. minikube or example folder) a user is required to have configured GPG and have the right key to decrypt secrets.
This forces the user to 2 choices:
The first option will force users to have alternative setups for demo/personal and prod environments
The second option will still force the user to setup gpg, and make more difficult to understand if an encrypted key is "unsecure" and auditing
I would like to encourage a healthy use of secrets that does not force users to diverge from the standard way for convenience.
My proposal is to create a new backend as an alternative to the gpg backend: a plaintext secret.
This enfoces a similar workflow but proving the benefit of a gpg-less setup.
Also, explicit use of this backend are easier to audit
example:
?{gpg:targets/prod/mysql/my_user_password}
?{plain:targets/example/mysql/my_user_password}
ultimately this can easily be templated with
?{${encryption_backend}:targets/${target_name}/mysql/my_user_password}
Based on kube.libjsonnet but using the new kadet/python input type create a new library that provides similar functionality for kubernetes.
If an incomplete/invalid secret tag is referenced from the inventory, compile will hang indefinitely:
Given the inventory (note the missing }
for my_secret):
parameters:
my_secret: ?{gpg:something_secret
other_stuff: yoda
and a jsonnet main.jsonnet file:
local kap = import "lib/kapitan.libjsonnet";
local inv = kap.inventory();
{
my_secret: inv.parameters.my_secret,
}
When many secrets are used, gpg will be running / invoked over and over again to decrypt each an every file containing just a little string.
It is certainly inefficient to run GPG for every such little string and it clutters the classes files to add gpg and the secret path for each time.
mysql:
storage: 10G
storage_class: standard
image: mysql:latest
users:
root:
password: ${gpg:allmysecretsinonefile:mysql:password}
-- This would allow decrypting and then caching a whole file containing multiple values once and allow for much faster compiles when many secrets are used.
Allow using whole gpg encrypted target files which are simply decrypted prior to rendering the templates.
this could be as easy as adding ".gpg" to your target yaml file by convention. Inside the classes those variables should be addressable just like any other variable, making the fact that this is actually sourced from an encrypted file transparent.
Let's assume the following use case:
A kapitan user wants to encrypt a password as a secret.
This password has to be used in:
Currently there is no nice way to do this with Kapitan.
Either you create a second inventory item manually and base64 it (in which case you don't have auto-generation with |randomstr) or you do |base64 in the inventory item but you have to base64 decode it everywhere you use it after kapitan secrets --reveal
happens (which means possibly way more code for developers or use cases where this is quite hard).
Suggestion:
password: ?{/secrets/mypassword|randomstr|encode:b64:sha256}
. This generates multiple secrets files which are encoded with the corresponding algorithm as well as plaintext.parameters....password_b64
or parameters....password
anywhere you want without having to apply other filters during reveal time.For an invalid target like the one below:
parameters:
target_name: mynamespace
namespace: ${target_name}
kapitan:
compile:
- output_path: pre-deploy
input_type: jsonnet
output_type: yaml
input_paths:
- components/namespace/main.jsonnet
Compilation fails with a KeyError, not suggesting that vars:
needs to be set:
Traceback (most recent call last):
File "/kapitan/kapitan/targets.py", line 84, in compile_targets
[p.get() for p in pool.imap_unordered(worker, target_objs) if p]
File "/kapitan/kapitan/targets.py", line 84, in <listcomp>
[p.get() for p in pool.imap_unordered(worker, target_objs) if p]
File "/usr/local/lib/python3.6/multiprocessing/pool.py", line 735, in next
raise value
KeyError: 'vars'
A valid target needs at least vars:target
, as below:
parameters:
target_name: mynamespace
namespace: ${target_name}
kapitan:
vars:
target: ${target_name}
compile:
- output_path: pre-deploy
input_type: jsonnet
output_type: yaml
input_paths:
- components/namespace/main.jsonnet
e.g. kapitan searchvar parameters.mysql.user
works, but kapitan searchvar parameters.mysql
doesn't return anything.
MacBook-Pro:empty_secret_issue rjmco$ echo -n 'important' | kapitan secrets -w gpg:important -t dummy -f -
Traceback (most recent call last):
File "/Users/rjmco/.pyenv/versions/3.7.0/bin/kapitan", line 11, in <module>
load_entry_point('kapitan==0.18.2', 'console_scripts', 'kapitan')()
File "/Users/rjmco/.pyenv/versions/3.7.0/lib/python3.7/site-packages/kapitan/cli.py", line 279, in main
secret_write(args, ref_controller)
File "/Users/rjmco/.pyenv/versions/3.7.0/lib/python3.7/site-packages/kapitan/cli.py", line 311, in secret_write
recipients = inv['nodes'][args.target_name]['parameters']['kapitan']['secrets']['recipients']
KeyError: 'secrets'
As a Vault user, I would like Kapitan to be able to pull information from Hashicorp Vault so that I can have a single authority on secrets in my organization.
Acceptance Criteria:
~/.vault-token
, uses VAULT_ADDR
, etc.)userpass
, token
, and cert
authentication mechanisms.Docker tags are not immutable and the same tag can be assigned to the same image over and over.
However they are easier to deal with than, for instance, sha digests.
In an example scenario, say that you have a deployment with 4 replicas, all referencing "my_image:latest".
If you scale up the deployment, or delete a pod say, a month later, it is not possible to guarantee that the exact same version is running.
The idea is to be able to specify an image by tag (i.e. my_image:latest) but a compile time, turn that tag into an immutable reference (sha)
This way it will be easier to have the guarantee of running the same image consistently for all pods.
When jsonnet fails to compile, the output does not easily allow to understand which target caused the jsonnet to fail.
Compiled working-target1 (1.86s)
Compiled working-target2 (1.86s)
Jsonnet error: failed to compile repository/components/cloudsql-proxy/main.jsonnet:
RUNTIME ERROR: field does not exist: cloudsql
repository/components/cloudsql-proxy/main.jsonnet:13:26-49 object <anonymous>
One possible solution is to write the target being compiled before attempting the compilation. Output could look like this
Compiling working-target1 (OK 1.86s)
Compiling working-target2 (OK 1.86s)
Compiling failing-target3
Jsonnet error: failed to compile repository/components/cloudsql-proxy/main.jsonnet:
RUNTIME ERROR: field does not exist: cloudsql
repository/components/cloudsql-proxy/main.jsonnet:13:26-49 object <anonymous>
When expiry key is set infinity, Kapitan fails to write a secret with GPG. Stack trace below:
2018-11-14 21:11:01,398 kapitan.refs.secrets.gpg DEBUG Key for recipient: XXX with fingerprint: YYY is expired, skipping
Traceback (most recent call last):
File "/Library/Python/3.6/bin/kapitan", line 11, in <module>
sys.exit(main())
File "/Library/Python/3.6/lib/python/site-packages/kapitan/cli.py", line 279, in main
secret_write(args, ref_controller)
File "/Library/Python/3.6/lib/python/site-packages/kapitan/cli.py", line 312, in secret_write
secret_obj = GPGSecret(data, recipients, encode_base64=args.base64)
File "/Library/Python/3.6/lib/python/site-packages/kapitan/refs/secrets/gpg.py", line 59, in __init__
fingerprints = lookup_fingerprints(recipients)
File "/Library/Python/3.6/lib/python/site-packages/kapitan/refs/secrets/gpg.py", line 188, in lookup_fingerprints
lookedup_fingerprint = fingerprint_non_expired(name)
File "/Library/Python/3.6/lib/python/site-packages/kapitan/refs/secrets/gpg.py", line 212, in fingerprint_non_expired
raise GPGError("Could not find valid key for recipient: %s" % recipient_name)
kapitan.refs.secrets.gpg.GPGError: Could not find valid key for recipient: XXX
kapitan lint
should check for things like:
If we have secrets/common
for example, kapitan secrets --validate-targets
or kapitan secrets --update-targets
will ignore it and only look at secrets/targets/
.
Kapitan is failing to create directories for Jinja targets.
$ kapitan compile -v -t global_tools
2018-04-06 23:54:41,982 gnupg DEBUG 231982: gpg --status-fd 2 --no-tty --debug ipc --fixed-list-mode --batch --with-colons --version
2018-04-06 23:54:41,983 gnupg DEBUG stderr reader: <Thread(Thread-1, initial daemon)>
2018-04-06 23:54:41,983 gnupg DEBUG stdout reader: <Thread(Thread-2, initial daemon)>
2018-04-06 23:54:41,985 gnupg DEBUG gpg: reading options from '/usr/local/google/home/ademaria/.gnupg/gpg.conf'
2018-04-06 23:54:41,986 gnupg DEBUG chunk: b'gpg (GnuPG) 2.2.2\nlibgcrypt 1.8.1\nCopyright (C) 2017 Free Software Foundation, Inc.\nLicense GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to t'
2018-04-06 23:54:41,995 kapitan.resources DEBUG Using reclass inventory config at: ./inventory/reclass-config.yml
2018-04-06 23:54:43,258 kapitan.targets DEBUG load_target_inventory: found valid kapitan target global_tools
2018-04-06 23:54:43,263 kapitan.resources DEBUG Using reclass inventory config at: /usr/local/google/home/ademaria/projects/health-templates/inventory/reclass-config.yml
2018-04-06 23:54:44,586 kapitan.utils DEBUG render_jinja2 rendering /usr/local/google/home/ademaria/projects/health-templates/scripts/global_tools/create_secrets.sh
2018-04-06 23:54:44,594 kapitan.targets DEBUG Wrote /tmp/tmpe0lyg75a.kapitan/global_tools/scripts/create_secrets.sh with mode 0750
2018-04-06 23:54:44,595 kapitan.targets INFO Compiled global_tools (1.34s)
2018-04-06 23:54:44,645 kapitan.targets DEBUG Compile pool terminated
2018-04-06 23:54:44,645 kapitan.targets ERROR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/site-packages/kapitan/targets.py", line 76, in compile_targets
shutil.rmtree(compile_path_target)
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/shutil.py", line 471, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/shutil.py", line 469, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: './compiled/global_tools'
2018-04-06 23:54:44,648 kapitan.targets DEBUG Removed /tmp/tmpe0lyg75a.kapitan
$ mkdir -p compiled/global_tools
$ kapitan compile -v -t global_tools
2018-04-06 23:55:26,086 gnupg DEBUG 232115: gpg --status-fd 2 --no-tty --debug ipc --fixed-list-mode --batch --with-colons --version
2018-04-06 23:55:26,087 gnupg DEBUG stderr reader: <Thread(Thread-1, initial daemon)>
2018-04-06 23:55:26,087 gnupg DEBUG stdout reader: <Thread(Thread-2, initial daemon)>
2018-04-06 23:55:26,091 gnupg DEBUG gpg: reading options from '/usr/local/google/home/ademaria/.gnupg/gpg.conf'
2018-04-06 23:55:26,091 gnupg DEBUG chunk: b'gpg (GnuPG) 2.2.2\nlibgcrypt 1.8.1\nCopyright (C) 2017 Free Software Foundation, Inc.\nLicense GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to t'
2018-04-06 23:55:26,106 kapitan.resources DEBUG Using reclass inventory config at: ./inventory/reclass-config.yml
2018-04-06 23:55:27,380 kapitan.targets DEBUG load_target_inventory: found valid kapitan target global_tools
2018-04-06 23:55:27,385 kapitan.resources DEBUG Using reclass inventory config at: /usr/local/google/home/ademaria/projects/health-templates/inventory/reclass-config.yml
2018-04-06 23:55:28,754 kapitan.utils DEBUG render_jinja2 rendering /usr/local/google/home/ademaria/projects/health-templates/scripts/global_tools/create_secrets.sh
2018-04-06 23:55:28,764 kapitan.targets DEBUG Wrote /tmp/tmplnho7_0j.kapitan/global_tools/scripts/create_secrets.sh with mode 0750
2018-04-06 23:55:28,764 kapitan.targets INFO Compiled global_tools (1.38s)
2018-04-06 23:55:28,765 kapitan.targets DEBUG Copied /tmp/tmplnho7_0j.kapitan/global_tools into ./compiled/global_tools
2018-04-06 23:55:28,765 kapitan.targets DEBUG Removed /tmp/tmplnho7_0j.kapitan
Similar to the current randomstrb64
, but instead sha256 the random string, with an optional parameter as the salt.
Would be great to allow pipes like |randomstr|base64
or randomstr|sha256:salt
to make the functions much more generic.
As a secrets encryption backend
When trying to update a secret e.g. secrets/targets/my-target/secret_sauce
, if a GPGError occurs, the content of secrets/targets/my-target/secret_sauce
is emptied.
Hi :)
As I've already posted in #kapitan on Slack I am trying to use https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus with kapitan.
For that I have fetched the libraries with jsonnet bundler jb
A vendor
directory was created containing the libraries.
ls -al vendor
total 0
drwxr-xr-x 11 bud staff 352 Jul 28 06:58 .
drwxr-xr-x 12 bud staff 384 Jul 28 07:04 ..
drwxr-xr-x 2 bud staff 64 Jul 28 06:58 .tmp
drwxr-xr-x 4 bud staff 128 Jul 28 06:58 etcd-mixin
drwxr-xr-x 5 bud staff 160 Jul 28 06:57 grafana
drwxr-xr-x 3 bud staff 96 Jul 28 06:57 grafana-builder
drwxr-xr-x 15 bud staff 480 Jul 28 06:57 grafonnet
drwx------ 20 bud staff 640 Jul 28 06:58 ksonnet
drwxr-xr-x 18 bud staff 576 Jul 28 06:56 kube-prometheus
drwx------ 15 bud staff 480 Jul 28 06:56 kubernetes-mixin
drwxr-xr-x 9 bud staff 288 Jul 28 06:58 prometheus-operator
I have defined a component monitoring that imports kapitan and kube-prometheus from vendor
directory
$ cat components/monitoring/main.jsonnet
local kap = import "lib/kapitan.libjsonnet";
local inventory = kap.inventory();
local kp = (import 'vendor/kube-prometheus/kube-prometheus.libsonnet') + {
_config+:: {
namespace: inventory.parameters.namespace,
},
};
{ ['00namespace-' + name]: kp.kubePrometheus[name] for name in std.objectFields(kp.kubePrometheus) } +
{ ['0prometheus-operator-' + name]: kp.prometheusOperator[name] for name in std.objectFields(kp.prometheusOperator) } +
{ ['node-exporter-' + name]: kp.nodeExporter[name] for name in std.objectFields(kp.nodeExporter) } +
{ ['kube-state-metrics-' + name]: kp.kubeStateMetrics[name] for name in std.objectFields(kp.kubeStateMetrics) } +
{ ['alertmanager-' + name]: kp.alertmanager[name] for name in std.objectFields(kp.alertmanager) } +
{ ['prometheus-' + name]: kp.prometheus[name] for name in std.objectFields(kp.prometheus) } +
{ ['grafana-' + name]: kp.grafana[name] for name in std.objectFields(kp.grafana) }
The inventory class for the above component.
$ cat inventory/classes/component/monitoring.yml
parameters:
namespace: "monitoring"
kapitan:
vars:
target: ${target}
namespace: ${namespace}
compile:
- output_path: kubernetes/manifests/monitoring
output_type: yaml
input_paths:
- components/monitoring/main.jsonnet
input_type: jsonnet
And the target.
$ cat inventory/targets/dev.yml
classes:
- component.monitoring
parameters:
a: b
target: dev
Compile without -J vendor
fails with
$ kapitan compile
Jsonnet error: failed to compile /Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:
RUNTIME ERROR: couldn't open import "grafana/grafana.libsonnet": [Errno 2] No such file or directory: '/Users/bud/code/valibud/kure/vendor/kube-prometheus/grafana/grafana.libsonnet'
/Users/bud/code/valibud/kure/vendor/kube-prometheus/kube-prometheus.libsonnet:4:2-36 thunk <import>
/Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:4:13-70 thunk <kp>
/Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:10:81-83 thunk <o>
std.jsonnet:1153:24
std.jsonnet:1153:5-33 function <anonymous>
/Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:10:64-99 thunk <a>
/Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:10:1-101 function <anonymous>
/Users/bud/code/valibud/kure/components/monitoring/main.jsonnet:10:1-101
Compile error: failed to compile target: dev
Import statements from all the libraries are relative.
Then I've ran with -J vendor
expecting it to work based on my understanding of the -J
flag.
$ kapitan compile -J vendor
Compile error: input_path for target: dev not found in search_path: components/monitoring/main.jsonnet
Copying all the libraries from vendor to repository root alongside components
and inventory
and running compile, as expected, works.
$ kapitan compile
Compiled dev (55.54s)
Thank you @ramaro for your patient support on Slack.
For posterity. Would be good to have a better/more helpful error message and a troubleshooting section.
I have experienced this on my linux desktop
Basically GPG is unable to call the pinentry-curses (or -tty) to read the password on the command line.
./compiled/my_target/vault/provision.sh
potential problem: ERROR: pkdecrypt_failed 83918950
Traceback (most recent call last):
File "/usr/local/google/home/ademaria/kapitan/bin/kapitan", line 11, in <module>
load_entry_point('kapitan==0.14.0', 'console_scripts', 'kapitan')()
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/site-packages/kapitan/cli.py", line 230, in main
secret_gpg_reveal_raw(gpg_obj, args.secrets_path, None, verify=(not args.no_verify))
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/site-packages/kapitan/secrets.py", line 249, in secret_gpg_reveal_raw
revealed = re.sub(SECRET_TOKEN_TAG_PATTERN, _reveal_gpg_replace, line)
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/re.py", line 191, in sub
return _compile(pattern, flags).sub(repl, string, count)
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/site-packages/kapitan/secrets.py", line 220, in reveal_gpg_replace
return secret_gpg_read(gpg_obj, secrets_path, token, **kwargs)
File "/usr/local/google/home/ademaria/kapitan/lib/python3.6/site-packages/kapitan/secrets.py", line 83, in secret_gpg_read
raise GPGError(dec.status)
kapitan.secrets.GPGError: decryption failed
export GPG_TTY=`tty`
sudo apt-get install pinentry-curses
sudo update-alternatives --config pinentry
eval $(gpg-agent --daemon --debug-level 9 --pinentry-program /usr/bin/pinentry-curses | tee /dev/tty)
We could of course use plain git submodules, but what if we only need only one file or folder from an external git repo, http url or even helm chart/templates ?
I would like to introduce the concept of modular dependencies in kapitan, and it would look something like this:
parameters:
kapitan:
compile:
...
dependencies:
- type: git
output_path: lib
source: [email protected]:grafana/grafonnet-lib.git
subdir: grafonnet
version: 0.1
- type: http
output_path: lib
source: https://raw.githubusercontent.com/bitnami-labs/kube-libsonnet/master/kube.libsonnet
version: 0.1
In this scenario, kapitan compile
will pull github.com/grafana/grafonnet-lib/grafonnet/*
and github.com/bitnami-labs/kube-libsonnet/master/kube.libsonnet
into lib
at the root. Dependencies will be cached and only re-pulled when the version changes, keeps things simple and we don't need to implement an extra kapitan subcommand.
Users have full control of what the source can be (jsonnet, libsonnet, jinja2 templates, k8s manifests etc.) and where it's placed inside the project (output_path), can be per target, class or global.
In terms of modularity, each type can be defined as a separate python module with the same name, as in kapitan/kapitan/dependencies/git.py
will control the functionality of type: git
; in the example above, subdir is a git module specific variable, we can also add branch, release etc.
@ademariag @gburiola @ramaro What do you think?
Compilation using the docker image on macOS fails randomly when trying to render jinja2 templates.
Kapitan v0.13.0, macOS 10.13.3, Docker version 17.12.0-ce, build c97c6d6.
...
Jsonnet jinja2 failed to render <path>/file.yml.j2: [Errno 9] Bad file descriptor: '<path>/file.yml.j2'
Jsonnet error: failed to compile <path>/main.jsonnet:
RUNTIME ERROR: [Errno 9] Bad file descriptor: '<path>/file.yml.j2'
/kapitan/kapitan/lib/kapitan.libjsonnet:2:32-89 function <anonymous>
<path>/config.jsonnet:13:17-87 object <anonymous>
<path>/kube.libjsonnet:208:37-49
<path>/kube.libjsonnet:208:28-50 thunk <a>
<path>/kube.libjsonnet:208:28-62 function <anonymous>
<path>/kube.libjsonnet:208:28-62 thunk <nonstrings>
<path>/kube.libjsonnet:209:23-33
<path>/kube.libjsonnet:209:12-34 thunk <a>
<path>/kube.libjsonnet:209:12-39 function <anonymous>
<path>/kube.libjsonnet:209:12-39 thunk <object_assert>
<path>/kube.libjsonnet:199:24-42 object <anonymous>
<path>/kube.libjsonnet:199:16-49 object <anonymous>
<path>/kube.libjsonnet:24:24-55 thunk <array_element>
<path>/kube.libjsonnet:170:14-45 object <anonymous>
<path>/deployment.jsonnet:(64:16)-(86:6) object <anonymous>
<path>/deployment.jsonnet:(62:10)-(87:4) object <anonymous>
<path>/deployment.jsonnet:(53:58)-(88:2) object <anonymous>
<path>/main.jsonnet:7:20-30 object <anonymous>
During manifestation
...
For example if we want to compile every time with --indent 4
, we could make use of the Makefile, but would be nicer to allow configuration of flag defaults in a config file such as .kapitan
, where we already store the latest kapitan version used for compilation.
.kapitan
can become something like:
version: 0.16.3
compile:
indent: 4
would be equivalent with running kapitan compile --indent 4
This is similar to how terraform
works.
tf saves the version of the terraform binary used to produce the terraform.tfstate
file and fails to run if you try to use an older version:
$ terraform plan
Error:
Terraform doesn't allow running any operations against a state
that was written by a future Terraform version. The state is
reporting it is written by Terraform '0.11.5'
Please run at least that version of Terraform to continue.
we should have a similar feature on kapitan
I want to generate google_iam_policy
for terraform, here is a sample code.
data "google_iam_policy" "admin" {
binding {
role = "roles/compute.instanceAdmin"
members = [
"serviceAccount:[email protected]",
]
}
binding {
role = "roles/storage.objectViewer"
members = [
"user:[email protected]",
]
}
}
How to generate two binding
sections with different values?
Thank you.
The function should accept parameters like key length
?{gpg:targets/${target_name}/private_key|rsa:4096}
Currently when running kapitan compile
, if gpg password is needed, the compilation for other targets will continue on the background making it difficult for a user to understand that entering the password is needed.
Eventually kapitan will error out with "encryption incomplete"
We should have a strategy to check if keyring is open/closed before compiling targets, or at least make it easier to enter the password.
cat .kapitan
version: 0.20.0-rc.0
kapitan compile
Traceback (most recent call last):
File "/Users/chifor/Library/Python/3.7/bin/kapitan", line 11, in <module>
sys.exit(main())
File "/Users/chifor/Library/Python/3.7/lib/python/site-packages/kapitan/cli.py", line 228, in main
check_version()
File "/Users/chifor/Library/Python/3.7/lib/python/site-packages/kapitan/utils.py", line 381, in check_version
if kapitan_config and kapitan_config["version"] and StrictVersion(kapitan_config["version"]) > StrictVersion(VERSION):
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/version.py", line 40, in __init__
self.parse(vstring)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/version.py", line 137, in parse
raise ValueError("invalid version number '%s'" % vstring)
ValueError: invalid version number '0.20.0-rc.0'
kapitan secrets --write/--update/--update-targets ...
does not base64 the encrypted data from secrets, instead it writes the encrypted binary blob directly to yaml:
data: !!binary |
...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.