carvel-dev / ytt Goto Github PK
View Code? Open in Web Editor NEWYAML templating tool that works on YAML structure instead of text
Home Page: https://carvel.dev/ytt
License: Apache License 2.0
YAML templating tool that works on YAML structure instead of text
Home Page: https://carvel.dev/ytt
License: Apache License 2.0
there might be cases where instead of removing, merging or replacing it might be necessary to modify yaml node. one simple example might be:
#@ def incr_by_2(val): return val + 2
spec:
#@overlay/modify incr_by_2
replicas:
to increment replicas by two in a particular environment. this assumes that base value is meaningful and relative values (or modification of base values in general) makes "business" sense.
@scothis brought another example where he was overlaying some data on top of knative manifests (https://github.com/knative/serving/blob/d7f29bd1a436bd651b2fa5ecf042ece6bf7cb09d/config/config-network.yaml#L105) and wanted to escape {{
as they would not be compatible with helm (he was trying to create helm chart on the fly).
I often use process substitution, especially if I'm bringing down secrets using a vault tool like lpass/credhub.
I want to use the command: ytt template -f pipeline.yml -f <(..cmd-that-gets-secrets..) but then I hit this:
https://github.com/k14s/ytt/blob/4b7b67ef6d88cc4777d38e9fbd310daa567961fa/pkg/files/file.go#L203
Eg:
$ ytt template -f <(echo "---")
Error: Expected file '/dev/fd/63' to be a regular file, but was not
Is this really just guarding against directories?
As of 0.17.0, text templating requires the annotation for all parent nodes.
An example where there is tedious is the construction of concourse pipelines: we use loops to design the jobs, and each job uses text templating (i.e. fill in a release name to build). Upgrading from prior to 0.17.0 to latest as of this writing requires the annotation in many places for us.
Would it be possible to have a CLI option or doc-wide annotation to enable? Is there a perceived/real performance cost to this?
Hi!
It would be nice to be able to pass expressions (that use variables) to the load()
statement. Currently only string literals are allowed. Is that a limitation of starlark
or was it a design choice of ytt
?
My case
I have a project where I'll be using functions for creating parts of the configuration and those functions have a different implementation depending on the "configuration variant" I'm using. In order to import the correct version of the function I wanted to use something load(variant + '.star', 'my_function')
, where variant
is a variable storing the name of my configuration variant.
Hi there!
According to Starlak's spec on the load
statement, the first "argument" is left to be interpreted by the application (ytt
itself in our case). I think it might be a good idea to add a note in the docs explaining how ytt
interprets that.
I'm raising this issue because I couldn't find in the documentation and examples information regarding a problem I faced while using load
. My use case requires passing yml files individually via ytt -f
, and it took me a while to figure that I have to pass the included file via -f
as well.
For example, if I go to example-load
and run the following command, it would fail:
$ ytt -f config.yml
Error:
- cannot load funcs.lib.yml: Expected to find file funcs.lib.yml
config.yml:5 in <toplevel>
L #@ load("funcs.lib.yml", "func1", "func2")
Now, if I include the files to the list, it works properly:
$ ytt -f config.yml -f funcs.star -f funcs.lib.yml
func1_key:
name: max
cities:
- SF
- LA
func2_key:
name: joanna
cities:
- SF
func3_key:
- 1
- 2
- key: value
It would be nice if there was documentation on the load
function (and maybe on ytt
CLI as well ๐).
Note: it seems that #9 is a related issue, but it talks about improving the error message but not about the project's documentation.
Best regards,
Gustavo Sousa
I happened to find this tool and tried it, but stumbled upon some unexpected behavior.
Hope the following improvements would help new users.
@data/values
annotation should be included in the input when --data-value
option is given.--output-directory
option will empty the target directory.(Apologies, this will be a vague github issue-- about to leave for vacation)
Any thoughts on if/how ytt should/could be leveraged as a templating engine for helm charts as an "official" pluggable backend to be used for a helm chart template? Noticed there weren't any github issues logged about it. And I'm under the impression the helm engine:
used by a chart swappable, though is gotpl
by default (and mostly only gotpl
appears used?)
Could be completely wrong about all of this, and I have done zero research into how helm templating backends work, let alone if they're swappable at all. Just figured I'd put the question out there. Feel free to close anytime.
Hi,
We (CF Release Integration) use the hasattr
helper function as part of our tooling to generate a Concourse pipeline from a template and an input file. One of the things we do is add debug jobs if the "debug" attribute is set for a release in the input file for the pipeline. For example, given this template snippet (greatly simplified):
#@ load("@ytt:data", "data")
---
groups:
- name: #@ r.name
jobs:
#@ for r in data.values.releases:
- #@ "update-" + r.name
#@ if hasattr(r, "debug"):
- #@ "destroy-" + r.name + "-debug-env"
#@ end
#@ end
and this input file:
#@data/values
---
releases:
- name: binary-buildpack
repository: cloudfoundry/binary-buildpack-release
debug: true
running ytt -f .
generates the following pipeline yml:
groups:
- name: binary-buildpack
jobs:
- update-binary-buildpack
- destroy-binary-buildpack-debug-env
During normal operation, we do not need to run in "debug" mode, so we simply remove the debug
attribute from the release in the input file and those jobs are not rendered in the pipeline manifest:
groups:
- name: binary-buildpack
jobs:
- update-binary-buildpack
However, since v0.19.0 of ytt
was released, we started receiving the following error instead of the expected generated pipeline yml:
Error:
- interface conversion: interface is nil, not starlark.Value
If I add the debug: x
attribute to the input file, then everything works as expected with the latest version of ytt
(v0.20.0 as of the time of writing), but we were hoping to not have to do this.
Please let me know if you have any questions about this issue.
Thanks,
Dave
ytt errors when parsing a string with the character sequence (@
. I noticed this when trying to load a file that contained a JSON path expression.
$ echo 'JSONPath: .status.conditions[?(@.type=="Ready")].status' | ytt -f -
Error: Compiling YAML template 'stdin.yml': Missing code closing '@)' at line 1 col 46
chatting with @cdutra, it might be interesting to generate json to template some of the more advanced terraform configuration
example-load-custom-library
does not seem to be working. I renamed the directory as instructed but I still get this error:
Error:
- cannot load @github.com/k14s/test-lib:funcs.star: Could not find private library (directory '_ytt_lib' missing?)
config.yml:11 in <toplevel>
L #@ load("@github.com/k14s/test-lib:funcs.star", "testfunc")
tree . :
.
โโโ _ytt_lib
โ โโโ github.com
โ โโโ k14s
โ โโโ test-lib
โ โโโ funcs.star
โโโ config.yml
Is it possible to also load a file in the same directory, eg:
#@ load("funcs.lib.yml", "source")
tree .:
.
โโโ config.yml
โโโ funcs.lib.yml
ytt version: 0.3.0
the ytt
user may wish to provide data values in the form of environment variables or by passing values through the command line. this is particularly beneficial when providing secrets.
As a user of ytt
I expect it to provide me with either of the options above to provide or override values.
currently following works
$ ytt --data-values-inspect -f values.yml=<(echo -e "#@data/values\n---\nstores: [1, 2, 3]")
stores:
- 1
- 2
- 3
however, it would be nice to have following work as well:
$ ytt --data-values-inspect --data-values-file-yaml <(echo "stores: [1, 2, 3]")
stores:
- 1
- 2
- 3
Related Conversations:
Hi!
The current documentation on @overlay/replace
says that it is "valid for both map and array items", however I was able to successfully apply it to a whole document as well. For example:
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({'foo': 'bar'}), expects="0+"
#@overlay/replace
---
foo: "baz"
note: "I am replacing the YAML document!"
The ability of replacing the YAML document is useful to me. Can I safely use it? If so, I think it would be nice to update the documentation.
alternatively, as @jbeda suggested, provide YAML linting facilities.
podAnnotations:
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({})
---
env:
#@overlay/match missing_ok=True
JAVA_OPTS: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaajjjj jjjjjjjjjjjjjjjjjjjjj jjjjjjjjjjjjjjjjjjjjjjjjjj jjjjjjjjjj jjjjj'
kubernetes:
#@overlay/remove
volumes:
--------------
Produces :
env:
JAVA_OPTS: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaajjjj jjjjjjjjjjjjjjjjjjjjj
jjjjjjjjjjjjjjjjjjjjjjjjjj jjjjjjjjjj jjjjj
kubernetes:
podAnnotations:
prometheus.io/scrape: true
prometheus.io/port: 8008
prometheus.io/path: prometheus_metrics
We see that JAVA_OPTS value is split across multiple lines. Now if I eliminate the spaces, it does preserve the line. Here is overlay with no spaces :
env:
JAVA_OPTS: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaajjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
kubernetes:
podAnnotations:
prometheus.io/scrape: true
prometheus.io/port: 8008
prometheus.io/path: prometheus_metrics
Input:
x: #@ { "first": "is included"}
second: is ignored
The second line is silently ignored.
Also here:
--- #@ { "first": "is included"}
this line: is ignored too
If this is not valid, I'd expect an error message instead of silently ignoring part of the input.
When trying to use a multiline comment in an anchor, ytt panics:
sth: &use
path:
args:
- |
foo
here: *use
error: panic: expected len of sequence children to match len of children line nums [recovered]
panic: expected len of sequence children to match len of children line nums
goroutine 1 [running]:
github.com/k14s/ytt/pkg/yamlmeta/internal/yaml%2ev2.handleErr(0xc00011d6a8)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamlmeta/internal/yaml.v2/yaml.go:297 +0x9a
panic(0x8acac0, 0xa81900)
/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/k14s/ytt/pkg/yamlmeta/internal/yaml%2ev2.(*decoder).sequence(0xc000094a40, 0xc0001322a0, 0x8c8a40, 0xc0001089a0, 0x194, 0x8c8a40)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamlmeta/internal/yaml.v2/decode.go:565 +0xa71
github.com/k14s/ytt/pkg/yamlmeta/internal/yaml%2ev2.(*decoder).unmarshal(0xc000094a40, 0xc0001322a0, 0x8c8a40, 0xc0001089a0, 0x194, 0x194)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamlmeta/internal/yaml.v2/decode.go:344 +0x105
github.com/k14s/ytt/pkg/yamlmeta/internal/yaml%2ev2.(*decoder).mappingSlice(0xc000094a40, 0xc0001320e0, 0x8bfee0, 0xc00007acc0, 0x197, 0x197)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamlmeta/internal/yaml.v2/decode.go:696 +0x4bd
github.com/k14s/ytt/pkg/yamlmeta/internal/yaml%2ev2.(*decoder).mapping(0xc000094a40, 0xc0001320e0, 0x8c8a40, 0xc000108910, 0x194,
#! Trying to use the multiline yaml ? : notation for map entries doesn't give the expected results:
#! This works
? ac
: bd
#! But this does not (it gives ee:null not ee:fg)
? ee
: #@ 'f' + 'g'
#! Nor this (it gives null:ax not ax:bd)
? #@ 'a' + 'x'
: #@ 'b' + 'd'
#! This one gives an error if you uncomment it.
#! ? #@ 'y' + 'z'
#! : asdf
Is there a reason standard YAML comments are not supported? Since ytt
seems to try and sort of follow YAML syntax, I expected it to be ignored by ytt
.
$ cat > example.yml <<EOF
earlier: former
# later: latter # TODO someday
EOF
$ ytt template -f example.yml
Error: Unknown comment syntax at line example.yml:2: ' later: latter # TODO someday': Unknown metadata format (use '#@' or '#!')
I could imagine #something
without a following space to be something ytt
doesn't support due to theoretical ambiguity, but #spacesomething
seems reasonable for ytt
to ignore as standard YAML convention and non-ytt
directive-style.
I understand #!
is an alternative, but that's just more of a migration/difference and don't think I understand why it should be needed.
(meant to submit as @dpb587 from personal side-project learnings)
currently some top level errors do not include file location:
pivotals-iMac-2:serving pivotal$ ytt -f config/ --file-mark '**/*:type=yaml-plain'
Error: Unmarshaling YAML template: yaml: line 6: did not find expected <document start>
in example-overlay-files, ops2.yml adds multiple map items (under annotations key) [1]. it has to specify missing_ok=True for per key because to indicate that it's ok to create-or-update. need to come up with a way to allow skipping such annotation as it's pretty verbose.
few options:
For example as part of linkerd installation I wanted to remove strategy
and status
fields on Deployments. To be more conservative we should probably allow value matching as well to avoid cases when key is being removed but it has an unexpected value (in this example i expected values to be {}
).
#@ load("@ytt:overlay", "overlay")
#@overlay/match by=overlay.subset({"kind":"Deployment"}),expects="1+"
---
spec:
#@overlay/remove
strategy: null
#@overlay/remove
status: null
originally @patricknelson added as a comment to related issue: #14 (comment):
It's not always essential for the resulting YAML to have comments for me, but it certainly would be very helpful!
What if we were to use a double #
to work as an escape method when at the beginning of a line, so that the resulting YAML will contain a standard comment? For example, taken from the main demo at https://get-ytt.io:
#@ def labels():
#! This is a ytt comment
## This is a yaml comment
app: echo
org: test
#@ end
kind: Pod
apiVersion: v1
metadata:
name: test-comments
labels: #@ labels()
With the output resulting in:
kind: Pod
apiVersion: v1
metadata:
name: echo-app
labels:
# This is a yaml comment
app: echo
org: test
I'm brand new to ytt
(just doing research right now), so I'm asking just in case this isn't already available with first class syntax support without any special flags.
Seems like there's a problem with {{}}
.
We're experimenting with using ytt for templating our concourse pipelines.
YTT blows up with the following error:
Error:
- runtime error: hash of unhashable type yaml.MapSlice (backtrace: goroutine 1 [running]:
stacks.yml:9 in <toplevel>
L uri: {{buildpacks-ci-git-uri-public}}
Error:
- runtime error: hash of unhashable type yaml.MapSlice (backtrace: goroutine 1 [running]:
stacks.yml:9 in <toplevel>
L uri: {{buildpacks-ci-git-uri-public}}
reason:
runtime/debug.Stack(0x14ec820, 0x14e69c0, 0xc0000b8090)
/usr/local/go/src/runtime/debug/stack.go:24 +0x9d
github.com/k14s/ytt/pkg/template/core.ErrWrapper.func1.1(0xc0004e6ba0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/core/errs.go:18 +0x9d
panic(0x14e69c0, 0xc0000b8090)
/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/k14s/ytt/pkg/yamltemplate.MapItemOverride.Apply(0xc0004085f0, 0xc000408690, 0xc0004e6908, 0xb)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamltemplate/map_key_overrides.go:26 +0x138
github.com/k14s/ytt/pkg/yamltemplate.EvaluationCtx.PrepareNode(0x16ad880, 0xc0004085f0, 0x16ad8e0, 0xc000408690, 0x12, 0xc000096300)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/yamltemplate/evaluation_ctx.go:24 +0x71
github.com/k14s/ytt/pkg/template.(*EvaluationCtx).startNode(0xc000128000, 0x13, 0x13, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/evaluation_ctx.go:191 +0x1bc
github.com/k14s/ytt/pkg/template.(*EvaluationCtx).TplStartNode(0xc000128000, 0xc0000dcff0, 0xc000143a40, 0xc000155ff0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x8, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/evaluation_ctx.go:157 +0x179
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).tplStartNode(...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:241
github.com/k14s/ytt/pkg/template/core.ErrWrapper.func1(0xc0000dcff0, 0xc000143a40, 0xc000155ff0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/core/errs.go:25 +0xe9
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Builtin).CallInternal(0xc000143a40, 0xc0000dcff0, 0xc000155ff0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x16ab320, 0x10, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/value.go:653 +0x8b
github.com/k14s/ytt/vendor/go.starlark.net/starlark.Call(0xc0000dcff0, 0x16aaa20, 0xc000143a40, 0xc000155ff0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x16ab320, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:1028 +0x19f
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Function).CallInternal(0xc0004d3200, 0xc0000dcff0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x13e88b7, 0x152ac40, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/interp.go:305 +0x4550
github.com/k14s/ytt/vendor/go.starlark.net/starlark.Call(0xc0000dcff0, 0x16aaaa0, 0xc0004d3200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:1028 +0x19f
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Program).Init(0xc0004ba018, 0xc0000dcff0, 0xc000143950, 0x0, 0x0, 0xc0004d2e80)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:358 +0x96
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).eval(0xc0004b2060, 0xc0000dcff0, 0xc000143950, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:140 +0x203
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).Eval(0xc0004b2060, 0xc0000dcff0, 0x16a9360, 0xc0001fca20, 0xc0000dcff0, 0xc0001438f0, 0xc0001299d0, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:101 +0x719
github.com/k14s/ytt/pkg/workspace.(*TemplateLoader).EvalYAML(0xc0001fca20, 0xc0000dc3c0, 0xc0000dc370, 0x0, 0xc00015ae60, 0x1, 0x1)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/workspace/template_loader.go:172 +0x8be
github.com/k14s/ytt/pkg/workspace.(*LibraryLoader).eval(0xc0004e7b68, 0x14d6ee0, 0xc0001fc9f0, 0x0, 0x0, 0xc00020f090, 0x0, 0x0, 0x100dca8)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/workspace/library_loader.go:126 +0x6c4
github.com/k14s/ytt/pkg/workspace.(*LibraryLoader).Eval(0xc0004e7b68, 0x14d6ee0, 0xc0001fc9f0, 0x14d6ee0, 0xc0001fc9f0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/workspace/library_loader.go:82 +0x5d
github.com/k14s/ytt/pkg/cmd/template.(*TemplateOptions).RunWithFiles(0xc000150000, 0xc0000b2048, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:118 +0x20d
github.com/k14s/ytt/pkg/cmd/template.(*TemplateOptions).Run(0xc000150000, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:83 +0x255
github.com/k14s/ytt/pkg/cmd/template.NewCmd.func1(0xc000152000, 0xc00014a2c0, 0x0, 0x2, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:53 +0x2a
github.com/k14s/ytt/vendor/github.com/cppforlife/cobrautil.WrapRunEForCmd.func1.1(0xc000152000, 0xc00014a2c0, 0x0, 0x2, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/cppforlife/cobrautil/misc.go:25 +0xaf
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).execute(0xc000152000, 0xc0000b6010, 0x2, 0x2, 0xc000152000, 0xc0000b6010)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:762 +0x465
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000152000, 0xc000152000, 0x1a227e0, 0xc0000f3f58)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:852 +0x2ec
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).Execute(...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:800
main.main()
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/cmd/ytt/ytt.go:17 +0x10b
And excertpy of the YAML in question:
#@ load("@ytt:data", "data")
---
resources:
#! Github Repos
- name: buildpacks-ci
type: git
source:
uri: {{buildpacks-ci-git-uri-public}}
branch: master
An "app" is actually a micro-service in my Kubernetes application
My motivation to use ytt is to be able to build all the YAML files on the fly from templates, data and code.
I would like to run ytt only once and get all the necessary YAML files for a deployment.
At the moment, I'm almost there
But I need the separate files
So I could have more control over the deployment process
WDYT?
It would be awesome to have something like this:#@ load("@ytt:data", "data") #@ load("@ytt:template", "template") #@ load("@functions:config.star", "get_app_list") #@ load("k8s/service.lib.yaml", "k8s_service") #@ load("k8s/secret.lib.yaml", "k8s_secret") #@ load("k8s/config-map.lib.yaml", "k8s_config_map") #@ load("k8s/deployment.lib.yaml", "k8s_deployment") #@ for app in get_app_list(): #@ output-to-file ("{}.yaml".format(app.name)): #@ print ("Creating manifests for '{}'".format(app.name)) #@ print ("-- Creating K8s Service") --- #@ template.replace(k8s_service(app)) #@ print ("-- Creating K8s Secret") --- #@ template.replace(k8s_secret(app)) #@ print ("-- Creating K8s ConfigMap") --- #@ template.replace(k8s_config_map(app)) #@ print ("-- Creating K8s Deployment") --- #@ template.replace(k8s_deployment(app)) #@ end #@ endoutput-to-file ([FILE-NAME]) ... end - just an idea (edited)
via slack
It'd be nice to allow ytt expressions in the keys of a yaml dict. E.g., see the file below, the direct attempt fails miserably, to get the example to work I have to jump through hoops with functions and template.replace.
#! This doesn't work, but would be convenient
#@ for x in [1]:
? #@ "key" + str(x)
:
- #@ x+1
- #@ x+2
#@ end
---
#! This does work, but it's kinda clunky.
#@ load("@ytt:template", "template")
#@ def f(x):
- #@ x+1
- #@ x+2
#@ end
---
#@ for x in 1, 2, 3, 4:
_: #@ template.replace({ "key" + str(x) : f(x) })
#@ end
This work:
#@ def some_data():
hello: world
#@ end
simple_key: #@ some_data()
This does't work
#@ def some_data():
"hello"
#@ end
simple_key: #@ some_data()
neither this:
#@ def some_data():
- hello: world
#@ end
simple_key: #@ some_data()
The error is not really understandable either: Unmarshaling YAML template: yaml: line 2: did not find expected
In order to make ytt
accessible via homebrew
package management, it would be nice if the ytt
binary were made available in a brew repository.
/cc @drnic I don't mean to be presumptive of starkandwayne
's ci/cd process for binaries like this, but maybe this is something you all might consider adding into one of the homebrew-*
repository lineups.
based on @ctaymor's feedback i think we should find out better content that explains why someone should use ytt instead of erb/jinja/etc on the home page.
How to get the index in the for
loop?
For example:
#@ for product in data.values.products:
- #@ "product-{}-{}".format(#@i, product)
#@ end
per discussion in slack (https://kubernetes.slack.com/archives/CH8KCCKA5/p1562863952459400) @hfjn needed a way to encode yaml fragments (chunks) as yaml so that applications can read that configuration from a ConfigMap.
it would look something like this: https://gist.github.com/cppforlife/395dfa60c4e753cd04d2a26eb1d88bf2
currently yaml.encode fails as it only supports encoding simple values (non yaml fragment).
- (p) Unexpected marshaling of map (backtrace: goroutine 1 [running]:
demo.yml:9 in <toplevel>
L config1.yml: #@ yaml.encode(config1())
as a side note, should allow yaml.decode to return yaml fragments if there are multiple docs (alternatively add yaml.decode_stream so that return types are not different based on input types)?
Hi,
I get an error when running ytt
against example-load-custom-library
when I specify the files one by one:
$ ytt -f _ytt_lib/github.com/k14s/test-lib/funcs.star -f config.yml
Error:
- cannot load @github.com/k14s/test-lib:funcs.star: Could not find private library (directory '_ytt_lib' missing?)
config.yml:11 in <toplevel>
L #@ load("@github.com/k14s/test-lib:funcs.star", "testfunc")
now that yaml.v3 is out and supports preserving comments on yaml nodes, it would be nice to have ytt fmt -R -f .
command that would format yaml files in place.
It's common to extract some keys and/or values from existing yaml files.
For example, given a product config file:
#! products.yml
products:
product-1: xxx
product-2: xxx
It'd be cool to extract all products out as an array, here might be ["product-1", "product-2"]
, so that I can do something like:
#@ load("@ytt:extract", "extract")
product-info:
#@ for product in (extract("products.yml", "products")):
- name: #@ product
#@ end
It would come in very handy to be able to call a function for url encoding strings.
Use Case:
A liveness probe queries a rest endpoint which requires a parameter that contains speical characters.
Example:
#@load("@ytt:url", "url")
livenessProbe:
httpGet:
path: #@ "/query?param={}".format(url.escape("I'm URL encoded!"))
Of course exact naming doesn't matter.
Should result in
livenessProbe:
httpGet:
path: "/query?param=I%27m%20URL%20encoded%21"
I'd like to be able to use ytt to apply overlays to semi-trusted yaml files. These files should not be compiled by ytt, but be matched/transformed by other templates. If the file contains a ytt directive, it should be ignored.
Problem: it's quite cumbersome to prefix each template command by #@
. Also the YAML inside the function body cannot (?) indented, which is inconsistent with programming best practices, at odd with YAML itself and starlark conventions.
Proposal: allow function definitions to be defined without the #@
prefix, as part of the template prologue, before the YAML. The starlark syntax should be extended to allow YAML values, eg:
def some_yaml():
name: max
cities:
- SF
- LA
end
I don't have the whole proposal fleshed out yet. I just putting this here to see if there is some interest in this strawman proposal. If yes I can expand and give more details.
this would be an improvement to existing error messages to indicate why ytt isnt allowing to load a file that was not explicitly included via -f/-R flags. outcome of #7
When there is a symlink in a library evaluated by ytt, ytt errors out with an error similar to:
Expected file 'pkg2/dir2' to be a regular file, but was not
Symlinks should be handled:
Additionally, ytt should allow from command-line to load additional directories. If a symlink points to this additional directory, the symlink should be followed.
Example:
$ ytt -f main.ytt/ -o main.out
Error: Evaluating main.ytt/symlink.yaml points to ../lib.ytt but this file is not loaded.
Use --load /home/user/Projects/lib.ytt to allow access
$ ytt -f main.ytt/ -o main.out --load lib.ytt
$
If we can agree on how this should be implemented, I can implement it
Implementation notes:
../
paths that can happen in symlinks, so the Library object must reference its parent.-f
flag, main.ytt
in the example), a relative path should be computed to go from the input directory to the extra loaded directories (--load
, lib.ytt
):
// Resolve symlinks
uniqueLoadedDir, err := filepath.EvalSymlinks(loadedDir)
uniqueInputDir, err := filepath.EvalSymlinks(inputDir)
// Make the path absolute so there is no error in filepath.Rel()
absLoadedDir, err := filepath.Abs(uniqueLoadedDir)
absInputDir, err := filepath.Abs(uniqueInputDir)
// Compute relative path
InputToLoaded, err := filepath.Rel(absInputDir, absLoadedDir)
Currently, load(0 works by detecting the file type depending on the file name suffix. In case of yaml libraries, especially, it must end up with .lib.yml
which is not always desired (you might just want to have the standard yaml
suffix).
I suggest this limitation could be lifted by explicitly defining the type we want to load at load time. This could be achieved by separate functions such as loadYaml()
or loadStarlark()
.
Another alternative would be to add a parameter to load()
but this would be a backwards incompatible change.
This following example is failing with an error:
#@ def some_data():
- "hello world"
#@ end
---
simple_key: #@ some_data()[0]
with the following error:
- unhandled index operation yamlfragment[int]
config.yml:5 in <toplevel>
L simple_key: #@ some_data()[0]
but it works with starlark literals:
#@ def some_data():
#@ return ["hello world"]
#@ end
---
simple_key: #@ some_data()[0]
Same for map indexing, this does not work:
#@ def some_data():
hello: world
#@ end
---
simple_key: #@ some_data()["hello"]
with the following error:
- unhandled index operation yamlfragment[string]
config.yml:5 in <toplevel>
L simple_key: #@ some_data()["hello"]
but this works:
#@ def some_data():
#@ return { "hello": "world" }
#@ end
---
simple_key: #@ some_data()["hello"]
comparison comes up pretty often so it would be nice to document some common comparisons.
In ytt, there is the possibility to load values from files, but it does not seem that it is possible to load another YAML file from the main ytt file.
Could this be considered as an addition?
Something like this:
main.yaml
---
test: #@ include("hello.yaml")
hello.yaml
---
#@ def world()
"world"
#@ end
hello: #@ world()
the result would be:
---
test:
hello: "world"
Having a way to pass variables to the included file would be nice. What this is is basically a #@ load()
but with the function definition implicit in the included file.
using a data field that is among the list of reserved keywrods, makes ytt
fail with the following error:
- keyword pass is not supported
demo.yml:32
L annotation: #@ data.values.pass
The error message should be improved and make it clear why the reserved keywords cannot be used.
Apologies if this is something already covered with the documentation, but:
Is there a way to pass in multiple values
files into a template? I'm trying to separate out configuration for values
into multiple files rather than one big-honkin' single file, though I'm unsure if this is supported or how to properly reference multiple data files. With that said, having a single values.yml
isn't horrible and may be preferable. But what I'm trying to do is hierarchical overriding, e.g.:
.
โโโ envs
โ โโโ aws
โ โ โโโ dev
โ โ โ โโโ values.yml
โ โ โโโ sandbox
โ โ โ โโโ values.yml
โ โ โโโ values.yml
โ โโโ values.yml
โ โโโ vsphere
โ โโโ dev
โ โ โโโ values.yml
โ โโโ sandbox
โ โ โโโ values.yml
โ โโโ values.yml
โโโ template.yml
... and extract out "common" values into the appropriate hierarchical level, e.g. global values defined under envs/
, "vsphere"-related values in envs/vsphere
, etc. and the values are overridden as you get closer to the specific values file you're trying to use.
Advice and/or feedback is appreciated, thank you for your time.
The following example is causing a crash:
#@ def func1():
hello: world
---
new: document
#@ end
--- #@ func1()
The error:
- (p) expected to find node tag 1 when unwinding (backtrace: goroutine 1 [running]:
config.yml:3 in func1
L ---
config.yml:7 in <toplevel>
L --- #@ func1()
reason:
runtime/debug.Stack(0x8d80a0, 0x8a6240, 0xc000116610)
/usr/local/go/src/runtime/debug/stack.go:24 +0x9d
github.com/k14s/ytt/pkg/template/core.ErrWrapper.func1.1(0xc00012e3f0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/core/errs.go:20 +0x1c4
panic(0x8a6240, 0xc000116610)
/usr/local/go/src/runtime/panic.go:522 +0x1b5
github.com/k14s/ytt/pkg/template.(*EvaluationCtx).unwindToTag(0xc000100690, 0x1)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/evaluation_ctx.go:237 +0x10d
github.com/k14s/ytt/pkg/template.(*EvaluationCtx).startNode(0xc000100690, 0x7, 0x7, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/evaluation_ctx.go:189 +0x163
github.com/k14s/ytt/pkg/template.(*EvaluationCtx).TplStartNode(0xc000100690, 0xc0000e3050, 0xc0000e3170, 0xc000116600, 0x1, 0x1, 0x0, 0x0, 0x0, 0x8, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/evaluation_ctx.go:157 +0x179
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).tplStartNode(...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:241
github.com/k14s/ytt/pkg/template/core.ErrWrapper.func1(0xc0000e3050, 0xc0000e3170, 0xc000116600, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/core/errs.go:25 +0xe9
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Builtin).CallInternal(0xc0000e3170, 0xc0000e3050, 0xc000116600, 0x1, 0x1, 0x0, 0x0, 0x0, 0xc000116600, 0xdbfca0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/value.go:622 +0x8b
github.com/k14s/ytt/vendor/go.starlark.net/starlark.Call(0xc0000e3050, 0xa8f7c0, 0xc0000e3170, 0xc000116600, 0x1, 0x1, 0x0, 0x0, 0x0, 0xa90080, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:968 +0x154
github.com/k14s/ytt/vendor/go.starlark.net/starlark.call(0xc0000e3050, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x0, 0xc00012eb20, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/interp.go:292 +0x405a
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Function).CallInternal(0xc000100620, 0xc0000e3050, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xa90080, 0xdbfca0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/interp.go:39 +0x1c1
github.com/k14s/ytt/vendor/go.starlark.net/starlark.Call(0xc0000e3050, 0xa8f840, 0xc000100620, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xa90080, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:968 +0x154
github.com/k14s/ytt/vendor/go.starlark.net/starlark.call(0xc0000e3050, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x70, 0xc0001004d0, 0xc00012f240, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/interp.go:292 +0x405a
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Function).CallInternal(0xc0001004d0, 0xc0000e3050, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x77c4b9, 0x9245a0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/interp.go:39 +0x1c1
github.com/k14s/ytt/vendor/go.starlark.net/starlark.Call(0xc0000e3050, 0xa8f840, 0xc0001004d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:968 +0x154
github.com/k14s/ytt/vendor/go.starlark.net/starlark.(*Program).Init(0xc00000e240, 0xc0000e3050, 0xc0000e30b0, 0x0, 0x0, 0xc000066840)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/go.starlark.net/starlark/eval.go:340 +0x9a
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).eval(0xc000066780, 0xc0000e3050, 0xc0000e30b0, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:140 +0x203
github.com/k14s/ytt/pkg/template.(*CompiledTemplate).Eval(0xc000066780, 0xc0000e3050, 0xa8e100, 0xc0000e27b0, 0xc0000e3050, 0xc0000e3020, 0xc0000d3f80, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/template/compiled_template.go:101 +0x719
github.com/k14s/ytt/pkg/workspace.(*TemplateLoader).EvalYAML(0xc0000e27b0, 0xc00001e460, 0xc0000628c0, 0xc000027328, 0xc0000e2780, 0x0, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/workspace/template_loader.go:160 +0x66d
github.com/k14s/ytt/pkg/cmd/template.(*TemplateOptions).RunWithFiles(0xc00009a2c0, 0xc00000e0d0, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:109 +0xfef
github.com/k14s/ytt/pkg/cmd/template.(*TemplateOptions).Run(0xc00009a2c0, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:81 +0x292
github.com/k14s/ytt/pkg/cmd/template.NewCmd.func1(0xc000091680, 0xc0000e21e0, 0x0, 0x3, 0xb, 0xb)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/template/cmd.go:54 +0x2a
github.com/k14s/ytt/pkg/cmd.reconfigureLeafCmd.func1(0xc000091680, 0xc0000e21e0, 0x0, 0x3, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/pkg/cmd/ytt.go:90 +0x1b7
github.com/k14s/ytt/vendor/github.com/cppforlife/cobrautil.WrapRunEForCmd.func1.1(0xc000091680, 0xc0000e21e0, 0x0, 0x3, 0x0, 0x0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/cppforlife/cobrautil/misc.go:25 +0xaf
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).execute(0xc000091680, 0xc0000e21b0, 0x3, 0x3, 0xc000091680, 0xc0000e21b0)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:762 +0x465
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000091180, 0xc000091180, 0xdf47e0, 0xc000055f58)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:852 +0x2ec
github.com/k14s/ytt/vendor/github.com/spf13/cobra.(*Command).Execute(...)
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/vendor/github.com/spf13/cobra/command.go:800
main.main()
/Users/argonaut/workspace/k14s-go/src/github.com/k14s/ytt/cmd/ytt/ytt.go:17 +0x10b
)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.