masterminds / sprig Goto Github PK
View Code? Open in Web Editor NEWUseful template functions for Go templates.
Home Page: http://masterminds.github.io/sprig/
License: MIT License
Useful template functions for Go templates.
Home Page: http://masterminds.github.io/sprig/
License: MIT License
Hi,
the doc to the default function states:
default: Give a default value. Used like this: trim " "| default "empty".
Since trim produces an empty string, the default value is returned. For
things with a length (strings, slices, maps), len(0) will trigger the default.
For numbers, the value 0 will trigger the default. For booleans, false will
trigger the default. For structs, the default is never returned (there is
no clear empty condition). For everything else, nil value triggers a default.
A common scenario is to template config files, where you want to define a default value if nothing is given by the user.
default true "my value"
# "my value"
default true ""
# true
If your default option needs to be true and the user supplies false to deactivate the option. The default function will use the default instead of the user supplied value.
default true false
# true
In my opinion this behavior is a pitfall and needs to be changed, that boolean true/false are passed as they are if the default is also boolean or needs to be more clear in the documentation.
You can circumvent the problem with the following template:
{{/*
defaulttrue: Give true as default value. Used like this: trim " "| defaulttrue.
Since trim produces an empty string, the default value true is returned. For
things with a length (strings, slices, maps), len(0) will trigger the default value true.
For numbers, the value 0 will trigger the default value true. For booleans, the value is
passed as is. For structs, the default is never returned (there is no clear
empty condition). For everything else, nil value triggers the default value true.
*/}}
{{- define "defaulttrue" -}}
{{- if typeIs "bool" . -}}
{{- . -}}
{{- else -}}
{{- default true . -}}
{{- end -}}
{{- end -}}
Given:
{{- define "foobar" }}
foo:
bar: baz
{{- end }}
qux:
{{- template "foobar" . | indent 2 }}
I'd like to get:
qux:
foo:
bar: baz
Given the above, I get:
executing "test" at <2>: wrong type for value; expected string; got map[string]interface {}
Using {{- template "foobar" . | toString | indent 2 }}
I get:
qux:
foo:
bar: baz
Meaning the line(s) never get indented. Even doing {{ template "foobar" . | join "_" | indent 2 }}
results in the above, so clearly piping after template
isn't working as I hoped it would.
Are there any ways to solve this?
I have a use for needing to easily serialize a map[string]string
given a sep
and assign
. I need it for a helm chart where I want to serialize a yaml map into an env var of the form key:value,key2:value
which will then be auto deserialized into a map with https://github.com/kelseyhightower/envconfig.
Do you think others might find this useful? I want to add it here because helm uses this lib for utility functions in templates.
If you're willing to add such a function here I'd be happy to create a PR.
I would make the implementation similar to the join
(join: strings.Join, but as join SEP SLICE
) function currently supported.
something like joinMap SEP ASSIGN MAP
and then joinMap "," ":" map[string][string]{"a":"b", "c", "d"}
results in a:b,c:d
Hello!
While reviewing recent charts PRs (example: helm/charts#2004), I see this interesting pattern:
{{- if .Values.persistence.storageClass }}
{{- if (eq "-" .Values.persistence.storageClass) }}
storageClassName: ""
{{- else }}
storageClassName: "{{ .Values.persistence.storageClass }}"
{{- end }}
{{- end }}
The explanation is:
## mariadb data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: "-"
My question was, since the template-rendered YAML will either be a storage class name, an empty string (to disable dynamic provisioning), or not set at all, can't we just do this instead?
## mariadb data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to empty string "", this disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: ""
Then the template logic could be simplified to:
{{- if .Values.persistence.storageClass }}
storageClassName: "{{ .Values.persistence.storageClass }}"
{{- end }}
But this would only work if an empty string is different from not set (nil
) when checking if .Values.some.value
in go template. Can someone help clarify this for me, or point to the relevant part of the docs? I seem to not be able to find this info.
Thanks!
It would be a nice utility to be able to pluralize a string. For example, {{ pluralize 1 "anchovy" "anchovies" }}
would return anchovy
, but ``{{ pluralize 2 "anchovy" "anchovies" }}would return
anchovies`
Hello,
The title contains a very specific description but the tooling could be generic.
We would like a file mode filter that can convert from unix octal permissions 0755 to decimal. The reason being that json does not support octal and it makes writing and maintaining charts quite cumbersome.
See helm/helm#4378 (comment) for use case.
Thanks,
Hello, I spent some time understanding how to extract the element at index n
of an Array
Es:
arr := [1,2,3]
I want to get arr[1] => 2
I finally found the index
function, but is not documented in the docs and I did't find it's declaration in the code.
The usage is
{{ index arr 2 }}
and I found references in:
Where does it come from? Should be documented in the list.md
doc file?
I'm sorry if it's something obvious, I'm new to go and sprig. I can provide a PR to the documentation if you point me in the right direction.
Thank you!
Just a suggestion: deprecate biggest
and add max
and min
funcs.
Most of the regex functions are unfortunately significantly broken when it comes to using them in pipelines. By broken I mean the functions technically work, but their usage I don't think would be intuitive to anyone, and that usage is difficult.
For example to use the regexReplaceAll
function, you cannot do this:
"foo subject string" | regexReplaceAll "foo" "bar"
You instead have to do this:
"bar" | regexReplaceAll "foo" "foo subject string"
This means that if you are operating in a pipeline, you have to put your pipeline as the 2nd argument to the function, or store it in a variable first, which can be really messy and hard to maintain.
Golang templates pass the result of the previous item in the pipeline as the last argument to the function being called. Thus the arguments of all the regex functions need to be swapped around.
Unfortunately changing this is a breaking change, but I think this is a fairly major usage issue. And since these functions aren't even documented (ref #84), the impact might be minor.
I have a case where I want a template to take an input dictionary and produce a copy, omitting a few keys at various levels deep in the hierarchy (in this helm chart).
I ran into a snare, where I didn't realize that after merging two dictionaries, editing the merged dictionary can modify the originals:
{{- $values := pick .Values "auth" "other"}}
{{- /* trim secret values. Update here if new secrets are added! */ -}}
{{- /* make a copy of values.auth to avoid modifying the original */ -}}
{{- $_ := set $values "auth" (merge dict .Values.auth) }}
{{- $_ := set $values.auth "state" (omit $values.auth.state "cryptoKey") }}
Without that set $values "auth" (merge dict .Values.auth)
,
the subsequent set $values.auth "state"
would modify the original .Values.auth.state
, which I found surprising.
Is there an existing mechanism to produce a deep copy of a dict, to ensure the original doesn't get modified?
The changelog (see version at time of issue) is from the 1.x version. We are now well into the 2.x version. It should be updated (from the release notes on GitHub?) or removed.
kebabcase "helloWorld"
would both produce hello-world
, similar to how camelcase
and snakecase
work.
Need a way to conditionally check a value against a regex.
{{- if match-regex $my_value '^[a-z]...' }}
{{- end }}
or something like that.
../../Masterminds/sprig/crypto.go:35:37: multiple-value uuid.NewV4() in single-value context
I don't think this was purposeful, but let me know why if it was a decision made.
Given the following go program:
package main
import (
"fmt"
"os"
"text/template"
"github.com/Masterminds/sprig"
)
func main() {
tpl := `{{- $i := genSelfSignedCert "example.com" nil nil 365 }}{{ $i.Cert }}`
t := template.Must(template.New("test").Funcs(sprig.TxtFuncMap()).Parse(tpl))
if err := t.Execute(os.Stdout, nil); err != nil {
fmt.Printf("Error during template execution: %s", err)
return
}
}
a certificate is printed to stdout. This certificate is a valid certificate readable by openssl, etc. However, given a verify command, openssl cannot verify this cert using itself:
# openssl verify -CAfile out.crt out.crt
test.crt: CN = example.com
error 20 at 0 depth lookup:unable to get local issuer certificate
It is a common use case of a self signed cert to use for a cert that is verified by itself to encrypt two systems, so I assume this should work for this use case.
The corresponding openssl commands being
# openssl req -x509 -nodes -batch -days 86400 -newkey rsa:2048 -subj "/CN=example.com" -out "./tls.crt" -keyout "./tls.key"
Generating a 2048 bit RSA private key
...+++
.............................+++
writing new private key to './tls.key'
-----
# openssl verify -CAfile tls.crt tls.crt
tls.crt: OK
I would note that making a sprig CA using genCA
and using it in genSignedCert
will allow you to verify the cert with the CA correctly - as expected.
EDIT:
adding x509.KeyUsageCertSign
to template.KeyUsage
in crypto.go::generateSelfSignedCertificate()
here does fix this use case - not sure about all the implications.
eg. parse a JSON string and return a complex datatype (eg. for looping)
Hey folks, do you think it makes sense to have a golang playground with sprig functions already loaded ? I think it will be nice to get a better feel of the functions that one can use, and how they behave.
A function that can generate the htpasswd
type hash from username & password would be great. In Kubernetes Helm charts, this would help in configuring basic auth for ingress resources.
sprig is currently locked to version 0.2.2 of mergo in glide.lock. Version 0.3.0 added this fix for merging slices correctly (deeply). Any chance of updating to 0.3.0? Here is a comparison between those versions for reference.
Hi,
Why do you type juggle with template.FuncMap and html.FuncMap in the public functions instead of using only map[string]interface{} ?
Also, in this, i suspect it is not working on a copy, compare with this playground
btw, i noticed a typo at => tpl := template.New("foo").Funcs(sprig.FuncMap())
In #10 @technosophos said:
I have been thinking of adding some crypto functions to sprig for use with public key cryptography use cases.
I'd actually love if sprig supported generation of 1024 bit dhparams. This would be tremendously useful to anyone, for instance, using github.com/helm/helm to manage Nginx or other web servers.
fwiw, generating 2048 bit dhparams would be lunacy because it takes soooo long. But what would we think about 1024 bit dhparams?
This might be crazy.
I'm trying to split a string into parts and take the last element.
{{regexSplit "/src/" "foo/src/bar" -1 | last}}
Expecting "bar" but getting
executing "" at <last>: wrong type for value; expected []interface {}; got []string
As a work around I have done this:
{{with $my:=regexSplit "/src/" "foo/src/bar" -1}}
{{index $my 1}} # Can't do arithmetic in templates :-(
{{end}}
Which feels unreasonably cumbersome and still not quite what I wanted.
{{ $envDict := pick .Values.global .Values.global.env }}
{{
{{
{{- define "app.release_labels" }}
app: {{ printf "%s-%s" .Release.Name .Chart.Name | trunc 63 }}
version: {{ .Chart.Version }}
release: {{ .Release.Name }}
{{- end }}
{{- define "app.full_name" -}}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 -}}
{{- end -}}
{{- define "app.repository" -}}
{{- printf "%s" .imgDict.repository | trunc 63 -}}
{{- end -}}
{{- define "app.tag" -}}
{{- printf "%s" .imgDict.tag -}}
{{- end -}}
{{- define "app.replica_count" -}}
{{- printf "%s" $.chartDict.replicaCount-}}
{{- end -}}
I am getting an error: executing "app.replica_count" at <$.charDict.replicaCo...>: can't evaluate field replicaCount in type interface {}
I am using this as a helper file in my helm chart repo. The input file to this looks like this:
global:
env: "dev"
dev:
hello-world:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
hello-kube:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
stage:
hello-world:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
hello-kube:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
Can some one please help me with what I am doing wrong?..
Many thanks in advance.
Recently I needed to work with string slices in templates and apply various functions to each element (title, lower, trim, etc). I wanted to avoid using loops and assignments, so I came up with eachTitle, eachLower, eachTrim, etc functions.
It's quite simple really:
// each applies a function to each string in a slice.
func each(s []string, fn func(string) string) []string {
for key, value := range s {
s[key] = fn(value)
}
return s
}
// eachFunc makes a simple string transformation function to an each function.
func eachFunc(fn func(string) string) func([]string) []string {
return func(s []string) []string {
return each(s, fn)
}
}
Usage:
funcMap["eachTitle"] = eachFunc(strings.Title)
Do you think it would fit into this package? I could extend it into a solution that would work with functions that have more than one parameter.
It would be nice to have strings.TrimSuffix
:
{
"trimSuffix": strings.TrimSuffix
}
The function Add, for example, takes two arguments. Why not take an arbitrary number? The language is prefix so it's easy to have multiple arguments:
{{Add 2 3 4}}
func Add(nums []int) int
You could make them even more general by accepting any numeric type and converting as necessary under the covers, but that's a lot more work.
We are using indent
function in helm templates to generate yaml
files. In lots of places we need to indent stuff, but that breaks template indentations (see below).
resources:
{{ .Values.resources | indent 4 }}
Jinja indent
filter allows skipping the first line. I think that would be useful so that we can write templates as the following snippet which allows keeping the natural indentation in template files:
resources:
{{ .Values.resources | indent 4 true }}
Wrap and WrapWith both lose the character at the wrap point
Test example :
import (
"github.com/Masterminds/sprig"
"html/template"
"bytes"
)
func main() {
// The template
tpl := "Word Wrap Test\n{{.Name | wrap 11 }}\n"
// The values to pass to the template
vars := map[string]string{"Name": "123456789_123456789_123456789_123456789_1234567890"}
// Compile the template
t := template.Must(template.New("example").Funcs(sprig.FuncMap()).Parse(tpl))
// Run the template, copying the output to the buffer.
var b bytes.Buffer
err := t.Execute(&b, vars)
if err != nil {
panic(err)
}
print(b.String())
}
Output showing issue - in the first split line 1
is lost and 2
starts the second line, and so on :
Word Wrap Test
123456789_
23456789_1
3456789_12
456789_123
567890
The list function has
is implemented as has needle haystack
,
see https://github.com/Masterminds/sprig/blob/master/list.go#L241
The documentation at https://github.com/Masterminds/sprig/blob/master/docs/lists.md
on the other hand claims the reverse: has haystack needle
.
Trying to follow the documentation, like has .Values.foo "x"
will cause the error Cannot find has on type string
.
Thanks for writing this! Great library
Just a quick note that took me a few minutes to track down... documentation for has is wrong.
Should be: has 4 $myList
instead of
has $myList 4
Hi,
I am trying to use derivePassword in loop but unfortunately counter is defined uint32 so it is hard (I didn't find any way to do it from HELM) to use it.
Thanks
N.B. The algorithm of this function moved to here
When passed an array, only pass unique values.
http://stackoverflow.com/a/26875358
Something like:
{{range $index, $thing := unique $}}
Hi Everyone,
I am sorry for writing this here. But I cannot find the answer and really wondering reason why the sprig uses the $_ when using the set function. in the documentation it is like {{ $_ := set .... }}
Is $_ a special variable or something in sprig, please?
Thank you very much and looking forward to learn more on this.
Thanks for building this lib! I really like it, but I'd like it even more if it haven't any dependency (outside the standard library)
Some of the current dependencies are to allow functions I don't care at all on templates, like UUID generation
For others, the following Go proverb applies:
"A little copying is better than a than a little dependency"
Rob Pike
For example, the xstrings
package is imported just because of two functions
Have a github.com/Masterminds/sprig/slim
package that have all the functions that doesn't require importing external libraries
The documentation doesn't mention many of the functions which are actually available:
regex*
b32*
fail
genCA
/genSelfSignedCert
/genSignedCert
merge
toJson
/toPrettyJson
Hello,
my go version is: go version go1.8.3 linux/amd64
When I want to use your library I get next message:
# github.com/Masterminds/sprig
../../Masterminds/sprig/crypto.go:35: multiple-value uuid.NewV4() in single-value context
Checking crypto.go file I have seen this:
// uuidv4 provides a safe and secure UUID v4 implementation
func uuidv4() string {
return fmt.Sprintf("%s", uuid.NewV4())
}
But uuid.NewV4()' return two values, the
UUIDand an
error`
I've send you one pull request #74.
Regards
It would be nice to add localization functions. Not sure how it would all be composed, but something like:
t "FR" "cheese"
-> "fromage"
would be nice.
It would be particularly nice to be able to set a "global" language and translation strings facility and then be able to just call t "cheese"
. This is how Drupal does it.
The backend would need something like:
type Translator interface {
T(string) string
}
An easy map-based backend could satisfy that interface.
For versioning, would it be possible to tag the releases with v#.#.#?
Doing a go get
will pull a potentially unstable master branch, where using gopkg.in will retrieve the latest tagged release branch.
$ go get -d gopkg.in/Masterminds/sprig.v2
** this will fail
POC: I forked sprig and did the following which now works correctly
**fork sprig via github
$ git clone [email protected]:TerraTech/sprig
$ cd sprig
$ git tag v2.12.0 2.12.0
$ git push --tags
$ go get -d gopkg.in/TerraTech/sprig.v2
** correctly pulls the v2.12.0 tag
Currently there is no way to add any key to already instantiated variable. It makes building large dictionaries difficult, as all key needs to be specified within a single {{ with }} command.
It would be great to be able to build dict variable by adding 1 key at the time.
example:
{{ $env := dict "blah-1" $blah-1 }}
{{ $env := dict "blah-2" $blah-2 }}
resulting $env having two pairs of keys.
There are cases when errors are not handled but it could be useful for debugging compilation errors. See 1, 2, 3
In some cases it is pretty much all right to return default-ish values silently but in some cases it would be useful to fail and print the error message.
Proposal:
randAlphaNum and randAscii functions can be very helpful to generate random strings. You can choose the length of the string and the kind of characters to use but you can't ensure the string will have all the kind of characters.
There are certain scenarios when you need to ensure that the generated string has at least one character of each kind. E.g. "I want a password that has at least one number and one letter".
I can assume that using a long password both will be generated but they might be not. E.g. "randAlphaNum 10" could generate "AUIjLlmiOk" with no numbers.
What do I expect?
It seems that all these functions defined at springs.go are based on randomstringutils.go which do not accept any parameter to modify this behaviour.
It's not clear what happens with div
if the result is not an int. Need to be able to explicitly control if it gets rounded up or down.
Example use case: calculate the min number of nodes required to form a quorum, i.e. {{ replicas | div 2 | ceil}}
. I'm not sure this will work even if we had ceil/floor functions because I'm guessing div
is already performing an implicit ceil or floor.
I suppose if div
is already doing an implicit floor then we can do {{ replicas | div 2 | add1}}
.
Thoughts?
So I need a template function that will wrap double quotes around a provided string. So something like this: {{ stringify "this string" }}
. Right now go templates will not enclose the string with quotes so I have to do something like this: `"{{"this string"}}". Which looks weird when Im using go templates to output configuration files.
A very useful addition to the Path function class would be functions dealing with URLs, like extracting domain, path or a query argument from a URL.
This is suggested here:
It'd stand beside the tuple function, which is the equivalent for arrays.
This is really useful for writing templates that are encapsulated and don't depend on the full dot value.
I wanted to iterate over a value defined size list of integers, and skip integers matching a given value.
Ideally this would be doable via..
{{range $k,$v := without (until $max) $skipMe}}{{$.Values.service.name}}-{{$v}}{{end}}
Except until $max
returns int[]
and without
expects interface[]
I tried a few other ways to build the list, and found that regexFindAll
splitList
and others also return primitive arrays that are incompatible with interface[]
The only way I could see to create a list of interface[]
was via the list
function, but that doesn't allow me to build my list from 0 to $max because I need to specify all the list members up front. (Couldn't build the list using range either, because that declared a new list in the range scope each time).
In the end, I found a workaround by going to/from json, with a bit of fudging, I ended up with ..
{{ $rawjson := ( until $max | toJson )}}{{ $jsonmap := cat "{\"values\":" $rawjson "}" }}{{ $newmap := replace "[" "[\"" $jsonmap }} {{ $newmap := replace "]" "\"]" $newmap }} {{ $newmap := replace "," "\",\"" $newmap}}{{ $currentasstr := cat $v }}{{ $parsedmap := fromJson $newmap }}{{ $filtered := (without $parsedmap.values $currentasstr) }}{{ range $o, $p := $filtered }}{{ if ne $o 0 }},{{end}}{{$.Values.service.name}}-{{$p}}{{end}}
Which is pretty nasty ! Ideally the methods returning arrays of things not compatible with the interface[]
type expected by without
etc would start returning compatible types.
https://github.com/Masterminds/sprig/blob/master/crypto.go#L35
need change from:
return fmt.Sprintf("%s", uuid.NewV4())
=>
return fmt.Sprintf("%s", uuid.Must(uuid.NewV4()))
In many projects you still need sha1sum. In my example I want to write a Helm chart for Jenkins with automatic scriptApproval for my jobdsl scripts.
I'd like to have requiredEnv which would work similar to this https://github.com/roboll/helmfile
The required_env function allows you to declare a particular environment variable as required for template rendering. If the environment variable is unset or empty, the template rendering will fail with an error message.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.