The following code succeeds in yaml.v1 but fails in yaml.v2:

package main
import (
    "gopkg.in/yaml.v1"
    "fmt"
    "log"
)

var content = `
top:
  x:
    foo: bar
`

type T struct {
    X interface{}
}

func main() {
    var t map[string] T
    err := yaml.Unmarshal([]byte(content), &t)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%#v\n", t)
}

would be awesome, it's not really clear what the flow option does etc, lots of trial and error to get it to unmarshal correctly for structs

Currently there is no way of controlling where a 'flow style' line will be split with a newline. Add something similar to pyYAML's width parameter:
http://pyyaml.org/wiki/PyYAMLDocumentation

Empty arrays (e.g. foo: []) are encoded to nil. That makes it hard do distinguish this case from foo being not present in the YAML or being null.

Any reasons not to change this (i.e. encoding it to an non-nil slice of zero length)?

I've mostly figured it out, except for the "tag" return of GetYAML - what am I supposed to put there?

e.g.

---
  basePath: "https://github.com"

That can not be unmarshaled into:

    type Data struct {
        BasePath    string
    }

But this can:

---
  basepath: "https://github.com"

This can be solved by support struct-tags in the same way json and XML un/marshaling works.

Defining more that 1 omitempty field tag, raises an error: Duplicated key 'omitempty' in struct core.Service.

type Foo struct {
Bar string yaml:"a, omitempty"
Foo string yaml:"b, omitempty"
...

There is a technical reason for having this constraint?

if a structure has the yam:key,omitempty tag, why not omit it as well if it's members are all empty:

To do so, simply add the following snippet to isZero in yaml.go - where we check for a structure and then recursively call isZero on all of it's members...

case reflect.Struct:
for i := 0; i < v.NumField(); i++ {
p := v.Field(i)
if !isZero(p) {
return false
}
}
return true

Merge key support "<<" allows to merge parts of a map with another map. The current implementation parses "<<" as a key instead of merging the contents.

http://yaml.org/type/merge.html

^^

Any options about not change struct field name by default?
I create a yaml library that fork form this one myself with a struct field name not change by default.

I think name style change stuff make everything complex.
I just want a kiss yaml library.

https://github.com/bronze1man/kmg/tree/master/encoding/kmgYaml
http://godoc.org/github.com/bronze1man/kmg/encoding/kmgYaml

It seems as if people are working with branch v1 when forking. Maybe we should bring master branch in sync with v1 again and then add some comment in the README so master will become the active dev branch.

It would be nice to have a Decoder.Decode like encoding/json for processing streams. I basically want to do:

f, _ := os.Open(filename)
yaml.NewDecoder(f).Decode(&obj)

Instead of:

b, _ := ioutil.ReadFile(filename)
yaml.Unmarshal(b, &obj)

This makes it nicer for decoding yaml documents from io.Readers without having to read everything into a []byte first.

The proper way to support this would be with something like:

type MapSlice []MapItem

type MapItem struct {
        Key, Value interface{}
}

and then implement first-class marshalling and unmarshalling of maps onto it.

Feature requested via the pull request #27.

After commit 72c33f6, this code fails.
Even if the yaml package was changed to quote "<" on output, we'd
still have a problem, as this character has not been quoted historically,
so it can fail to parse old files (or output produced by other yaml producers,
such as python's yaml module).

package main

import (
    "log"
    "fmt"
    "gopkg.in/yaml.v1"
)

func main() {
    d := struct {
        Net string
    }{
        `<foo>=bar`,
    }
    data, err := yaml.Marshal(d)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%s", data)
    var x interface{}
    err = yaml.Unmarshal(data, &x)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("%#v", x)
}

var x interface{}
err := yaml.Unmarshal([]byte("null"), &x)

This fails with the error "YAML error: reflect: reflect.Value.Set using unaddressable value".

I believe it should work. The heart of the problem is in the
implementation of the decoder.setter method, which does
not dereference the out pointer if the tag is !!null (as it is
in this case).

The actual error occurs in the decoder.scalar method, inside the
reflect.Ptr arm of the type switch. out.Set(reflect.Zero(out.Type())
fails because out is not settable, because it has not
been dereferenced.

I've tried a few possible fixes, but haven't got there yet - decoder.setter
is sensitive code! Filing this bug as a place holder.

I have a struct with a field "Path" with a yaml: tag name "path". For historical reasons, I want to accept YAML that sends either "path" or "mountPath" as the key. Is it possible to do this without defining two fields in the struct?

in applications the order of things in yaml and json tend to matter, but it appears we're using a map[interface{}]interface{} as the underlying datastructure for what are conceptually ordered maps.

Even after implementing an OrderedMap I can't populate its keys in the order specified in the yaml because it is returned with go's officially randomly sorted maps. Most other parsers in other languages i've tried return an ordered mapping for something like this:

- s3sync:
    key_id: $AWS_ACCESS_KEY_ID
    key_secret: $AWS_SECRET_ACCESS_KEY
    bucket_url: $AWS_BUCKET_URL
    source_dir: build/

becomes:

{"sync": {"key_id": ...., "key_secret": ...}}

ingest this yaml: https://github.com/discourse/discourse_docker/blob/master/samples/standalone.yml
marshal it into a simple map[interface{}]interface{} and then unmarshal it back into a string and it becomes something like this: http://pastebin.ubuntu.com/8329740/ (this example is slightly modified but the important part is under expose, which is "unmodified".

Note that the entries under expose: are unquoted strings ... somehow this confuses yaml parsers due to the colons and they blow up. The use case was starting a docker container, but trying to ingest the output with a python yaml parser also fails. I'll update this issue with more info once I get a smaller example written.

package main

import (
    "fmt"
    "gopkg.in/v1/yaml"
)

const yamlStringDouble = `

---
time: "2014-08-04 12:13:14"
`
const yamlStringNone = `

---
time: 2014-08-04 12:13:14
`

type Time string

type A struct {
    Time *Time
}

func (t *Time) SetYAML(tag string, val interface{}) bool {
    fmt.Printf("Tag: %s, %T = %v\n", tag, val, val)
    return true
}

func main() {
    var a A
    fmt.Println("Quoted:")
    err := yaml.Unmarshal([]byte(yamlStringDouble), &a)
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println("Unquoted:")
    err = yaml.Unmarshal([]byte(yamlStringNone), &a)
    if err != nil {
        fmt.Println(err)
    }
}

If you change the receiver type to not be a pointer, SetYAML doesn't get called in either case. I think that both of these are bugs since UnmarshalJSON handles this correctly.

This may be a "we didn't intend for it to work that way" kind of issue, so pardon me close this if that's the case.

If I'm marshalling into a struct, say:

type config struct {
    []string values
}

And if do the following:

c := &config{
    values: []string{"one"}
}
yaml.Unmarshal([]byte(`config:
    values: ["two"]
`), config)

Then I would expect the final struct to have values just be ["two"], overwriting the ["one"] entry. However it gets appended, and my final struct has config.values = ["one", "two"]. Is this expected or something that needs to be fixed?

The yaml package allows an anchor to be dereferenced before it's defined, eg:

foo:
  <<: *bar

bar: &bar

while it could be argued it's convenient that this works, it's not in line with the yaml spec and other yaml parsers.

Hi! How can I unmarshal an array of struct elements? (I've tried with ,inline, without , inline)

var data = `
current: batman
options:
  batman: something
  robin: another
`
type Target struct {
    Label    string
    Endpoint string
}
type Targets struct {
    Current string
    Options []Target ",inline"
}

func LoadTargets() (*Targets, error) {
    m := Targets{}
    err := yaml.Unmarshal([]byte(data), &m)
    if err != nil {
        log.Fatalf("error: %v", err)
    }
    fmt.Printf("--- m:\n%v\n\n", m)
    return &m, nil
}

thanks!

I'm trying to unmarshall a list of 10 integers into *[10]int{} and it doesn't work. Array is not filled.

$ gofmt -l .
decode.go
encode_test.go

Support the standard marshaling/unmarshaling interfaces from the encoding package:

• BinaryMarshaler
• BinaryUnmarshaler

I'd love to incrementally upgrade the use of JSON to YAML for config files in my projects, but to do that I'd have to add "yaml" tags to all of my structs so that they match the "json" tags.

It would be great to have an unmarshalling mode that uses the json packages defaults (Upper case keys) and reads the "json" tags from structs, so I can use go-yaml as a drop in replacement until the config files can be rewritten.

yaml.Marshal("3\xc0\x8e\xf0q/\x04b_\xe8_L\v\x97WN\xdc\xc0") never returns.

It would be great to have sth. similar to http://golang.org/pkg/encoding/json/#RawMessage.

I'm using go-yaml in http://github.com/michaelsauter/crane and would like to unmarshal the JSON/YAML partially (to provide defaults for objects inside arrays).

I'm working out how to parse and encoding time.Time in YAML.

I have a prototype for marshal but I'm stuck on how to unmarshal a time.Time.

Can anyone suggest a starting point?

I'm happy to submit a PR if it's interesting.
https://github.com/ewalk153/yaml/tree/add_time_marshal

The is_break function (and probably others) can cause
a panic when marshalling strings containing invalid utf-8
(specifically a truncated line-break sequence in this case).

For example:

yaml.Marshal([]string{"hello\nfoo\xe2"})

panics.

Currently, any uint64 which overflows int64 gets unmarshalled as a float64 instead.

At best, this loses precision (since the float64 only has 52 bits to store the fraction in).

At worst, a bug in the boundary condition prevents the unmarshalling from happening at all:
if resolved < 1<<64-1

This should be if resolved <= 1<<64-1...

Test case:

package main

import (
    "fmt"
    "math"
    "strconv"

    "gopkg.in/yaml.v2"
)

func main() {
    fmt.Println("\nTesting loss of precision\n=======================================")
    test(12345678901234567890)

    fmt.Println("\nTesting uint64 max\n=======================================")
    test(math.MaxUint64)
}

type Test struct {
    X uint64
}

func test(x uint64) {
    m := Test{x}

    fmt.Printf("Original (base-2):      %s\n", strconv.FormatUint(m.X, 2))
    fmt.Printf("Original (base-10):     %s\n\n", strconv.FormatUint(m.X, 10))

    y, _ := yaml.Marshal(m)
    fmt.Printf("Marshalled YAML:     %s\n", string(y[:]))

    var m_yaml Test
    yaml.Unmarshal(y, &m_yaml)
    fmt.Printf("Unmarshalled (base-2):  %s\n", strconv.FormatUint(m_yaml.X, 2))
    fmt.Printf("Unmarshalled (base-10): %s\n\n", strconv.FormatUint(m_yaml.X, 10))
}

I'm trying to validate my data in UnmarshalYAML

package main

import (
    "errors"
    "fmt"
    "gopkg.in/yaml.v2"
)

type Format string

const (
    JPG Format = "jpg"
    GIF Format = "gif"
)

type Config struct {
    Formats []Format
}

func main() {
    s := `formats:
    - jpg
    - gif`
    var c Config
    err := yaml.Unmarshal([]byte(s), &c)
    fmt.Printf("%v, %v\n", err, c)
}

func (f *Format) UnmarshalYAML(unmarshal func(interface{}) error) error {
    var fm string
    if err := unmarshal(&fm); err != nil {
        return err
    }
    format := Format(fm)
    switch format {
    case JPG, GIF:
        f = &format
    default:
        return errors.New(fmt.Sprintf("Invalid file format: %s", fm))
    }
    return nil
}

The output for this is : <nil>, {[ ]} instead of <nil>, {[jpeg gif]}. Does UnmarshalYAML work only with structures?

I've been using YAML for about a decade, using parsers written in Perl, Ruby, and (most recently) Julia. In all of these, the parser reads the YAML text and creates a corresponding data structure. It appears, however, that the Go parser needs to have the structure laid out in advance before it can parse the file. Say what?

YAML is really good at encoding semi-structured data (eg, ASTs). How am I supposed to know the exact structure of this sort of file, at compile time? Am I missing something?

-r

Is there any reason against inlining member being pointer to a struct, complementary to a struct value?

No issue with extending the switch here?

Support the standard marshaling/unmarshaling interfaces from the encoding package:

• TextMarshaler
• TextUnmarshaler

Looks like yaml.Unmarshal() and json.Unmarshal() handle composite types differently. The yaml unmarshalling does not appear to read the tags on the inner type. Note ValueA is unmarshalled correctly in the json case, but not in the yaml case.

package main

import (
    "encoding/json"
    "fmt"

    "github.com/go-yaml/yaml"
)

type A struct {
    ValueA int `json:"a" yaml:"a"`
}
type B struct {
    A
    ValueB int `json:"b" yaml:"b"`
}

const jsonData = `{ "a" : 1, "b" : 2 }`
const yamlData = `
a : 1
b : 2
`

func main() {
    jdata := B{}
    ydata := B{}

    json.Unmarshal([]byte(jsonData), &jdata)
    yaml.Unmarshal([]byte(yamlData), &ydata)

    // $ go run main.go
    // json: {{1} 2}
    // yaml: {{0} 2}
    fmt.Printf("json: %v\n", jdata)
    fmt.Printf("yaml: %v\n", ydata)
}

Say I have:

struct A {
  X string
}
struct B {
  A
  Y string
}
var b B
yaml.Unmarshal([]byte("x: abc\ny: def"), &b)

This will unmarshal "b.Y" but leave "b.A.X" uninitialized. This behavior differs from the encoding/xml and encoding/json packages, which will iterate over the fields of all embedded types as if they were at the same level.

Further, I am unable to workaround this with a custom UnmarshalYAML method. I can unmarshal to the embedded type:

func (b *B) UnmarshalYAML(unmarshal func(v interface{}) error) error {
   if err := unmarshal(&b.A); err != nil {
     return err
   }
   return nil
}

this will ignore the fields of B, and if I unmarshal to both:

func (b *B) UnmarshalYAML(unmarshal func(v interface{}) error) error {
   if err := unmarshal(&b.A); err != nil {
     return err
   }
   return unmarshal(b)
}

I get infinite recursion.

We have some yaml data which we parse and want to validate. In this context, validate generally means:

• check required fields are present
• check formatting of strings (regex)
• set default values (if y is not specified, default to value 123 or "same as another field")

I find the Setter interface, but I just don't grok what to do with it (still sort of new to Go). What is the value when I am implementing SetYAML() on a multi-field structure? Is there an example of this? Or do people just open code the validation logic for complex types?

I see two ways (there are probably others) of doing this cleanly:

func UnmarshalMultiple([]byte, interface{}) error
func UnmarshalFirst([]byte, interface{}) (int, error)

The first expects a pointer to a Slice and does the same as Unmarshal, but appends each document to the passed in slice. The second does the same as Unmarshal but returns the index into the []byte of the beginning of the next document or -1 if it's reached the end.

My current use-case is implementing something like TAP-Y, which is defined as being a stream of documents. Parsing this into a slice of map[string]interface{} would make things nicer to work with.

When I try to Marshal multiline string containing '/' or non-english symbols, for examle, I get this:

- text: Python script start
  note: "#!/usr/bin/env python3\n# -*- coding: utf-8 -*- \n"

and original yaml looked likes this

- text: Python script start
  note: |
    #!/usr/bin/env python3
    # -*- coding: utf-8 -*-

Is this intentional behavior? And if so, can I do something with it? Because its really hard to read multiline strings in form like "str\nstr2\n".

I have the following yaml configuration:

default: &default
  user: "adm"
  file_perms: "664"

folders:
  - path: "/var"
    user: "www-data"
    dir_perms: "775"

  - path: "/var/site/"
    group: "www-editors"

  - <<: *default
    path: "/lib/folder2"

And I'm expected to be able to merge the key values for 'default' into the third folders list item.

My struct that I'm providing to the Unmarshal method looks like:

type config struct {
    Folders []*folder
}

type folder struct {
    Path      string
    User      string
    Uid       int
    Group     string
    Gid       int
    DirPerms  string `yaml:"dir_perms"`
    DirMode   os.FileMode
    FilePerms string `yaml:"file_perms"`
    FileMode  os.FileMode
    Children  folders
}

Note there's no 'default' key provided as I don't care about loading the default value, but I still would expect its keys to be merged into the folders list.

Am I doing something wrong, or does go-yaml not handle merges?

The following program produces a panic of the form:

panic: runtime error: hash of unhashable type []interface {} [recovered]
panic: runtime error: hash of unhashable type []interface {}

package main
import (
    "fmt"
    "gopkg.in/yaml.v1"
)
func main() {
    var x interface{}
    msg := `{".", ["+"]}`
    yaml.Unmarshal([]byte(msg), &x)
    fmt.Println(x)
}

I'm porting this bug over as I see the repository has changed: https://bugs.launchpad.net/goyaml/+bug/1279042

There is no option to use yaml_emitter_set_unicode, although it is defined in encode.go. That means when I have an input file like:

line: 你好

And I unmarshal and re-marshal it, when I write it to a file I will get:

line: "\u4F60\u597D"

which is unexpected -- I prefer the unescaped version.

When I add this line as L20 in encoder.go:

yaml_emitter_set_unicode(&e.emitter, true)

The problem is resolved, and line: 你好 displays normally on output. I think there should be an option to yaml_emitter_set_unicode(..., true), otherwise I have to keep a forked repo.

If you can provide some direction on a proper fix (I don't know if that one line is enough for public use), I'm happy to contribute a change. Thanks for writing the library!

Given these stucts:

    type Path struct {
        Description string
        Paths       map[string]Path
    }

    type Top struct {
        Title       string
        Path
    }

I should be able to unmarshal a YAML like this:

---
  title: Recursive
  paths:
    a:
      paths:
        b:
          paths:
            c:
              description: Bottom

This works but should be the same thing as above:

    type Path struct {
        Description string
        Paths       map[string]Path
    }

    type Top struct {
        Title       string
        Paths    map[string]Path
    }

Sometimes you want to define your own marshaling/unmarshaling way, it is useful for things like time or if you want to treat empty values differently.

So please add support for Text Marshaler/Unmarshaler interfaces.

If I have camel case names in the conf with variables in a struct with camel case as well it fails to decode the variables into the struct. If I rename the field to not have camel case it works.

broken conf file looks like this:

reportSeconds: 30
something: "woo"
monitor:
  - maintenance
  - dispatch
  - elasticsearch
  - cassandra
  - columbo
  - heartThrob
  - carpenter

code to read the file looks like this:

import (
        "gopkg.in/yaml.v2"
        "io/ioutil"
)

type conf struct {
        ReportSeconds         int64
        Something             string
        Monitor               []string
}

func getConf(path string) (c conf, err error) {
        c = conf{}
        data, err := ioutil.ReadFile(path)
        if nil != err {
                return c, err
        }
        err = yaml.Unmarshal([]byte(data), &c)
        return c, err
}

if I just change it to look like this :

reportseconds: 30
something: "woo"
monitor:
  - maintenance
  - dispatch
  - elasticsearch
  - cassandra
  - columbo
  - heartThrob
  - carpenter

Everything works fine.