Working with nested structures about terraform-provider-yaml HOT 10 CLOSED

Glad it was helpful and I'm happy you found the yaml provider plugin to be useful!

from terraform-provider-yaml.

Hi! Thanks for reporting this issue!

I do not think you're doing anything wrong - instead I see your use case as perfectly valid and a nice and smart way to use the provider! If I were you I'd definitely expect that to work. In fact, I being myself was very surprised this doesn't work. :)

I believe we need a unit-test to cover such a use-case and a fix that would ensure this works. I'd welcome a contribution on that or I will try to fix it myself once I have some free time, maybe next week.

from terraform-provider-yaml.

Actually, fixed - give it a try https://github.com/ashald/terraform-provider-yaml/releases/tag/v2.0.2

from terraform-provider-yaml.

Thank's a lot! Works perfect!

from terraform-provider-yaml.

Hi!

I believe I'm running into a similar problem but with a slightly different error.

YAML:

name: example_name
owner: example_owner
columns:
  - name: column0
    type: boolean
  - name: column1
    type: integer

Terraform:

data "yaml_map_of_strings" "table" {
  input = "${file("~/table.yaml"}"
}

data "yaml_list_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table.output["columns"]}"
}

output "columns" {
  value = "${data.yaml_map_of_strings.columns.output}"
}

Errors:

* data.yaml_map_of_strings.columns: data.yaml_map_of_strings.columns: yaml: unmarshal errors:
  line 1: cannot unmarshal !!seq into map[string]interface {}

from terraform-provider-yaml.

@victoriaalee I tried to reproduce your issue but wasn't able to get to the same result.

When I tried exactly your configuration it actually didn't work and failed with an error like:

Error: output 'columns': unknown resource 'data.yaml_map_of_strings.columns' referenced in variable data.yaml_map_of_strings.columns.output

which I believe is because of:

I adjusted the reference accordingly:

data "yaml_map_of_strings" "table" {
  input = <<EOF
name: example_name
owner: example_owner
columns:
  - name: column0
    type: boolean
  - name: column1
    type: integer
EOF
}

data "yaml_list_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table.output["columns"]}"
}

output "columns" {
  value = "${data.yaml_list_of_strings.columns.output}"
}

and it worked well for me:

$ terraform apply
data.yaml_map_of_strings.table: Refreshing state...
data.yaml_list_of_strings.columns: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

columns = [
    {name: column0, type: boolean},
    {name: column1, type: integer}
]

Are you using the latest version of the provider?

from terraform-provider-yaml.

Thanks for catching that! Unfortunately, I still have the error.

I installed the provider according to the README:

$ wget "https://github.com/ashald/terraform-provider-yaml/releases/download/v2.0.2/terraform-provider-yaml_v2.0.2-$(uname -s | tr '[:upper:]' '[:lower:]')-amd64"
$ chmod +x ./terraform-provider-yaml*

YAML:

name: example_name
owner: example_owner
columns:
  - name: column0
    type: boolean
  - name: column1
    type: integer

Terraform:

data "yaml_map_of_strings" "table_test" {
  input = "${file("table.yaml")}"
}

data "yaml_map_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table_test.output["columns"]}"
}

Error:

* data.yaml_map_of_strings.columns: data.yaml_map_of_strings.columns: yaml: unmarshal errors:
  line 1: cannot unmarshal !!seq into map[string]interface {}

from terraform-provider-yaml.

Ah, I see! When I made an assumption about the broken reference I made it wrong in a sense that I assumed you wanted to use yaml_list_of_strings while it appears that you intendeed to use yaml_map_of_strings.

While the error message is not ideal the behavior is correct. As much as I love YAML, it sometimes may be a little bit too obscure. In this case:

data "yaml_map_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table_test.output["columns"]}"
}

tried to parse content of the field column as a map while it's a list of maps!

  - name: column0
    type: boolean
  - name: column1
    type: integer

If I were to rewrite this in flow-style YAML or JSON, this would be equivalent to:

[{"name": "column0", "type": "bolean"}, {"name": "column1", "type": "integer"}]

As we can see, we cannot really parse this as a map.

Does it make sense to you?

At this point I'm not really sure what is the desired behavior. If you could share what are you trying to achieve maybe I will be able to assist you with that?

For instance, if I were to assume that you need to get a mapping between column names and their types then it can be achieved like this:

data "yaml_map_of_strings" "table_test" {
  input = <<EOF
name: example_name
owner: example_owner
columns:
  - name: column0
    type: boolean
  - name: column1
    type: integer
EOF
}

data "yaml_list_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table_test.output["columns"]}"
}

data "yaml_map_of_strings" "columns" {
  count = "${length(data.yaml_list_of_strings.columns.output)}"
  input = "${element(data.yaml_list_of_strings.columns.output, count.index)}"
}

// A hacky way to get a list of values in "name" field
data "template_file" "names" {
  count = "${length(data.yaml_list_of_strings.columns.output)}"
  template = "${lookup(data.yaml_map_of_strings.columns.*.output[count.index], "name")}"
}

// A hacky way to get a list of values in "type" field
data "template_file" "types" {
  count = "${length(data.yaml_list_of_strings.columns.output)}"
  template = "${lookup(data.yaml_map_of_strings.columns.*.output[count.index], "type")}"
}

output "columns" {
  value = "${zipmap(data.template_file.names.*.rendered, data.template_file.types.*.rendered)}"
}

so that:

$ terraform init
...
* provider.template: version = "~> 2.1"
* provider.yaml: version = "~> 2.0"
...

$ terraform apply
data.yaml_map_of_strings.table_test: Refreshing state...
data.yaml_list_of_strings.columns: Refreshing state...
data.yaml_map_of_strings.columns[1]: Refreshing state...
data.yaml_map_of_strings.columns[0]: Refreshing state...
data.template_file.names[0]: Refreshing state...
data.template_file.names[1]: Refreshing state...
data.template_file.types[0]: Refreshing state...
data.template_file.types[1]: Refreshing state...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

columns = {
  column0 = boolean
  column1 = integer
}

from terraform-provider-yaml.

I'm interested in getting a list of maps with the column info. The code you provided with using the output of the yaml_list_of_strings in another yaml_map_of_strings data source was helpful. Thanks!

data "yaml_map_of_strings" "table_test" {
  input = <<EOF
name: example_name
owner: example_owner
columns:
  - name: column0
    type: boolean
  - name: column1
    type: integer
EOF
}

data "yaml_list_of_strings" "columns" {
  input = "${data.yaml_map_of_strings.table_test.output["columns"]}"
}

data "yaml_map_of_strings" "columns" {
  count = "${length(data.yaml_list_of_strings.columns.output)}"
  input = "${element(data.yaml_list_of_strings.columns.output, count.index)}"
}

output "columns" {
  value = "${data.yaml_map_of_strings.columns.*.output}"
}

Output:

columns = [
    {
        data_type = boolean,
        name = column0,
        pii = confidential
    },
    {
        data_type = integer,
        name = column1,
        pii = restricted
    }
]

Now I'm trying to use data.yaml_map_of_strings.columns.*.output as list input into a different resource but somehow it complains about that not being a list while at the same time using that list as input into the element function does not produce any errors saying that that isn't a list... I think this is a general Terraform question at this point though.

EDIT

I forgot that you have to have the brackets around it:

[${data.yaml_map_of_strings.columns.*.output}]

Things work as expected!

from terraform-provider-yaml.

And given that v0.12 is out - take a look at https://github.com/ashald/terraform-provider-yaml/blob/master/docs/data_source_yaml_to_json.md as it may simplify things even further!

from terraform-provider-yaml.