Giter VIP home page Giter VIP logo

cloudformation-ruby-dsl's People

Contributors

amosshapira avatar bkruger99 avatar carles-figuerola avatar epelletier avatar flyinbutrs avatar grimm26 avatar harlanbarnes avatar harman28 avatar hjkatz avatar ianomaly avatar jonaf avatar kierranm avatar lethalpaga avatar markatwood avatar noirbot avatar olyhuta avatar ozbillwang avatar scarybot avatar ssuprun avatar temujin9 avatar tianx2 avatar zhelyan avatar zmingxie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cloudformation-ruby-dsl's Issues

Migrate away from cfn-cmd

Currently, we drop to the shell to evoke cfn-cmd for interaction with the Cloudformation API.

  • It's cleaner, all within Ruby, and doesn't require separate configuration/installation of tools
  • We can do more to refactor everything and be more object-oriented
  • It fixes a few bugs (like having spaces in parameter or tag values)
  • It's not deprecated :)

Can I reference a parameter/tag/value inside a description?

I'd like to be able to define resources with dynamic descriptions such as:

resource 'DBSubnet', :Type => 'AWS::RDS::DBSubnetGroup', :Properties => { :DBSubnetGroupDescription => 'DBSubnet for #{ref(mystack)}' :CidrBlock => '10.0.201.0/24' :VpcId => 'vpc123'

service binary interferes with service command

Because the gem comes with a command called service, I can no longer use the usual service command on Ubuntu Linux without using its full path, since service now resolves to the gem's command rather than to the default, which manages starting or stopping services. Is there a way this command could be renamed to something less generic, or is there a way to selectively ignore commands or otherwise resolve this conflict so that I can use the gem? Right now all I can do is rm the service command out of /usr/local/bin. Thanks!

How do I put a backslash in interpolate?

Hello,

I am using

:UserData => base64(interpolate(my_content))

mycontent returns a cloud-init yaml. One of them is this:

runcmd:
  - hostname $(curl -s http://169.254.169.254/latest/meta-data/instance-id/ | sed -r -e "s/i-([0-9a-f]{,8})/#{env.downcase}-#{purpose.downcase}-\1-ops.example.com/g")

The \1 in the sed command, interpolate turns it into a \u000 which Cloudformation dislikes.

I'm tryiing to make my system's hostname to be:
from

i-abcd1234

to this

dev-php-abcd1234.ops.example.com

Using a regex backreference is not my first choice, but it appears this is the only thing available to me. I'm open to doing it another way as long as I achieve the name. Thank you.

Keyword arguments do not work in Ruby 1.9

In this commit you've added new method and variable with keyword argument:

  def initialize(**args)
    Aws.config[:region] = args[:region] if args.key?(:region)
  end

  aws_cfn = AwsCfn.new(region: template.aws_region)

but keyword arguments work only with Ruby 2.0.
Does it mean that we should use cloudformation-ruby-dsl with Ruby 2.0 or it's just a mistake?

Thanks!

diff - failed to satisfy constraint

The cloudformation template EC2InstanceWithSecurityGroupSample.template is downloaded from https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/EC2InstanceWithSecurityGroupSample.template

$ bundle exec cfntemplate-to-ruby EC2InstanceWithSecurityGroupSample.template > sample.rb

$ bundle exec ruby sample.rb diff EC2InstanceWithSecurityGroupSample.template
Error: 1 validation error detected: Value 'EC2InstanceWithSecurityGroupSample.template' at 'stackName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z][-a-zA-Z0-9]*|arn:[-a-zA-Z0-9:/._+]*

Testbed needed!

Really, this should have been done when we first created the DSL, but we need to go ahead and put together a test suite to improve our releases and minimize breakage. There are quite a few edge cases and it gets quite complicated to test each change thoroughly.

How can AWS CloudFormation help?

cloudformation-ruby-dsl contributors and users,

Is there anything the AWS CloudFormation service could provide that can help in using and enhancing cloudformation-ruby-dsl? (For example, any additional API, any update to the template format, etc.)

Sincerely,
The AWS CloudFormation Team

cfn-validate-template not working

Using 0.4.7, I get the following error. I have AWS_ACCESS_KEY and AWS_SECRETY_KEY set and exported.

$ ./demo.rb cfn-validate-template cfn-validate-template: Malformed input-No Credentials were provided - cannot access the service
Usage:
cfn-validate-template
[--template-file value ] [--template-url value ] [General Options]
For more information and a full list of options, run "cfn-validate-template --help"

Execution of 'cfn-cmd' failed. To facilitate debugging, the generated JSON template file was not deleted. You may delete the file manually if it isn't needed: /Users/ksetzer/Projects/outbreak/templates/demo.rb.expanded.json

bringing in template snippets

I have a cloudwatch alarm taht I would like to add any time I create an EC2 instance. The snippet looks like so:

    resource instance_name + 'RecoveryTestAlarm',
      Type: 'AWS::CloudWatch::Alarm',
      Properties: {
        AlarmDescription: 'Trigger a recovery when instance status check fails for 15 consecutive minutes.',
        Namespace: 'AWS/EC2' ,
        MetricName: 'StatusCheckFailed_System',
        Statistic: 'Minimum',
        Period: '60',
        EvaluationPeriods: '15',
        ComparisonOperator: 'GreaterThanThreshold',
        Threshold: '0',
        AlarmActions: [ join(':', "arn:aws:automate", ref('AWS::Region'), 'ec2:recover') ],
        Dimensions: [{Name: 'InstanceId',Value: ref(instance_name)}]
      }

Now, instance_name is the name of the AWS::EC2::Instance resource. Eventually I'll have more than this, so I'd like to abstract it into a file or a method. I have a class that I wrote to use with the dsl so I can pull it out of there.
The problem with loading it from a file is the existence of the instance_name variable. load_from_file generates a complaint about not knowing what instance_name is. interpolate(file()) assumes I want to use it as UserData. Using a method to generate it seems to work but I'm not sure how to inject that into the template.

Add cfn-replace-stack feature

Create a subcommand that:

  • Creates a new stack, automatically appending a timestamp
  • Waits for it to be successfully created
  • (Optionally) Deletes an old stack matching the new stack name + some timestamp

And, if we're feeling particularly generous, let's add a few features to it:

  • Create multiple stacks in parallel
  • Have a DSL for creating multiple stacks in parallel and watching them (similar to a config file)

This is essentially a migration of an internal tool at Bazaarvoice that does a very similar thing, but migrates the functionality to the Ruby DSL.

diff does not work - cfn-cmd not found

[2014-04-21 15:05:08] pavel.lishin@plishin:/gitwork/Hubs-Core
# bundle exec cfntemplate-to-ruby scripts/cloudformation/abba.autoscale.json > scripts/cloudformation/abba.autoscale.rb

[2014-04-21 15:05:24] pavel.lishin@plishin:/gitwork/Hubs-Core
# chmod u+x scripts/cloudformation/abba.autoscale.rb

[2014-04-21 15:04:33] pavel.lishin@plishin:/gitwork/Hubs-Core
# scripts/cloudformation/abba.autoscale.rb diff scripts/cloudformation/abba.autoscale.json
sh: cfn-cmd: command not found

Add default CAPABILITY_IAM capability

The --capabilities parameter now requires the CAPABILITY_IAM value for cfn-create-stack and cfn-update-stack otherwise the command fails with error Malformed input-Requires capabilities : [CAPABILITY_IAM]

Add support for Stack Policies

Although the immutability feature of parameters offers a layer of protection around Cloudformation stack updates, Amazon themselves have provided a feature called "stack policies," which allows you to specify a policy (very similar to IAM policies) regarding which resources may be updated (and how -- replaced, updated, or deleted). Let's consider supporting this feature in the Ruby DSL.

Amazon stack policy documentation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html

Related concepts: change sets

Tagging lacks good error reporting on failure (was: Setup for authentication...)

Apologies for all of these 'issues' that not really issues, but I can't find documentation anywhere...

I have a template which validates just fine:
bundle exec ruby my-stack.rb validate --parameters "Param1=foo;Param2=bar" --stack-name my-stack --region us-east-1

However when I try to create:
bundle exec ruby auvik-cluster.rb create --parameters "Param1=foo;Param2=bar" --stack-name my-stack --region us-east-1
Failed to create stack:

My guess is I am unable to authenticate to cloudformation?

I am able to expand the template and upload it to do the cloudformation run, my question is how are credentials picked up? ENV variables, Profiles/Credentails file, Roles?

Use optparse or thor

I want to improve the cli handling. In the gemfile there aren't lots of dependencies, which makes me think optparse will be preferred. Thor is a little prettier. Any feedback before I make a PR?

cfn-update-stack fails when stack does not have tags

The failure occurs because cfn-describe-stacks returns the string "(nil)" for empty values and cfntemplate.rb is comparing a NilClass with a String:

if cfn_tags != old_cfn_tags .... end

This causes this error:

$ ruby app.rb cfn-update-stack qa-services
CloudFormation stack tags do not match and cannot be updated. You must either use the same tags or create a new stack.
< (nil)

---
$

Unable to install cloudformation-ruby-dsl with gem

[2014-04-21 13:23:49] pavel.lishin@plishin:~
# gem install cloudformation-ruby-dsl
ERROR:  Could not find a valid gem 'cloudformation-ruby-dsl' (>= 0) in any repository
ERROR:  Possible alternatives: cloud_formation, cloud_formation_template

[2014-04-21 13:42:55] pavel.lishin@plishin:~
# gem sources
*** CURRENT SOURCES ***

https://rubygems.org/

[2014-04-21 13:43:56] pavel.lishin@plishin:~
#

Add support for stack metadata

It would be useful to have metadata available for this cloudformation feature:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-interface.html

I dug up PR #65 and confirmed that this code:

  metadata 'AWS::Cloudformation::Interface' => {
    :ParameterGroups => [ {
      :Label => {
        :default => "App1 Settings"
      },
      :Parameters => [
        :App1InstanceType,
        :App1MinInstanceCount,
        :App1DesiredInstanceCount,
        :App1MaxInstanceCount
      ],
    },
    {
      :Label => {
        :default => "App2 Settings"
      },
      :Parameters => [
        :App2InstanceType,
        :App2MinInstanceCount,
        :App2DesiredInstanceCount,
        :App2MaxInstanceCount
      ]
    } ]
  }

produces this json output:

{
  "Metadata": {
    "AWS::Cloudformation::Interface": {
      "ParameterGroups": [
        {
          "Label": {
            "default": "App1 Settings"
          },
          "Parameters": [
            "App1InstanceType",
            "App1MinInstanceCount",
            "App1DesiredInstanceCount",
            "App1MaxInstanceCount"
          ]
        },
        {
          "Label": {
            "default": "App2 Settings"
          },
          "Parameters": [
            "App2InstanceType",
            "App2MinInstanceCount",
            "App2DesiredInstanceCount",
            "App2MaxInstanceCount"
          ]
        }
      ]
    }
  }
}

Suggestion: add get_output function

It will be handy if we can do e.g:

  parameter 'VpcId',
            Type: 'String',
            Default: get_output('VpcId', 'my_vpc_template'),
            Description: 'The ID of you Virtual Private Cloud (VPC)',
            Immutable: true

example does not work

% ./cloudformation-ruby-script.rb expand
./cloudformation-ruby-script.rb:176:in `block in <main>': undefined method `equals' for #<TemplateDSL:0x007f8c9aa87720> (NoMethodError)
        from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:260:in `instance_eval'
        from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:260:in `initialize'
        from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:293:in `initialize'
        from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:284:in `new'
        from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:284:in `template'
        from ./cloudformation-ruby-script.rb:24:in `<main>'

Credentials regression error in 1.2.3

When using:

./cloudformation/policies.rb delete my_test_stack

The following error occurs when using 1.2.3

.../gems/aws-sdk-core-2.5.8/lib/aws-sdk-core/plugins/request_signer.rb:100:in `require_credentials': unable to sign request without credentials set (Aws::Errors::MissingCredentialsError)
...
...
from .../gems/cloudformation-ruby-dsl-1.2.3/lib/cloudformation-ruby-dsl/cfntemplate.rb:305:in `cfn'
from .../gems/cloudformation-ruby-dsl-1.2.3/lib/cloudformation-ruby-dsl/cfntemplate.rb:564:in `exec!'

This worked fine in 1.2.1.

It does of course work if you execute this with the --profile and --region switches.

Its not clear from the change why the environment variables capability was removed:

# Following line can be uncommented only when Amazon will provide the stable version of this functionality.
# Aws.config[:credentials] = Aws::SharedCredentials.new(profile_name: args[:aws_profile]) unless args[:aws_profile].nil?

What was unstable? AWS have released a number of new versions of the Ruby SDK since this change was made. Without knowing the nature of the issue, I can't check this. Would be happy to provide a PR once I have some more information.

Thanks

CSV parse_char issue

cfn-update-stack fails because the CSV parser splits comma separated parameter values into different fields if the parameter string is double quoted. This , I believe, happens because of the change in 6ead33c. The actual error is: cfn-update-stack may not update immutable parameter and that's because the retrieved template parameters are never parsed beyond the parameter at which the split error occurs.

To reproduce: create a stack with description like "Stack, the" or one having a parameter which is a comma separated list and run cfn-update-stack on it

no implicit conversion of FnCall into Array (TypeError)

$ curl https://raw.githubusercontent.com/thefactory/cloudformation-zookeeper/master/zookeeper-vpc.json
$ cfntemplate-to-ruby zookeeper-vpc.json

/opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:307:in `simplify': no implicit conversion of FnCall into Array (TypeError)
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `block in simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `map'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'`
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:51:in `block in main'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:50:in `each'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:50:in `main'
    from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:327:in `<top (required)>'
    from /opt/boxen/rbenv/versions/2.1.1/bin/cfntemplate-to-ruby:23:in `load'
    from /opt/boxen/rbenv/versions/2.1.1/bin/cfntemplate-to-ruby:23:in `<main>'

Cannot create stack

Ok, so this is a weird one.

I've been testing out this gem for a while and I think my company is going to make use of it. However I can't seem to create stacks.

When calling ./my_stack.rb create my_stack --parameters "blah" I receive the cryptic response of "Failed to create stack:" with no message. Looking into this with binding.pry I find that the exception being caught is on line 302 of lib/cloudformation-ruby-dsl/cfntemplate.rb. The exception class is Aws::CloudFormation::Errors::ValidationError which is not a class that is documented (I think it's autogenerated by the aws-sdk service layer). This exception does not have a message, code, or really any useful information.

Things that I have tried with varying results:

  • Validating the stack with the gem (ok)
  • Validating the stack with aws online tool (ok)
  • Validating the stack with aws cli tool (ok)
  • Creating the stack with aws cli tool (ok) <-- This is the weird one
  • Creating the stack with the gem (not ok, validation error)

I have attached the ruby template I wrote, and I would love any and all help in fixing this issue.

Thanks!
--Katz

Here's the exact commands I'm running:

  • ./database.cfn.rb create test-stack --parameters "ParamDBRole=read-write;ParamDBSizePostgres=50;ParamDBSizeCyanAudit=100;ParamDBIops=100;ParamCustomerLabel=testing"
  • aws cloudformation create-stack --stack-name test-stack --parameters ParameterKey=ParamDBRole,ParameterValue=read-write ParameterKey=ParamDBSizePostgres,ParameterValue=50 ParameterKey=ParamDBSizeCyanAudit,ParameterValue=100 ParameterKey=ParamDBIops,ParameterValue=100 ParameterKey=ParamCustomerLabel,ParameterValue=testing --template-body file://test.json

Note: test.json is just ./database.cfn.rb expand > test.json

database.cfn.rb.txt

Variables aren't passed down to the other objects

Not sure this is a bug but I cannot find the "right" way to do this. I have defined a variable in the root RB template and it is not in scope when calling the load_from_file method. I used super global (@@) variables to get around this but this doesn't seem quite right.

Suggestions?

pass additional options to cloudformation

Any additional options don't appear to be forwarded on to cloudformation during stack creation. We need to be able to pass "disable-rollback" for debugging bootstrapping issues.

Assumed Roles are ignored

Suppose assumed roles are used as follows:

[default]
aws_access_key_id = foo
aws_secret_access_key = bar

[test]
role_arn = arn:aws:iam::123456789012:role/group/username
source_profile = default

With the provided access configuration in either ~/.aws/credentials or ~/.aws/config it not possible to create a stack via cloudformation-ruby-dsl v1.2.1, aws-sdk-ruby 2.4.2 and ruby 2.3.0:

$ ./stack.rb create test-stack --profile test --region us-east-1 --parameters "\
Param1=1-20;\
Param2=teamname"

 /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:100:in `require_credentials': unable to sign request without credentials set (Aws::Errors::MissingCredentialsError)
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:90:in `sign_authenticated_requests'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:83:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/query/handler.rb:27:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/endpoint.rb:41:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/response_target.rb:21:in `call'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/request.rb:70:in `send_request'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/cloudformation-ruby-dsl-1.2.1/lib/cloudformation-ruby-dsl/cfntemplate.rb:297:in `cfn'
   from /home/user/vendor/bundle/ruby/2.3.0/gems/cloudformation-ruby-dsl-1.2.1/lib/cloudformation-ruby-dsl/cfntemplate.rb:556:in `exec!'
   from ./stack.rb:291:in `<main>'

sh: cfn- commands not found

After updating the cloudformation-ruby-dsl gem to ~> 0.5.0 the cfn-*-stack commands are not found.

It started working again after brew install aws-cfn-tools.

build_nested_map Depth Limit and Concatenation

Hi @jonaf,

I have a some questions about your intended use of build_nested_map before I propose a change:

  • I noticed that the map depth has no practical limit and will be as deep as there are values for the path parameter. However, CloudFormation has a hard limit on mapping depth: you can only map a top-level key to one level of key-value pairs. Would you mind if I enforced this limit in the method (i.e. only first 3 values of path parameter)? Are there any other utility benefits to having an uncapped depth that I am missing or that we might want in the future?
  • I was hoping to then enable the use of any values after the first 3 to be concatenated with value number 3 so that it is possible to form a compound value (note that region and zone are concatenated in the output JSON). So for example:
logical_id     region        zone  env          physical_id
GlobalSubnetA  eu-west-1     a     prod         subnet-abc123
GlobalSubnetB  eu-west-1     b     prod         subnet-def456
GlobalSubnetC  eu-west-1     c     prod         subnet-ghi789
  subnets = Table.load 'maps/subnet-ids.txt'
  mapping 'AZsByRegion',
    subnets.get_multimap({}, :region, :env, :region, :zone)
    "AZsByRegion": {
      "eu-west-1": {
        "prod": [
          "eu-west-1a",
          "eu-west-1b",
          "eu-west-1c"
        ]
      },

Thoughts?

Thank you,
Cameron

Parse CSV field using header name instead of specific position

We're accessing values in CSV data using specific field positions, which are susceptible to failure if the field position changes. It is better to refer to the field based on the header name, for example:

Before:

CSV.parse_line(old_stack_description)[6]

After:

CSV.parse_line(old_stack_description, :headers => true)['DESCRIPTION']

How to use tags?

I'm not sure how to use the tag capabilities in the DSL...
By the example below I was hoping to see all the resources created to have both tags Foo and Bar...
Is this incorrect?
`

Tags

tag :Foo => ref('Foo')
tag :Bar => ref('Bar')

Resources

resource "vpc",
:Type => 'AWS::EC2::VPC',
:Properties => {
:CidrBlock => '10.0.0.0/16',
:InstanceTenancy => 'default',
:EnableDnsSupport => 'true',
:EnableDnsHostnames => 'true',
}`

Add --help option

We need --help! Our documentation doesn't do a good job of providing usage info, and in general, we should be able to add --help to a command and get some usage instructions. This should alleviate some of the confusion / issues we have seen, such as #67 .

Search local path for table files

Currently, when including a map file, we do something like:

map = Table.load 'my-map.txt'

This is, of course, relative to Cloudformation template file. Therefore, if your directory structure looks like this:

.
└── cloudformation
    ├── includes
    │   └── my-map.yaml
    └── my-template.rb

In order for my-template.rb to work, you must first change your working directory to cloudformation:

cd cloudformation
./my-template.rb cfn-create-stack ...

The Table class should search paths relative to the CloudFormation Ruby DSL template file in addition to the present working directory. This way, my-template.rb may be called from anywhere and still find and read its map files.

As a temporary workaround, as demonstrated by @epelletier, the map path can be calculated when the script is run:

map = Table.load File.dirname(__FILE__) + '/includes/my-map.yaml'

Ideally, this shouldn't be necessary, as setting up multiple maps requires unnecessary code copy. I think this should "just work," so that a Cloudformation template doesn't suddenly break because a map was included in the template.

Spaces in Tag Value causes "Malformed input-No value separator" error

I realize this is such an edge case ... but I thought I'd bring it up in case anybody else runs across this. I'd fix it if I could, but it looks like it's bigger change that I'm comfortable with. I'm just happy to have this tool. 😄

We use some tags that have spaces in the value. For example:

tag Application: 'My App'

This might be a bad idea for other reasons, but at the moment, it's what we do and we've got a lot of existing tags in this format.

It looks like system calls escape spaces for the command line according to this and I'm guessing that's what's fouling up the cfn-create stack call. It might cause other problems with the diff and cfn-update-stack too even if the cfn-create-stack case could be worked around.

It didn't look like there's an obvious way to fix this without a lot of testing. So presuming that the spaces are just going to not be supported for now, I was wondering if you'd accept a PR that checks the tag values for spaces and causes an error that explains the problem and how to workaround (i.e. remove the spaces)? The reason I ask is the current error (using a tag example like the 'My App' above) isn't super obvious about the actual problem:

cfn-create-stack:  Malformed input-No value separator = found in App

Thanks. And again, great tool!

Mapping method returns entire map instead of requested map

There's a bug in the mapping method which causes the produced JSON to be malformed when using external map files. For example, when using this external map file:

=> maps.json

{
    "Mappings": {
        "JsonExampleMap": {
            "foo": "bar"
        }
    }
}

with

mapping 'JsonExampleMap', 'maps/map.json'

The produced Cloudformation stack JSON contains an incorrect duplicate map:

"JsonExampleMap": {
  "JsonExampleMap": {
    "foo": "bar"
  }
},

The expected resulting map should have been (without duplication):

"JsonExampleMap": {
    "foo": "bar"
}

This problem affects all external maps in the following formats: JSON, Ruby and YAML

Intrinsic Join Function

Hey,

Awesome utility, thanks for writing it. I've found that the join function doesn't seem to be evaluated/respected when used in certain cases:

Example:

Ruby template (see SecurityGroups)

  resource 'Name', 
    :Type => 'AWS::AutoScaling::LaunchConfiguration', 
    :Metadata => { 
      :'AWS::CloudFormation::Init' => {
        ...,
        ...
      }, 
    :Properties => {
      :IamInstanceProfile => 'value',
      :ImageId => ref('ImageId'),
      :InstanceType => ref('InstanceType'),
      :KeyName => ref('KeyName'),
      :SecurityGroups => [ join(',', ref('SecurityGroups')) ]
    }

Expanded template (see SecurityGroups)

...
...
...
"Properties": {
  "IamInstanceProfile": "value",
  "ImageId": {
    "Ref": "ImageId"
  },
  "InstanceType": {
    "Ref": "InstanceType"
  },
  "KeyName": {
    "Ref": "KeyName"
  },
  "SecurityGroups": [
    {
      "Ref": "SecurityGroups"
    }
  ]
}

What I would expect to see

...
...
"Properties": {
  "IamInstanceProfile": "value",
  "ImageId": {
    "Ref": "ImageId"
  },
  "InstanceType": {
    "Ref": "InstanceType"
  },
  "KeyName": {
    "Ref": "KeyName"
  },
  "SecurityGroups": [ 
    { 
      "Fn::Join": [ 
        ",", 
        [
          {
            "Ref": "SecurityGroups"
          }
        ]
      ]
    }
  ]
}

The inverse is fine however (converting JSON to the Ruby DSL).

$ cfntemplate-to-ruby template.json
# Would output
...
  :SecurityGroups => [ join(',', ref('SecurityGroups')) ],
...

Use Case: Use cloudformation-ruby-dsl as a library

I'm not sure if I'm going to explain this well, but I'll give it a shot.

I use cloudformation-ruby-dsl in a slightly different way. I "add features" to it by the fact that it is ruby script and can do extra things like "lookup the latest AMI ID" or "read a YAML file for more information". As such, I pass in command-line arguments and parse them using a rubygem (mixlib-cli in my case, but it could be anything.)

As a side note, prior to aws-sdk I wasn't able to do this because all of ARGV got passed directly to the old cfn tools (at which case, it would fail because of it not recognizing CLI options.) At the moment, I can ALMOST get it to work. The quirk is that my (the "outer") parsing of CLI options has to be able to handle any options that the cloudformation-ruby-dsl requires (i.e. --stack-name has to be handled by my options parsing ... but this is manageable) ... while using any CLI options that cloudformation-ruby-dsl doesn't support can cause problems (so far, the biggest problem I've found is that diff action seems to pass in all of ARGV that the cloudformation-ruby-dsl doesn't recognize to the Diffy object as options ... which causes Diffy to not work.)

I guess at the heart of it, I'd want to be able to use the template DSL more like a library ... but, of course, let others use it as a pure standalone script too.

Is there a way to do this that I am just missing?

profile fix broke it

#91 broke authentication for people who either have no default profile set in awscli config or are implicitly using that profile anyway. #93 fixes.

Suggestion: Better name

This is Ruby. While "cloudformation-ruby-dsl" is informative, it lacks a certain panache that all the other cool Ruby library names have.

I have no suggestions for any specific "better name" but I'd like to see such a great tool have an equally great name.

Also, cloudformation-ruby-dsl has saved me countless hours and has made it much easier to manage and grok otherwise complicated CloudFormation JSON templates. Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.