bazaarvoice / cloudformation-ruby-dsl Goto Github PK
View Code? Open in Web Editor NEWRuby DSL for creating Cloudformation templates
License: Apache License 2.0
Ruby DSL for creating Cloudformation templates
License: Apache License 2.0
Currently, we drop to the shell to evoke cfn-cmd
for interaction with the Cloudformation API.
I'd like to be able to define resources with dynamic descriptions such as:
resource 'DBSubnet', :Type => 'AWS::RDS::DBSubnetGroup', :Properties => { :DBSubnetGroupDescription => 'DBSubnet for #{ref(mystack)}' :CidrBlock => '10.0.201.0/24' :VpcId => 'vpc123'
Because the gem comes with a command called service
, I can no longer use the usual service
command on Ubuntu Linux without using its full path, since service
now resolves to the gem's command rather than to the default, which manages starting or stopping services. Is there a way this command could be renamed to something less generic, or is there a way to selectively ignore commands or otherwise resolve this conflict so that I can use the gem? Right now all I can do is rm
the service
command out of /usr/local/bin
. Thanks!
Hello,
I am using
:UserData => base64(interpolate(my_content))
mycontent
returns a cloud-init yaml. One of them is this:
runcmd:
- hostname $(curl -s http://169.254.169.254/latest/meta-data/instance-id/ | sed -r -e "s/i-([0-9a-f]{,8})/#{env.downcase}-#{purpose.downcase}-\1-ops.example.com/g")
The \1
in the sed command, interpolate
turns it into a \u000
which Cloudformation dislikes.
I'm tryiing to make my system's hostname to be:
from
i-abcd1234
to this
dev-php-abcd1234.ops.example.com
Using a regex backreference is not my first choice, but it appears this is the only thing available to me. I'm open to doing it another way as long as I achieve the name. Thank you.
In this commit you've added new method and variable with keyword argument:
def initialize(**args)
Aws.config[:region] = args[:region] if args.key?(:region)
end
aws_cfn = AwsCfn.new(region: template.aws_region)
but keyword arguments work only with Ruby 2.0.
Does it mean that we should use cloudformation-ruby-dsl with Ruby 2.0 or it's just a mistake?
Thanks!
When I convert this JSON and expand it back
http://us-east-1-aws-training.s3.amazonaws.com/self-paced-lab-14/Template1.json
The Fn::FindInMap is lost from the JSON output, and it instead it hardcodes into the JSON the first element of the mapping.
The cloudformation template EC2InstanceWithSecurityGroupSample.template
is downloaded from https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/EC2InstanceWithSecurityGroupSample.template
$ bundle exec cfntemplate-to-ruby EC2InstanceWithSecurityGroupSample.template > sample.rb
$ bundle exec ruby sample.rb diff EC2InstanceWithSecurityGroupSample.template
Error: 1 validation error detected: Value 'EC2InstanceWithSecurityGroupSample.template' at 'stackName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z][-a-zA-Z0-9]*|arn:[-a-zA-Z0-9:/._+]*
Really, this should have been done when we first created the DSL, but we need to go ahead and put together a test suite to improve our releases and minimize breakage. There are quite a few edge cases and it gets quite complicated to test each change thoroughly.
cloudformation-ruby-dsl contributors and users,
Is there anything the AWS CloudFormation service could provide that can help in using and enhancing cloudformation-ruby-dsl? (For example, any additional API, any update to the template format, etc.)
Sincerely,
The AWS CloudFormation Team
Using 0.4.7, I get the following error. I have AWS_ACCESS_KEY and AWS_SECRETY_KEY set and exported.
$ ./demo.rb cfn-validate-template cfn-validate-template: Malformed input-No Credentials were provided - cannot access the service
Usage:
cfn-validate-template
[--template-file value ] [--template-url value ] [General Options]
For more information and a full list of options, run "cfn-validate-template --help"
Execution of 'cfn-cmd' failed. To facilitate debugging, the generated JSON template file was not deleted. You may delete the file manually if it isn't needed: /Users/ksetzer/Projects/outbreak/templates/demo.rb.expanded.json
I have a cloudwatch alarm taht I would like to add any time I create an EC2 instance. The snippet looks like so:
resource instance_name + 'RecoveryTestAlarm',
Type: 'AWS::CloudWatch::Alarm',
Properties: {
AlarmDescription: 'Trigger a recovery when instance status check fails for 15 consecutive minutes.',
Namespace: 'AWS/EC2' ,
MetricName: 'StatusCheckFailed_System',
Statistic: 'Minimum',
Period: '60',
EvaluationPeriods: '15',
ComparisonOperator: 'GreaterThanThreshold',
Threshold: '0',
AlarmActions: [ join(':', "arn:aws:automate", ref('AWS::Region'), 'ec2:recover') ],
Dimensions: [{Name: 'InstanceId',Value: ref(instance_name)}]
}
Now, instance_name is the name of the AWS::EC2::Instance resource. Eventually I'll have more than this, so I'd like to abstract it into a file or a method. I have a class that I wrote to use with the dsl so I can pull it out of there.
The problem with loading it from a file is the existence of the instance_name variable. load_from_file generates a complaint about not knowing what instance_name is. interpolate(file()) assumes I want to use it as UserData. Using a method to generate it seems to work but I'm not sure how to inject that into the template.
Create a subcommand that:
And, if we're feeling particularly generous, let's add a few features to it:
This is essentially a migration of an internal tool at Bazaarvoice that does a very similar thing, but migrates the functionality to the Ruby DSL.
[2014-04-21 15:05:08] pavel.lishin@plishin:/gitwork/Hubs-Core
# bundle exec cfntemplate-to-ruby scripts/cloudformation/abba.autoscale.json > scripts/cloudformation/abba.autoscale.rb
[2014-04-21 15:05:24] pavel.lishin@plishin:/gitwork/Hubs-Core
# chmod u+x scripts/cloudformation/abba.autoscale.rb
[2014-04-21 15:04:33] pavel.lishin@plishin:/gitwork/Hubs-Core
# scripts/cloudformation/abba.autoscale.rb diff scripts/cloudformation/abba.autoscale.json
sh: cfn-cmd: command not found
I'm investigating this now, so hopefully there's a PR following this pretty closely.
The --capabilities parameter now requires the CAPABILITY_IAM value for cfn-create-stack and cfn-update-stack otherwise the command fails with error Malformed input-Requires capabilities : [CAPABILITY_IAM]
Although the immutability feature of parameters offers a layer of protection around Cloudformation stack updates, Amazon themselves have provided a feature called "stack policies," which allows you to specify a policy (very similar to IAM policies) regarding which resources may be updated (and how -- replaced, updated, or deleted). Let's consider supporting this feature in the Ruby DSL.
Amazon stack policy documentation: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html
Related concepts: change sets
Apologies for all of these 'issues' that not really issues, but I can't find documentation anywhere...
I have a template which validates just fine:
bundle exec ruby my-stack.rb validate --parameters "Param1=foo;Param2=bar" --stack-name my-stack --region us-east-1
However when I try to create:
bundle exec ruby auvik-cluster.rb create --parameters "Param1=foo;Param2=bar" --stack-name my-stack --region us-east-1
Failed to create stack:
My guess is I am unable to authenticate to cloudformation?
I am able to expand the template and upload it to do the cloudformation run, my question is how are credentials picked up? ENV variables, Profiles/Credentails file, Roles?
I want to improve the cli handling. In the gemfile there aren't lots of dependencies, which makes me think optparse will be preferred. Thor is a little prettier. Any feedback before I make a PR?
The failure occurs because cfn-describe-stacks returns the string "(nil)" for empty values and cfntemplate.rb is comparing a NilClass with a String:
if cfn_tags != old_cfn_tags .... end
This causes this error:
$ ruby app.rb cfn-update-stack qa-services
CloudFormation stack tags do not match and cannot be updated. You must either use the same tags or create a new stack.
< (nil)
---
$
[2014-04-21 13:23:49] pavel.lishin@plishin:~
# gem install cloudformation-ruby-dsl
ERROR: Could not find a valid gem 'cloudformation-ruby-dsl' (>= 0) in any repository
ERROR: Possible alternatives: cloud_formation, cloud_formation_template
[2014-04-21 13:42:55] pavel.lishin@plishin:~
# gem sources
*** CURRENT SOURCES ***
https://rubygems.org/
[2014-04-21 13:43:56] pavel.lishin@plishin:~
#
It would be useful to have metadata available for this cloudformation feature:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-interface.html
I dug up PR #65 and confirmed that this code:
metadata 'AWS::Cloudformation::Interface' => {
:ParameterGroups => [ {
:Label => {
:default => "App1 Settings"
},
:Parameters => [
:App1InstanceType,
:App1MinInstanceCount,
:App1DesiredInstanceCount,
:App1MaxInstanceCount
],
},
{
:Label => {
:default => "App2 Settings"
},
:Parameters => [
:App2InstanceType,
:App2MinInstanceCount,
:App2DesiredInstanceCount,
:App2MaxInstanceCount
]
} ]
}
produces this json output:
{
"Metadata": {
"AWS::Cloudformation::Interface": {
"ParameterGroups": [
{
"Label": {
"default": "App1 Settings"
},
"Parameters": [
"App1InstanceType",
"App1MinInstanceCount",
"App1DesiredInstanceCount",
"App1MaxInstanceCount"
]
},
{
"Label": {
"default": "App2 Settings"
},
"Parameters": [
"App2InstanceType",
"App2MinInstanceCount",
"App2DesiredInstanceCount",
"App2MaxInstanceCount"
]
}
]
}
}
}
It will be handy if we can do e.g:
parameter 'VpcId',
Type: 'String',
Default: get_output('VpcId', 'my_vpc_template'),
Description: 'The ID of you Virtual Private Cloud (VPC)',
Immutable: true
% ./cloudformation-ruby-script.rb expand
./cloudformation-ruby-script.rb:176:in `block in <main>': undefined method `equals' for #<TemplateDSL:0x007f8c9aa87720> (NoMethodError)
from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:260:in `instance_eval'
from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:260:in `initialize'
from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:293:in `initialize'
from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:284:in `new'
from /Users/mkeisler/git/github/cloudformation-ruby-dsl/lib/cloudformation-ruby-dsl/cfntemplate.rb:284:in `template'
from ./cloudformation-ruby-script.rb:24:in `<main>'
When using:
./cloudformation/policies.rb delete my_test_stack
The following error occurs when using 1.2.3
.../gems/aws-sdk-core-2.5.8/lib/aws-sdk-core/plugins/request_signer.rb:100:in `require_credentials': unable to sign request without credentials set (Aws::Errors::MissingCredentialsError)
...
...
from .../gems/cloudformation-ruby-dsl-1.2.3/lib/cloudformation-ruby-dsl/cfntemplate.rb:305:in `cfn'
from .../gems/cloudformation-ruby-dsl-1.2.3/lib/cloudformation-ruby-dsl/cfntemplate.rb:564:in `exec!'
This worked fine in 1.2.1
.
It does of course work if you execute this with the --profile
and --region
switches.
Its not clear from the change why the environment variables capability was removed:
# Following line can be uncommented only when Amazon will provide the stable version of this functionality.
# Aws.config[:credentials] = Aws::SharedCredentials.new(profile_name: args[:aws_profile]) unless args[:aws_profile].nil?
What was unstable? AWS have released a number of new versions of the Ruby SDK since this change was made. Without knowing the nature of the issue, I can't check this. Would be happy to provide a PR once I have some more information.
Thanks
cfn-update-stack
fails because the CSV parser splits comma separated parameter values into different fields if the parameter string is double quoted. This , I believe, happens because of the change in 6ead33c. The actual error is: cfn-update-stack may not update immutable parameter
and that's because the retrieved template parameters are never parsed beyond the parameter at which the split error occurs.
To reproduce: create a stack with description like "Stack, the" or one having a parameter which is a comma separated list and run cfn-update-stack on it
Hi,
Is there any way to set the stack name in the ruby template instead of having to use --stack-name each time?
Thanks
$ curl https://raw.githubusercontent.com/thefactory/cloudformation-zookeeper/master/zookeeper-vpc.json
$ cfntemplate-to-ruby zookeeper-vpc.json
/opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:307:in `simplify': no implicit conversion of FnCall into Array (TypeError)
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `block in simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `map'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:321:in `simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'`
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `block in simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `each'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `map'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:291:in `simplify'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:51:in `block in main'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:50:in `each'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:50:in `main'
from /opt/rubies/2.1.1/lib/ruby/gems/2.1.0/gems/cloudformation-ruby-dsl-0.4.5/bin/cfntemplate-to-ruby:327:in `<top (required)>'
from /opt/boxen/rbenv/versions/2.1.1/bin/cfntemplate-to-ruby:23:in `load'
from /opt/boxen/rbenv/versions/2.1.1/bin/cfntemplate-to-ruby:23:in `<main>'
Ok, so this is a weird one.
I've been testing out this gem for a while and I think my company is going to make use of it. However I can't seem to create stacks.
When calling ./my_stack.rb create my_stack --parameters "blah"
I receive the cryptic response of "Failed to create stack:" with no message. Looking into this with binding.pry
I find that the exception being caught is on line 302 of lib/cloudformation-ruby-dsl/cfntemplate.rb
. The exception class is Aws::CloudFormation::Errors::ValidationError
which is not a class that is documented (I think it's autogenerated by the aws-sdk service layer). This exception does not have a message, code, or really any useful information.
Things that I have tried with varying results:
I have attached the ruby template I wrote, and I would love any and all help in fixing this issue.
Thanks!
--Katz
Here's the exact commands I'm running:
./database.cfn.rb create test-stack --parameters "ParamDBRole=read-write;ParamDBSizePostgres=50;ParamDBSizeCyanAudit=100;ParamDBIops=100;ParamCustomerLabel=testing"
aws cloudformation create-stack --stack-name test-stack --parameters ParameterKey=ParamDBRole,ParameterValue=read-write ParameterKey=ParamDBSizePostgres,ParameterValue=50 ParameterKey=ParamDBSizeCyanAudit,ParameterValue=100 ParameterKey=ParamDBIops,ParameterValue=100 ParameterKey=ParamCustomerLabel,ParameterValue=testing --template-body file://test.json
Note: test.json is just ./database.cfn.rb expand > test.json
Not sure this is a bug but I cannot find the "right" way to do this. I have defined a variable in the root RB template and it is not in scope when calling the load_from_file method. I used super global (@@) variables to get around this but this doesn't seem quite right.
Suggestions?
Any additional options don't appear to be forwarded on to cloudformation during stack creation. We need to be able to pass "disable-rollback" for debugging bootstrapping issues.
Apply a template (ERB) to the bash script erb before embedding it. Add a context file as a parameter that the bash template uses it could have have things like an s3 bucket name/key and other server destinations etc.
I'm having a problem loading the CFN gem. The only way I can use it is if I run it from a working directory which contains the cloudformation-ruby-dsl git repo. Otherwise I get the above error.
Suppose assumed roles are used as follows:
[default]
aws_access_key_id = foo
aws_secret_access_key = bar
[test]
role_arn = arn:aws:iam::123456789012:role/group/username
source_profile = default
With the provided access configuration in either ~/.aws/credentials or ~/.aws/config it not possible to create a stack via cloudformation-ruby-dsl v1.2.1, aws-sdk-ruby 2.4.2 and ruby 2.3.0:
$ ./stack.rb create test-stack --profile test --region us-east-1 --parameters "\
Param1=1-20;\
Param2=teamname"
/home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:100:in `require_credentials': unable to sign request without credentials set (Aws::Errors::MissingCredentialsError)
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:90:in `sign_authenticated_requests'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/request_signer.rb:83:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/retry_errors.rb:87:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/query/handler.rb:27:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/endpoint.rb:41:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/plugins/response_target.rb:21:in `call'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/request.rb:70:in `send_request'
from /home/user/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.4.2/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
from /home/user/vendor/bundle/ruby/2.3.0/gems/cloudformation-ruby-dsl-1.2.1/lib/cloudformation-ruby-dsl/cfntemplate.rb:297:in `cfn'
from /home/user/vendor/bundle/ruby/2.3.0/gems/cloudformation-ruby-dsl-1.2.1/lib/cloudformation-ruby-dsl/cfntemplate.rb:556:in `exec!'
from ./stack.rb:291:in `<main>'
After updating the cloudformation-ruby-dsl gem to ~> 0.5.0 the cfn-*-stack commands are not found.
It started working again after brew install aws-cfn-tools
.
Currently, we do a system diff to compare old and new templates. This is not particularly informative, especially as one is pretty formatted, and the other is not. Replace this with something that is more structurally aware.
Hi @jonaf,
I have a some questions about your intended use of build_nested_map before I propose a change:
logical_id region zone env physical_id
GlobalSubnetA eu-west-1 a prod subnet-abc123
GlobalSubnetB eu-west-1 b prod subnet-def456
GlobalSubnetC eu-west-1 c prod subnet-ghi789
subnets = Table.load 'maps/subnet-ids.txt'
mapping 'AZsByRegion',
subnets.get_multimap({}, :region, :env, :region, :zone)
"AZsByRegion": {
"eu-west-1": {
"prod": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
]
},
Thoughts?
Thank you,
Cameron
We're accessing values in CSV data using specific field positions, which are susceptible to failure if the field position changes. It is better to refer to the field based on the header name, for example:
Before:
CSV.parse_line(old_stack_description)[6]
After:
CSV.parse_line(old_stack_description, :headers => true)['DESCRIPTION']
I'm not sure how to use the tag capabilities in the DSL...
By the example below I was hoping to see all the resources created to have both tags Foo and Bar...
Is this incorrect?
`
tag :Foo => ref('Foo')
tag :Bar => ref('Bar')
resource "vpc",
:Type => 'AWS::EC2::VPC',
:Properties => {
:CidrBlock => '10.0.0.0/16',
:InstanceTenancy => 'default',
:EnableDnsSupport => 'true',
:EnableDnsHostnames => 'true',
}`
Github supports a pull request template feature. We should add one with some kind of checklist, such as adding tests, etc.
We need --help
! Our documentation doesn't do a good job of providing usage info, and in general, we should be able to add --help
to a command and get some usage instructions. This should alleviate some of the confusion / issues we have seen, such as #67 .
cloudformation-ruby becomes unfunctional the moment your expanded template hits 50KB because of the maximum payload of a CLI based body. Uploading to S3 and passing that as the template-url would increase the size by 10x.
A week ago Amazon added an ability to update Cloudformation tags: https://aws.amazon.com/about-aws/whats-new/2016/02/aws-cloudformation-adds-support-for-amazon-vpc-nat-gateway-amazon-ec2-container-registry-and-more/
It would be nice to have this feature supported by cloudformation-ruby-dsl.
Currently, when including a map file, we do something like:
map = Table.load 'my-map.txt'
This is, of course, relative to Cloudformation template file. Therefore, if your directory structure looks like this:
.
└── cloudformation
├── includes
│ └── my-map.yaml
└── my-template.rb
In order for my-template.rb
to work, you must first change your working directory to cloudformation
:
cd cloudformation
./my-template.rb cfn-create-stack ...
The Table class should search paths relative to the CloudFormation Ruby DSL template file in addition to the present working directory. This way, my-template.rb
may be called from anywhere and still find and read its map files.
As a temporary workaround, as demonstrated by @epelletier, the map path can be calculated when the script is run:
map = Table.load File.dirname(__FILE__) + '/includes/my-map.yaml'
Ideally, this shouldn't be necessary, as setting up multiple maps requires unnecessary code copy. I think this should "just work," so that a Cloudformation template doesn't suddenly break because a map was included in the template.
I realize this is such an edge case ... but I thought I'd bring it up in case anybody else runs across this. I'd fix it if I could, but it looks like it's bigger change that I'm comfortable with. I'm just happy to have this tool. 😄
We use some tags that have spaces in the value. For example:
tag Application: 'My App'
This might be a bad idea for other reasons, but at the moment, it's what we do and we've got a lot of existing tags in this format.
It looks like system
calls escape spaces for the command line according to this and I'm guessing that's what's fouling up the cfn-create stack
call. It might cause other problems with the diff
and cfn-update-stack
too even if the cfn-create-stack
case could be worked around.
It didn't look like there's an obvious way to fix this without a lot of testing. So presuming that the spaces are just going to not be supported for now, I was wondering if you'd accept a PR that checks the tag values for spaces and causes an error that explains the problem and how to workaround (i.e. remove the spaces)? The reason I ask is the current error (using a tag example like the 'My App' above) isn't super obvious about the actual problem:
cfn-create-stack: Malformed input-No value separator = found in App
Thanks. And again, great tool!
There's a bug in the mapping method which causes the produced JSON to be malformed when using external map files. For example, when using this external map file:
=> maps.json
{
"Mappings": {
"JsonExampleMap": {
"foo": "bar"
}
}
}
with
mapping 'JsonExampleMap', 'maps/map.json'
The produced Cloudformation stack JSON contains an incorrect duplicate map:
"JsonExampleMap": {
"JsonExampleMap": {
"foo": "bar"
}
},
The expected resulting map should have been (without duplication):
"JsonExampleMap": {
"foo": "bar"
}
This problem affects all external maps in the following formats: JSON, Ruby and YAML
Hey,
Awesome utility, thanks for writing it. I've found that the join function doesn't seem to be evaluated/respected when used in certain cases:
Example:
Ruby template (see SecurityGroups)
resource 'Name',
:Type => 'AWS::AutoScaling::LaunchConfiguration',
:Metadata => {
:'AWS::CloudFormation::Init' => {
...,
...
},
:Properties => {
:IamInstanceProfile => 'value',
:ImageId => ref('ImageId'),
:InstanceType => ref('InstanceType'),
:KeyName => ref('KeyName'),
:SecurityGroups => [ join(',', ref('SecurityGroups')) ]
}
Expanded template (see SecurityGroups)
...
...
...
"Properties": {
"IamInstanceProfile": "value",
"ImageId": {
"Ref": "ImageId"
},
"InstanceType": {
"Ref": "InstanceType"
},
"KeyName": {
"Ref": "KeyName"
},
"SecurityGroups": [
{
"Ref": "SecurityGroups"
}
]
}
What I would expect to see
...
...
"Properties": {
"IamInstanceProfile": "value",
"ImageId": {
"Ref": "ImageId"
},
"InstanceType": {
"Ref": "InstanceType"
},
"KeyName": {
"Ref": "KeyName"
},
"SecurityGroups": [
{
"Fn::Join": [
",",
[
{
"Ref": "SecurityGroups"
}
]
]
}
]
}
The inverse is fine however (converting JSON to the Ruby DSL).
$ cfntemplate-to-ruby template.json
# Would output
...
:SecurityGroups => [ join(',', ref('SecurityGroups')) ],
...
Is this on the todo list or was it excluded on purpose?
I'm not sure if I'm going to explain this well, but I'll give it a shot.
I use cloudformation-ruby-dsl in a slightly different way. I "add features" to it by the fact that it is ruby script and can do extra things like "lookup the latest AMI ID" or "read a YAML file for more information". As such, I pass in command-line arguments and parse them using a rubygem (mixlib-cli in my case, but it could be anything.)
As a side note, prior to aws-sdk I wasn't able to do this because all of ARGV got passed directly to the old cfn tools (at which case, it would fail because of it not recognizing CLI options.) At the moment, I can ALMOST get it to work. The quirk is that my (the "outer") parsing of CLI options has to be able to handle any options that the cloudformation-ruby-dsl requires (i.e. --stack-name has to be handled by my options parsing ... but this is manageable) ... while using any CLI options that cloudformation-ruby-dsl doesn't support can cause problems (so far, the biggest problem I've found is that diff action seems to pass in all of ARGV that the cloudformation-ruby-dsl doesn't recognize to the Diffy object as options ... which causes Diffy to not work.)
I guess at the heart of it, I'd want to be able to use the template DSL more like a library ... but, of course, let others use it as a pure standalone script too.
Is there a way to do this that I am just missing?
This is Ruby. While "cloudformation-ruby-dsl" is informative, it lacks a certain panache that all the other cool Ruby library names have.
I have no suggestions for any specific "better name" but I'd like to see such a great tool have an equally great name.
Also, cloudformation-ruby-dsl has saved me countless hours and has made it much easier to manage and grok otherwise complicated CloudFormation JSON templates. Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.