Giter VIP home page Giter VIP logo

sceptre's Introduction

Sceptre

GitHub Actions Workflow Status Docker Image Version (latest semver) PyPI PyPI - Status PyPI - Python Version PyPI - Downloads License

About

Sceptre is a tool to drive AWS CloudFormation. It automates the mundane, repetitive and error-prone tasks, enabling you to concentrate on building better infrastructure.

Features

  • Code reuse by separating a Stack's template and its configuration
  • Support for templates written in JSON, YAML, Jinja2 or Python DSLs such as Troposphere
  • Dependency resolution by passing of Stack outputs to parameters of dependent Stacks
  • Stack Group support by bundling related Stacks into logical groups (e.g. dev and prod)
  • Stack Group-level commands, such as creating multiple Stacks with a single command
  • Fast, highly parallelised builds
  • Built in support for working with Stacks in multiple AWS accounts and regions
  • Infrastructure visibility with meta-operations such as Stack querying protection
  • Support for inserting dynamic values in templates via customisable Resolvers
  • Support for running arbitrary code as Hooks before/after Stack builds

Benefits

  • Utilises cloud-native Infrastructure as Code engines (CloudFormation)
  • You do not need to manage state
  • Simple templates using popular templating syntax - Yaml & Jinja
  • Powerful flexibility using a mature programming language - Python
  • Easy to integrate as part of a CI/CD pipeline by using Hooks
  • Simple CLI and API
  • Unopinionated - Sceptre does not force a specific project structure

Install

Using pip

$ pip install sceptre

More information on installing sceptre can be found in our Installation Guide

Using Docker Image

View our Docker repository. Images available from version 2.0.0 onward.

To use our Docker image follow these instructions:

  1. Pull the image docker pull sceptreorg/sceptre:[SCEPTRE_VERSION_NUMBER] e.g. docker pull sceptreorg/sceptre:2.5.0. Leave out the version number if you wish to run latest or run docker pull sceptreorg/sceptre:latest.

  2. Run the image. You will need to mount the working directory where your project resides to a directory called project. You will also need to mount a volume with your AWS config to your docker container. E.g.

docker run -v $(pwd):/project -v /Users/me/.aws/:/root/.aws/:ro sceptreorg/sceptre:latest --help

If you want to use a custom ENTRYPOINT simply amend the Docker command:

docker run -ti --entrypoint='' sceptreorg/sceptre:latest sh

The above command will enter you into the shell of the Docker container where you can execute sceptre commands - useful for development.

If you have any other environment variables in your non-docker shell you will need to pass these in on the Docker CLI using the -e flag. See Docker documentation on how to achieve this.

Example

Sceptre organises Stacks into "Stack Groups". Each Stack is represented by a YAML configuration file stored in a directory which represents the Stack Group. Here, we have two Stacks, vpc and subnets, in a Stack Group named dev:

$ tree
.
├── config
│   └── dev
│        ├── config.yaml
│        ├── subnets.yaml
│        └── vpc.yaml
└── templates
    ├── subnets.py
    └── vpc.py

We can create a Stack with the create command. This vpc Stack contains a VPC.

$ sceptre create dev/vpc.yaml

dev/vpc - Creating stack dev/vpc
VirtualPrivateCloud AWS::EC2::VPC CREATE_IN_PROGRESS
dev/vpc VirtualPrivateCloud AWS::EC2::VPC CREATE_COMPLETE
dev/vpc sceptre-demo-dev-vpc AWS::CloudFormation::Stack CREATE_COMPLETE

The subnets Stack contains a subnet which must be created in the VPC. To do this, we need to pass the VPC ID, which is exposed as a Stack output of the vpc Stack, to a parameter of the subnets Stack. Sceptre automatically resolves this dependency for us.

$ sceptre create dev/subnets.yaml
dev/subnets - Creating stack
dev/subnets Subnet AWS::EC2::Subnet CREATE_IN_PROGRESS
dev/subnets Subnet AWS::EC2::Subnet CREATE_COMPLETE
dev/subnets sceptre-demo-dev-subnets AWS::CloudFormation::Stack CREATE_COMPLETE

Sceptre implements meta-operations, which allow us to find out information about our Stacks:

$ sceptre list resources dev/subnets.yaml

- LogicalResourceId: Subnet
  PhysicalResourceId: subnet-445e6e32
  dev/vpc:
- LogicalResourceId: VirtualPrivateCloud
  PhysicalResourceId: vpc-c4715da0

Sceptre provides Stack Group level commands. This one deletes the whole dev Stack Group. The subnet exists within the vpc, so it must be deleted first. Sceptre handles this automatically:

$ sceptre delete dev

Deleting stack
dev/subnets Subnet AWS::EC2::Subnet DELETE_IN_PROGRESS
dev/subnets - Stack deleted
dev/vpc Deleting stack
dev/vpc VirtualPrivateCloud AWS::EC2::VPC DELETE_IN_PROGRESS
dev/vpc - Stack deleted

Note: Deleting Stacks will only delete a given Stack, or the Stacks that are directly in a given StackGroup. By default Stack dependencies that are external to the StackGroup are not deleted.

Sceptre can also handle cross Stack Group dependencies, take the following example project:

$ tree
.
├── config
│   ├── dev
│   │   ├── network
│   │   │   └── vpc.yaml
│   │   ├── users
│   │   │   └── iam.yaml
│   │   ├── compute
│   │   │   └── ec2.yaml
│   │   └── config.yaml
│   └── staging
│       └── eu
│           ├── config.yaml
│           └── stack.yaml
├── hooks
│   └── stack.py
├── templates
│   ├── network.json
│   ├── iam.json
│   ├── ec2.json
│   └── stack.json
└── vars
    ├── dev.yaml
    └── staging.yaml

In this project staging/eu/stack.yaml has a dependency on the output of dev/users/iam.yaml. If you wanted to create the Stack staging/eu/stack.yaml, Sceptre will resolve all of it's dependencies, including dev/users/iam.yaml, before attempting to create the Stack.

Usage

Sceptre can be used from the CLI, or imported as a Python package.

CLI

Usage: sceptre [OPTIONS] COMMAND [ARGS]...

  Sceptre is a tool to manage your cloud native infrastructure deployments.

Options:
  --version                  Show the version and exit.
  --debug                    Turn on debug logging.
  --dir TEXT                 Specify sceptre directory.
  --output [text|yaml|json]  The formatting style for command output.
  --no-colour                Turn off output colouring.
  --var TEXT                 A variable to replace the value of an item in
                             config file.
  --var-file FILENAME        A YAML file of variables to replace the values
                             of items in config files.
  --ignore-dependencies      Ignore dependencies when executing command.
  --merge-vars               Merge variables from successive --vars and var
                             files.
  --help                     Show this message and exit.

Commands:
  create         Creates a stack or a change set.
  delete         Deletes a stack or a change set.
  describe       Commands for describing attributes of stacks.
  estimate-cost  Estimates the cost of the template.
  execute        Executes a Change Set.
  generate       Prints the template.
  launch         Launch a Stack or StackGroup.
  list           Commands for listing attributes of stacks.
  new            Commands for initialising Sceptre projects.
  set-policy     Sets Stack policy.
  status         Print status of stack or stack_group.
  update         Update a stack.
  validate       Validates the template.

Python

Using Sceptre as a Python module is very straightforward. You need to create a SceptreContext, which tells Sceptre where your project path is and which path you want to execute on, we call this the "command path".

After you have created a SceptreContext you need to pass this into a SceptrePlan. On instantiation the SceptrePlan will handle all the required steps to make sure the action you wish to take on the command path are resolved.

After you have instantiated a SceptrePlan you can access all the actions you can take on a Stack, such as validate(), launch(), list() and delete().

from sceptre.context import SceptreContext
from sceptre.plan.plan import SceptrePlan

context = SceptreContext("/path/to/project", "command_path")
plan = SceptrePlan(context)
plan.launch()

Full API reference documentation can be found in the Documentation

Tutorial and Documentation

Communication

Sceptre community discussions happen in the #sceptre chanel in the og-aws Slack. To join click on http://slackhatesthe.cloud/ to create an account and join the #sceptre channel.

Contributing

See our Contributing Guide

Sponsors

Sage Bionetworks

GoDaddy

Cloudreach

sceptre's People

Contributors

1oglop1 avatar alexharv074 avatar b-t-g avatar cbosss avatar connec avatar cornerman avatar craighurley avatar dboitnot avatar giuliocalzolari avatar iainelder avatar jamesroutley avatar jfalkenstein avatar jfgiroux avatar lucasvieirasilva avatar lukeplausin avatar mrowlingfox avatar nabeelamjad avatar ngfgrant avatar nunogrl avatar omer-azmon avatar pre-commit-ci[bot] avatar seanrankine avatar stig avatar tarkatronic avatar theseanything avatar uhinze avatar ybeumer avatar ykhalyavin avatar zaro0508 avatar zxiiro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sceptre's Issues

Reference external templates from another repository.

It would be nice to have the ability to reference a centralised repository containing Raw or Python (Troposphere) CloudFormation templates. This would allow multiple Sceptre projects to use the same templates stored in a single repository.

Something like in template: vpc.py in Stack config and template_repo: github.com/person/cloudfromation-templates

Wait for for stacks to be created when resolving

My terminal crashed mid way through launching an environment, so I ran the command to launch the environment again, however hit an resolver error. The reason for this was that stacks were still in a "create_in_progress" state and sceptre didn't wait for "create_complete" to resolve outputs for a dependee template.

[seanrankine:.../Project/project-infrastructure]$ sceptre launch-env prod/cybermation
[2017-02-10 14:55:28] - sceptre.stack - prod/cybermation/security-groups - Launching stack
[2017-02-10 14:55:28] - sceptre.stack - prod/cybermation/iam-role - Launching stack
[2017-02-10 14:55:33] - sceptre.stack - prod/cybermation/iam-role - Stack is in the CREATE_COMPLETE state
[2017-02-10 14:55:33] - sceptre.stack - prod/cybermation/iam-role - Updating stack
[2017-02-10 14:55:33] - sceptre.stack - prod/cybermation/security-groups - Stack is in the CREATE_COMPLETE state
[2017-02-10 14:55:33] - sceptre.stack - prod/cybermation/security-groups - Updating stack
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/security-groups - No updates to perform.
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/iam-role - No updates to perform.
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/iam-instance-profile - Launching stack
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/iam-instance-profile - Stack is in the CREATE_IN_PROGRESS state
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/iam-instance-profile - Stack action is already in progress state and cannot be updated
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/ec2-instances - Launching stack
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/ec2-instances - Stack is in the PENDING state
[2017-02-10 14:55:34] - sceptre.stack - prod/cybermation/ec2-instances - Creating stack
[2017-02-10 14:55:35] - sceptre.environment - Stack prod/cybermation/ec2-instances failed to launch
Traceback (most recent call last):
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/environment.py", line 244, in _manage_stack_build
    status = getattr(stack, command)()
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/helpers.py", line 79, in decorated
    return func(self, *args, **kwargs)
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/stack.py", line 273, in launch
    status = self.create()
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/hooks/__init__.py", line 70, in decorated
    response = func(self, *args, **kwargs)
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/helpers.py", line 79, in decorated
    return func(self, *args, **kwargs)
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/stack.py", line 193, in create
    "Parameters": self._format_parameters(self.parameters),
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/resolvers/__init__.py", line 72, in __get__
    return self.resolve_values(getattr(instance, self.name))
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/resolvers/__init__.py", line 91, in resolve_values
    attr[key] = value.resolve()
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/resolvers/stack_output.py", line 119, in resolve
    return self._get_output_value(stack_name, self.output_key)
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/resolvers/stack_output.py", line 34, in _get_output_value
    outputs = self._get_stack_outputs(stack_name)
  File "/Users/seanrankine/.virtualenvs/project/lib/python2.7/site-packages/sceptre/resolvers/stack_output.py", line 71, in _get_stack_outputs
    outputs = response["Stacks"][0]["Outputs"]
KeyError: 'Outputs'

Remove dev docs

@seanrankine am I correct in thinking that the dev docs are no longer used? We don't have a develop branch any more. If so, we should remove them.

CLI export AWS_PROFILE not working.

When using the AWS CLI in the getting started tutorial it doesn't mention how to set your AWS CLI details. (Whilst this isn't inherently necessary, I found the following bug)

export AWS_PROFILE=default doesn't seem to allow me to run sceptre create-stack dev vpc however, export AWS_DEFAULT_PROFILE=default works.

Address 'profile' in the documents

I'd like to play with sceptre, but currently I'm unsure if which of my AWS profiles it is going to use from .aws/credentials

The only mention I see of profile is that it has been removed from CLI options, so I am unsure of what to do.

Thanks!

Question about using Environment underscore methods, e.g. _get_available_stacks()

Sorry, not necessarily an issue, but I didn't see another medium to ask questions (IRC, Slack, ...)

When using Sceptre as a Python Module, the docs state there should be an attribute such as:

env.available_stacks

If I attempt that I get:

Traceback (most recent call last):
  File "/Users/sam/Projects/sceptre-example/test.py", line 11, in <module>
    env.available_stacks
AttributeError: 'Environment' object has no attribute 'available_stacks'

I do see underscore methods and one of those being _get_available_stacks(). If I use that (env._get_available_stacks()), I get the expected results.

The question is, should I be using the underscore method of Environment directly? Due to the underscore, I was presuming you are indicating they are private methods.

thanks in advance,
Sam

Stack with id does not exist

Hi,

I have this directory structure:

config/
├── infrastructure
│   ├── config.yaml
│   ├── eni.yaml
│   ├── infrastructure.yaml
│   └── security.yaml

and eni.yaml file has this content:

template_path: templates/infrastructure/enis.json
stack_name: eni
parameters:
  ManagementSecurityGroup: "{{ var.management_security_group }}"
  SecurityGroup: !stack_output security::InstancesSecurityGroup
  Subnets: "{{ var.subnets|join(', ') }}"
  Enis:  "{{ var.enis|join(', ') }}"

why have I this exception?
`stack with id random-infrastructure-security does not exist``

Improve project_variables resolver docs

In the example given, the resolver passes a dictionary to a parameter, which is invalid.

We should also mention that the a use case of this resolver is for storing tags for multiple stacks in one place.

Remove python module support

Sceptre officially supports use via the CLI and as a Python module. Use via the CLI is far more common (I'm not sure if anyone uses Sceptre as a Python module). We guarantee that both our CLI and public Python APIs won't change. Guaranteeing the Python API leads to two problems:

  • It makes it difficult to refactor the inner workings of Sceptre
  • It makes our versioning unusual, as we are effectively versioning two independent APIs with a single version number. If we commit a breaking change to Sceptre's Python internals, we must bump our major version number, even though users of Sceptre as a CLI tool (most of our users) would see no change.

This would be a relatively superficial change. Our Python function and classes would still be public, and people could still use Sceptre as a module, but we wouldn't officially support it or guarantee its API.

Implementation

Remove references to use as a Python module from the documentation.

Pros

  • Reduces our public API
  • Makes it easier to develop Sceptre as we have fewer contracts to uphold
  • Makes our versioning less strange

Cons

  • Discontinued support for Python module

Include `scope` in output for update-stack-cs

Would like to have more details about what is being changed when executing update-stack-cs. Resources in change-sets have a "scope" attribute which would be useful to include in the output when sceptre describes a change set before executing it.

Reference to sceptre-tools and bitbucket repo in docs.

Under the "Configuring an IAM role for Sceptre" in the 'advance patterns" section of the documentation there are still references to the old bitbucket repository. There is also reference to sceptre-tools repository which is not publicly available.

Azure support

Support to launch ARM Templates. Provide multi-cloud support for projects.

Initially looking to add support for:
create-stack
update-stack
delete-stack

With templates being uploaded to Azure Blob storage.

Investigate automatic documentation of CLI in Sphinx

Our Python API is automatically documented in our online documentation using Sphinx. Our CLI API is currently not automatically documented. We should investigate how difficult it would be to add automatic Sphinx docs for our CLI.

Add validate-env command

I would like to test that all stacks in an environment generate templates properly and return a status code of 0 (no error).

If Sceptre user data has stack resolvers, could we some how use STS to avoid MFA for every stack verification.

Could we possibly have generate-env-template?

Blank Project Code - fully configurable stack name.

If I make project_code: ""
I get :

An error occurred (ValidationError) when calling the UpdateStack operation: 1 validation error detected: Value '-folder-template' at 'stackName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z][-a-zA-Z0-9]*|arn:[-a-zA-Z0-9:/._+]*

Which is a nicely handled error.

However if I leave it blank (rather than an empty string "")

[2017-02-07 15:46:50] - sceptre.config - Reading in 'config.yaml' files...
[2017-02-07 15:46:50] - sceptre.config - Config: {'region': 'eu-west-1', 'dependencies': [], 'project_code': None}

Then you get a poorly handled error

[2017-02-07 15:46:50] - sceptre.stack - folder/template- Hooks: {}
[2017-02-07 15:46:50] - sceptre.stack - folder/template - Updating stack
Traceback (most recent call last):
  File "/usr/local/bin/sceptre", line 9, in <module>
    load_entry_point('sceptre==1.0.0', 'console_scripts', 'sceptre')()
  File "/Library/Python/2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Library/Python/2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Library/Python/2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Library/Python/2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Library/Python/2.7/site-packages/sceptre/cli.py", line 93, in decorated
    return func(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/sceptre/cli.py", line 273, in update_stack
    response = env.stacks[stack].update()
  File "/Library/Python/2.7/site-packages/sceptre/helpers.py", line 79, in decorated
    return func(self, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/sceptre/hooks/__init__.py", line 70, in decorated
    response = func(self, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/sceptre/stack.py", line 226, in update
    "StackName": self.external_name,
  File "/Library/Python/2.7/site-packages/sceptre/stack.py", line 177, in external_name
    get_external_stack_name(self.project, self.name)
  File "/Library/Python/2.7/site-packages/sceptre/helpers.py", line 183, in get_external_stack_name
    stack_name.replace("/", "-")
TypeError: sequence item 0: expected string, NoneType found

👍 @theseanything asked me to add this.

Can I have a fully configurable stack name? Why are there restrictions?

Improve CLI Documentation

After chatting to @theseanything

Please can you improve the documentation so it explains that awesome change set thing because that's awesome if it creates it for you. In general can you improve the CLI documentation to show off the awesome features.

update-stack-cs - basically creates a change-set, tells you what's different in the stack, then if you say yes will update the stack...
it's a kind of safety net

Show CloudFormation Events for a stack.

Could there be a function to describe all the previous CloudFormation Events for a given stack?
It's sometimes useful to see this before making any changes to a stack to see whether someone else has made changes to the stack since you last interacted with it. Not having to go to the console to see this would be great!

Simplify CLI

Full change list:

Current:                  Proposed:
continue-update-rollback  [unchanged]
create-change-set         [unchanged]
create-stack              create
delete-change-set         [unchanged]
delete-env                delete -r
delete-stack              delete
describe-change-set       [unchanged]
describe-env              [unchanged] (maybe it would be working adding a describe-stack command for symmetry)
describe-env-resources    describe-resources -r
describe-stack-outputs    describe-outputs
describe-stack-resources  describe-resources
execute-change-set        [unchanged]
generate-template         [unchanged]
get-stack-policy          get-policy
launch-env                launch -r
launch-stack              launch
list-change-sets          [unchanged]
lock-stack                lock
set-stack-policy          set-policy
unlock-stack              unlock
update-stack              update
update-stack-cs           update-cs (maybe implement this as a flag: update -cs)
validate-template         [unchanged]

Implementation

Rename CLI commands.

Pros

  • The -r/--recursive syntax is more in keeping with Unix standards (e.g. cp, mv, grep)
  • Commands are less verbose

Cons

  • The original reason for having separate commands for building stacks and environments was to avoid users accidentally deleting whole environments. This would be mitigated by proposal 01.

Incorrectly formatted exception message

We raise:

raise UnsupportedTemplateFileTypeError(
                "Template has file extension %s. Only .py, .yaml, and .json "
                "are supported.",
                os.path.splitext(self.path)[1]
            )

However this formatting is incorrect, and renders as: 'Template has file extension %s. Only .py, .yaml, and .json are supported.', '.yml'.

Switch to:

raise UnsupportedTemplateFileTypeError(
                "Template has file extension {0}. Only .py, .yaml, and .json "
                "are supported.".format(os.path.splitext(self.path)[1])
            )

https://github.com/cloudreach/sceptre/blob/master/sceptre/template.py#L250

launch-env with order

Hi,

We have some templates that dependes to others, example:
config
dev
config.yaml
instance.yml
eni.yaml

We need launch eni before instance, how can I guarantee that?

Address stacks and envs by path

Implementation

We currently build stacks and environments with the syntax:

sceptre <action>-stack env stack
sceptre <action>-env env

This proposal would change the addressing to:

sceptre <action> path/to/stack.yaml
sceptre <action> --recursive path/to/env/

Pros

  • By supplying file paths, we get native tab completion (only tested on zsh on macOS)

Cons

Check return code for hooks

When a hook is run on before_create or before_update, sceptre should check for a return code and abort the stack create or update and abort if the result is non-zero.

Currently even if a hook fails the stack operation proceeds.

How can I modify a var from the same file

I Have this var:
BrokersInstance: !stack_output_external security::InstanceProfile

An now I have to do:
BrokersInstanceProfile: "{{ BrokersInstance.split('/')[-1] }}"

How I can?

README.rst is not rendering on PyPI

The README.rst is not rendering on the PyPi site. The PyPi can be quite unforgiving on small errors or warnings when rendering. Running python setup.py check --restructuredtext -s shows a few warnings that could be the cause:

running check
warning: check: Cannot analyze code. Pygments package not found. (line 130)

warning: check: Duplicate explicit target name: "documentation".

error: Please correct your package.

Remove cascading stack config

04 Remove cascading config for stack config files

Stack config files support cascading config. This is mainly used for reducing repeated config when launching the same stack in multiple environments, such as launching a stack which enables CloudTrail in each region.

The following problems stem from cascading stack config:

  1. It makes it impossible to launch stacks which are contained in an environment which also contains a sub environment.

Given the following sceptre project:

.
└── config
    └── dev
        ├── ew1
        │   └── vpc.yaml
        └── vpc.yaml

We have no way of knowing whether dev/vpc.yaml is a stack in its own right, or if it's an incomplete stack that dev/ew1/vpc.yaml completes. When we run launch-env, we have to exclude dev/vpc.yaml.
2. Using cascaded stack config can lead to complex environments with lots of empty files:

.
└── config
    └── dev
        ├── ew1
        │   └── cloud_trail.yaml  # empty file
        ├── ew2
        │   └── cloud_trail.yaml  # empty file
        ├── ...
        └── cloud_trail.yaml

If there are lots of cascaded stack config files, it can be difficult to reason where config items are set.
3. Cascading config for stacks is poorly implemented, and a better implementation would introduce complexity.

Stack config items, such as sceptre_user_data can be nested. Cascaded config currently replaces the whole item:

# dev/cloud_trail.yaml
...
sceptre_user_data:  # All of these items are overwritten, even though users may think that only repeated_param should be overwritten
  repeated_param: old_value
  unique_param: value

---

# dev/ew1/cloud_trail.yaml
...
sceptre_user_data:
  repeated_param: new_value

I think the benefits of cascaded stack config are outweighed by the negatives.

Implementation

This is probably easiest implemented by creating a new stack config object as stack and environment config will be sufficiently different. The existing config object already has two initialisers..

Pros

  • Fixes problems listed above
  • Reduces Sceptre complexity

Cons

  • Removes a feature

Tutorial has a mistake in it.

The tutorial says to point to templates/vpc.yaml but the template is templates/vpc.json

`vpc.yaml
Add the following config to vpc.yaml:

template_path: templates/vpc.yaml
parameters:
CidrBlock: 10.0.0.0/16
template_path specifies the relative path to the CloudFormation or Troposphere template to use to launch the stack.`

Can't figure out how project_variables resolver works from documentation

I've followed the Get Started section of the documentation, which worked well enough (ignoring the fact that it asks you to create a .json template when the stack config sample refers to a .yaml template). On my own, I've tried to extend it a bit to use the project_variables Resolver, but I can't get it to work with the way I'm interpreting the documentation.

Changes I've made to the sample code from the Get Started page:

Created config/vars.yaml:

$ touch config/vars.yaml
---
dev:
  vpc:
    CidrBlock: 10.0.0.0/16

Updated config/dev/vpc.yaml:

template_path: templates/vpc.yaml
parameters:
  CidrBlock: !project_variables config/vars.yaml

Launching the environment:

sceptre --debug launch-env dev

Error I'm seeing:

[2017-02-17 18:26:42] - sceptre.environment - Stack dev/vpc failed to launch
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\sceptre\environment.py", line 244, in _manage_stack_build
    status = getattr(stack, command)()
  File "c:\python27\lib\site-packages\sceptre\helpers.py", line 79, in decorated
    return func(self, *args, **kwargs)
  File "c:\python27\lib\site-packages\sceptre\stack.py", line 273, in launch
    status = self.create()
  File "c:\python27\lib\site-packages\sceptre\hooks\__init__.py", line 70, in decorated
    response = func(self, *args, **kwargs)
  File "c:\python27\lib\site-packages\sceptre\helpers.py", line 79, in decorated
    return func(self, *args, **kwargs)
  File "c:\python27\lib\site-packages\sceptre\stack.py", line 205, in create
    kwargs=create_stack_kwargs
  File "c:\python27\lib\site-packages\sceptre\helpers.py", line 44, in decorated
    return func(*args, **kwargs)
  File "c:\python27\lib\site-packages\sceptre\connection_manager.py", line 158, in call
    return getattr(client, command)(**kwargs)
  File "c:\python27\lib\site-packages\botocore\client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "c:\python27\lib\site-packages\botocore\client.py", line 513, in _make_api_call
    api_params, operation_model, context=request_context)
  File "c:\python27\lib\site-packages\botocore\client.py", line 566, in _convert_to_request_dict
    api_params, operation_model)
  File "c:\python27\lib\site-packages\botocore\validate.py", line 270, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
ParamValidationError: Parameter validation failed:
Invalid type for parameter Parameters[0].ParameterValue, value: {'CidrBlock': '10.0.0.0/16'}, type: <type 'dict'>, valid types: <type 'basestring'>

It looks like the CidrBlock CFN parameter is being passed the full dict {'CidrBlock': '10.0.0.0/16'} instead of just the value 10.0.0.0/16. I've tried formatting the project_variables and the associated var file a few different ways but this is as close as I've got. Any ideas? Thanks!

Allow .yml as well as .yaml

Could you allow using .yml as well as .yaml for the configuration files? I'd say they are equally common and they are both valid. .yml files are simply ignored at the moment.

Add confirmation step when deleting

Sceptre should ask for user confirmation when deleting stacks or environments. A CLI flag --yes/-y will be added, allowing scriptable deletion of resources. This confirmation will only happen on the CLI, not when delete commands are run from

Implementation

  • Add a confirmation prompt with Click to the function sceptre.cli.delete_stack.
  • Add a Click option -y, which skips the confirmation prompt and runs env.stacks[stack].delete().

Pros

  • Syntax is in keeping with Unix standards (e.g. yum install)

Cons

Chain multiple resolvers in stack config to create an AWS list string

Add a way to specify multiple resolvers for a single parameter in a stack config. The result should create an AWS CloudFormation compatible list string (List<String> such as "value1,value2,value3"). This would reduce the need for many separate AWS CloudFormation parameters in a template, which is particularly handy for when using raw AWS CloudFormation templates.

Instantiate stacks directly

Change CLI commands that perform actions to a single stack to only initialise and use a Stack object, not use an Environment object. This would be more in-keeping with the code's API and enforce better code structure.

Breadcrumb trail in docs - from getting started.

Final line of the getting started page is the following:

"Full details about Sceptre’s capabilities can be found in the full documentation."

Could you change "full documentation" into a link back to the actual full docs - trying to keep people hooked in to carry on once they've started would be pretty neat.

update-env-cs

It would be nice to have a way to update an environment from the CLI. Ideally this would step me through Change Sets for the stacks in that env that had changes.

I realize that launch-env can update an environment but there would never be a need for a change set when creating a stack so I think we need update-env - thoughts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.