kristofwillen / cfntagger Goto Github PK

View Code? Open in Web Editor NEW

4.0 2.0 0.0 231 KB

Mass tag resources in Cloudformation templates

License: GNU General Public License v3.0

Python 99.56% Makefile 0.44%

cloudformation tagging

cfntagger's Introduction

cfntagger

cfntagger is a Cloudformation tagging tool. It sets tags on resources which support them in Cloudformation templates.

Why tagging ?

Tagging resources in AWS is crucial for larger environments. It helps you to quickly find resources based upon classifiers set in resource tags. Tag keys are often set by convention in larger teams. So they are prone to frequent change. And nothing is more boring than sifting through heaps of yaml files modifying tags. Cfntagger helps you with that.

Installation

$ pip install cfntagger

Usage

$ export CFN_TAGS='{"Creator": "Erlich", "Team": "Incubator"}'
$ cfntagger -h
usage: cfntagger [-h] (--file FILE | --directory DIRECTORY) [--simulate] [--git]

Add bulk tags to CloudFormation resources

options:
  -h, --help            show this help message and exit
  --file FILE, -f FILE  The CloudFormation template file to modify
  --directory DIRECTORY, -d DIRECTORY
                        A directory containing CFN templates to modify
  --simulate, -s        simulate, do not overwrite the inputfile
  --git, -g             add git remote and file info as tags
  --version, -v         show version

where :

filename : the Cloudformation file to tag
directory : a directory filled with Cloudformation templates, recursively to search
simulate : whether or not to overwrite the file in place. If specified, output the changed template to stdout. If not specified as argument (default behavior), replace the file with the corrected version.
addgit: add git information, like git repo and file in which the resource has been defined

The 'file' and 'directory' arguments are mutually exclusive.

WARNING: make sure your files are committed in git before running this tool !

Configuration

There are two ways of configuring cfntagger:

Configfile

The configfile .cfntaggerrc must be present in either:

the root of your git repo
the current directory from where you launch cfntagger

The configfile is in ini-file format, with one section Tags, where tags are placed in key-value pairs:

[Tags]
tag1=tag1value
tag2=tag2value

Example:

[Tags]
Environment=Production
CostCenter=Sales
Application=Aviato

If the file .cfntaggerrc isn't present in one of those locations, the configuration via environment variables is used.

Environment variables

You need to define your custom obligatory tags via the CFN_TAGS environment variable. It must be in JSON format:

$ export CFN_TAGS='{"Creator": "Dinesh", "Team": "Devops"}'

This will add those tags at all resources who support tags.

Prereqs

Have a look at the Pipfile in order to know which Python modules are required.

Unit tests

Unit tests are provided, sample Cloudformation templates are placed in tests/templates. Run the tests with :

$ pytest -v

Reference

Amazon documentation

cfntagger's People

Contributors

Stargazers

Watchers

cfntagger's Issues

add a --version (-v) option (feature requests)

Could you add parameters to easily get the version of cfntagger ?

SAM lambda functions receive list-based tags

Tags for AWS::Serverless::Function based resources have their tags converted to lists by cfntagger. They should remain map based:

$ cfntagger -f myfile.yml
$ git diff
       Tags:
-        "Application": foo
+      - Key: "Application"
+        Value: foo

AttributeError when adding new tags

Wen adding new tags on a json based tag resource, I get an AttributeError:

$ export CFN_TAGS='{"test": "pytest"}' 

$ cat myfile.yml
...
  ResourceWithJsonTags:
    Type: AWS::SSM::Parameter
    Properties:
      Description: APoliciyArnParameter
      Name: APolicyParameterName
      Type: String
      Value: ADeployPolicy
      Tags:
          CostCenter: Sales

$ cfntagger -f myfile.yaml # cfntagger tags correctly
...
$ export CFN_TAGS='{"test": "pytest", "Creator": "Dinesh"}'      
$ cfntagger -f ./myfile.yml

[./myfile.yml][Resource] ResourceWithJsonTags => AWS::SSM::Parameter
Traceback (most recent call last):
  File "/usr/local/bin/cfntagger", line 73, in <module>
    cfn_tagger.tag()
  File "/usr/local/lib/python3.11/site-packages/cfntagger/cfntagger.py", line 721, in tag
    updated = self.change_tags(taglist=restags, resource=item)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/cfntagger/cfntagger.py", line 682, in change_tags
    key, value = get_tag_kv(tags, taglist)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/cfntagger/cfntagger.py", line 29, in get_tag_kv
    return resourcetag, resourcetaglist.get(resourcetag)
                        ^^^^^^^^^^^^^^^^^^^
AttributeError: 'CommentedSeq' object has no attribute 'get'

[Sonarqube] Reduce cognitive complexity in Tagger.tag

Tagger.tag function is too complex, needs some rewrite.

cfntagger tags wrong resources

cfntagger tags resources which does not support tags, like AWS::Config::ConfigRule, or AWS::Glue::Database.

Please make sure to include only supported resources!

Add configuration file option

As a devops engineer, I would like to have an option so cfntagger can check a local configuration file per repo/dir. Some repos can have different tagging requirements or tag values (eg 'Application' tag), and working with an envvar can be confusing, as switching between directories might wipe out certain tag values.

I propose a .cfntaggerrc file which overrides the CFN_TAGS envvar.

Cases:

both .cfntaggerrc & CFN_TAGS exist => .cfntaggerrc is used
only .cfntaggerrc or CFN_TAGS exist => use the values from the provided tag source
both are absent => error

cfntagger sometimes drops comments

Take this cfn template:

---
AWSTemplateFormatVersion: 2010-09-09
Description: Creates EC2 instances
# Top line of comment
Resources:
  # Line of comment in resource
  NOkEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t3.nano
      ImageId: ami-deadbeef

Result of cfntagger:

---
AWSTemplateFormatVersion: 2010-09-09
Description: Creates EC2 instances
# Top line of comment
Resources:
  NOkEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t3.nano
      ImageId: ami-deadbeef
      Tags:
      - Key: Owner
        Value: John

The line of comments in the resource block has been dropped.

.cfntaggerrc converts tag names (keys) to lowercase

Consider the following .cfntaggerrc:

[Tags]
Creator=Bighead
Application=NipAlert

When using this config, cfntagger converts the tag names to lowercase:

$ cfntagger -f ./myfile.yml -s
...
Resources:
  ResourceWithTags:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: a1.xlarge
      ImageId: ami-deadb33f
      Tags:
      - Key: creator
        Value: BigHead
      - Key: application
        Value: NipAlert

I expect the tags 'Creator' and 'Application', not 'creator' and 'application'.

Incorrect parameters definitions for resource of type AWS::SSM::Parameter

Hello,
I went through an issue when deploying a cfn template after it was processed by cfntagger.
The error encounterred was

Property validation failure: [Value of property {/Tags} does not match type {Map}]
After cfntagger process the template, the definition for this resource was

APolicyArnParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Description: APoliciyArnParameter
      Name: !Ref APolicyParameterName
      Type: String
      Value: !Ref ADeployPolicy
      Tags:
        - Key: CostCenter
          Value: xxxx
        - Key: Application
          Value: xxxx-vmmgmt
        - Key: Owner
          Value: [email protected]
        - Key: Environment
          Value: production

After investigating this issue, using this link I fixed the issue by manually updating the resource definition like

APolicyArnParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Description: APoliciyArnParameter
      Name: !Ref APolicyParameterName
      Type: String
      Value: !Ref ADeployPolicy
      Tags:
          CostCenter: xxxx
           Application: xxxx-vmmgmt
           Owner: [email protected]
           Environment: production

Wrong gitfile value

When executed in a subdir, cfntagger -g assigns the relative path from that directory for 'gitfile', instead of the path relative to the git repo root dir :

$ pwd # '.' is the git repo root dir
/tmp/myrepo
$ cd ./tests/templates

$ export CFN_TAGS='{"Team": "XYZ"}'

$ cfntagger -s -g -f ./ec2.yml
[...]
      - Key: gitfile
        Value: ./ec2.yml

TypeError if resource has no Properties defined

This cfn snippet is valid cfn:

Resources:
  PipelineS3Bucket:
    Type: AWS::S3::Bucket

Running cfntagger on it produces a TypeError:

[/tmp/foo.yml][Resource] PipelineS3Bucket => AWS::S3::Bucket
Traceback (most recent call last):
  File "/home/kristof/.local/bin/cfntagger", line 65, in <module>
    cfn_tagger.tag()
  File "/home/kristof/.local/lib/python3.10/site-packages/cfntagger/cfntagger.py", line 206, in tag
    if "Tags" in self.resources[item].get("Properties"):
TypeError: argument of type 'NoneType' is not iterable

cfntagger converts existing list based tags to Key/Value items

When a resource has list-based tags (eg AWS::SSM::Parameter), cfntagger converts existing tags to Key/Value items:

$ cat myfile.yml
...
  ResourceWithJsonTags:
    Type: AWS::SSM::Parameter
    Properties:
      Description: APoliciyArnParameter
      Name: APolicyParameterName
      Type: String
      Value: ADeployPolicy
      Tags:
          CostCenter: Sales
          Application: Aviato

$ cfntagger -f ./myfile.yml -s
...
  ResourceWithJsonTags:
    Type: AWS::SSM::Parameter
    Properties:
      Description: APoliciyArnParameter
      Name: APolicyParameterName
      Type: String
      Value: ADeployPolicy
      Tags:
      - Key: CostCenter
        Value: Sales
      - Key: Application
        Value: Aviato

This breaks CFN deployments.
I'm running cfntagger v0.10.0

Provide a Docker image for Github Action use

As a DevOps engineer, I would like to use cfntagger in a CI/CD chain in Github Actions. Please provide a Docker image so this can be launched from a workflows yaml file.

cfntagger fails when running with empty CFN_TAGS envvar

I'm running cfntagger with an empty config, as I only want to add the git tags. Doing so, cfntagger fails with an UnboundLocalError:

$ export CFN_TAGS='{}'
$ cfntagger -f ./template.yaml -s -g
[./template.yaml][Resource] CheckovStats => AWS::Serverless::Function
[./template.yaml][Resource] CheckovLogGroup => AWS::Logs::LogGroup

Traceback (most recent call last):
  File "/usr/local/bin/cfntagger", line 65, in <module>
    cfn_tagger.tag()
  File "/usr/local/lib/python3.10/site-packages/cfntagger/cfntagger.py", line 401, in tag
    self.resources[item]["Properties"]["Tags"] = [addtags]
UnboundLocalError: local variable 'addtags' referenced before assignment