Giter VIP home page Giter VIP logo

bridgecrewio / yor Goto Github PK

View Code? Open in Web Editor NEW
806.0 17.0 123.0 166.33 MB

Extensible auto-tagger for your IaC files. The ultimate way to link entities in the cloud back to the codified resource which created it.

Home Page: https://www.yor.io

License: Apache License 2.0

Go 93.40% HCL 5.84% Dockerfile 0.19% Shell 0.55% Makefile 0.02%
iac terraform devops cloud cloudsecurity cloudformation serverless tagging infrastructure-as-code hacktoberfest

yor's Introduction

Coverage Maintained by Bridgecrew.io golangci-lint security slack-community Go Report Card Go Reference Docker pulls Chocolatey downloads GitHub All Releases

Yor is an open-source tool that helps add informative and consistent tags across infrastructure as code (IaC) frameworks. Today, Yor can automatically add tags to Terraform, CloudFormation, and Serverless Frameworks.

Yor is built to run as a GitHub Action automatically adding consistent tagging logics to your IaC. Yor can also run as a pre-commit hook and a standalone CLI.

Features

  • Apply tags and labels on infrastructure as code directory
  • Tracing: yor_trace tag enables simple attribution between an IaC resource block and a running cloud resource.
  • Change management: git-based tags automatically add org, repo, commit and modifier details on every resource block.
  • Custom taggers: user-defined tagging logics can be added to run using Yor.
  • Skips: inline annotations enable developers to exclude paths that should not be tagged.
  • Dry-Run: get a preview of what tags will be added without applying any.

Demo

Table of contents

Getting Started

Installation

MacOS / Linux

brew tap bridgecrewio/tap
brew install bridgecrewio/tap/yor

If not using Brew:

pip3 install lastversion
lastversion bridgecrewio/yor -d --assets
tar -xzf $(find . -name *.tar.gz)
chmod +x yor
sudo mv yor /usr/local/bin

OR

Windows

choco install yor

OR

Docker

docker pull bridgecrew/yor

docker run --tty --volume /local/path/to/tf:/tf bridgecrew/yor tag --directory /tf

GitHub Action

name: IaC trace

on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  yor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        name: Checkout repo
        with:
          fetch-depth: 0
          ref: ${{ github.head_ref }}
      - name: Run yor action and commit
        uses: bridgecrewio/yor-action@main

Azure DevOps Pipeline

Install Yor with:

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    curl -s -k https://api.github.com/repos/bridgecrewio/yor/releases/latest | jq '.assets[] | select(.name | contains("linux_386")) | select(.content_type | contains("gzip")) | .browser_download_url' -r | awk '{print "curl -L -k " $0 " -o yor.tar.gz"}' | sh
    sudo tar -xf yor.tar.gz -C /usr/bin/ 
    rm yor.tar.gz 
    sudo chmod +x /usr/bin/yor 
    echo 'alias yor="/usr/bin/yor"' >> ~/.bashrc
    yor --version

Pre-commit

  - repo: https://github.com/bridgecrewio/yor
    rev: 0.1.143
    hooks:
      - id: yor
        name: yor
        entry: yor tag -d
        args: ["."]
        language: golang
        types: [terraform]
        pass_filenames: false

Usage

tag : Apply tagging on a given directory.

# Apply all the tags in yor on the directory tree terraform.
yor tag --directory terraform/

# Apply all the tags in yor on the directory terraform, without tagging subdirectories.
yor tag --directory terraform/ --non-recursive

# Apply only the specified tags git_file and git_org
yor tag --directory terraform/ --tags git_file,git_org

# Apply all the tags in yor except the tags starting with git and yor_trace
yor tag --directory terraform/ --skip-tags git*,yor_trace

# Apply only the tags under the git tag group
yor tag --tag-groups git --directory terraform/

# Apply key-value tags on a specific directory
export YOR_SIMPLE_TAGS='{ "Environment" : "Dev" }'
yor tag --tag-groups simple --directory terraform/dev/

# Perform a dry run to get a preview in the CLI output of all of the tags that will be added using Yor without applying any changes to your IaC files.
yor tag -d . --dry-run

# Use an external tag group configuration file path
yor tag -d . --config-file /path/to/conf/file/

# Apply tags to all resources except of a specified type
yor tag -d . --skip-resource-types aws_s3_bucket

# Apply tags with a specifix prefix
yor tag -d . --tag-prefix "module_"

# Apply tags to all resources except with the specified name
yor tag -d . --skip-resources aws_s3_bucket.operations

# Apply tags to only the specified frameworks
yor tag -d . --parsers Terraform,CloudFormation

# Run yor with custom tags located in tests/yor_plugins/example and custom taggers located in tests/yor_plugins/tag_group_example
yor tag -d . --custom-tagging tests/yor_plugins/example,tests/yor_plugins/tag_group_example

-o : Modify output formats.

# Default cli output
yor tag -d . -o cli

# json output
yor tag -d . -o json

# Print CLI output and additional output to a JSON file -- enables programmatic analysis alongside printing human readable results
yor tag -d . --output cli --output-json-file result.json

--skip-dirs : Skip directory paths you can define paths that will not be tagged.

## Run on the directory path/to/files
yor tag -d path/to/files

## Run yor on the directory path/to/files, skipping path/to/files/skip/ and path/to/files/another/skip2/
yor tag -d path/to/files --skip-dirs path/to/files/skip,path/to/files/another/skip2

list-tag

# List tag classes that are built into yor.
yor list-tag-groups

# List all the tags built into yor
yor list-tags

# List all the tags built into yor under the tag group git
yor list-tags --tag-groups git

Supporting comment format

To prevent resource from being tagged, apply the following comment pattern above the resource, currently supported only in Terraform and CloudFormation files.

Example

skip specific resource - #yor:skip

## for terraform files
#yor:Skip 
resource "aws_instance" "example_instance" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"
  subnet_id     = aws_subnet.example_subnet.id }

## for cloudformation files
#yor:skip
  ExampleInt:
    Type: AWS::Lambda::Function
    Properties:
      Description: An example template

skip all rsources in the page - #yor:skipAll

## for terraform files
#yor:skipAll
resource "aws_vpc" "example_vpc" {
  cidr_block = "10.0.0.0/16" }

resource "aws_subnet" "example_subnet" {
  vpc_id            = aws_vpc.example_vpc.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-west-1a" }

## for cloudformation files
#yor:skipAll
Resources:
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 100

  NewVolume2:
    Type: AWS::EC2::Volume
      Tags:
        - Key: MyTag
          Value: TagValue
        - Key: Name

What is Yor trace?

yor_trace is a magical tag creating a unique identifier for an IaC resource code block.

Having a yor_trace in place can help with tracing code block to its cloud provisioned resources without access to sensitive data such as plan or state files.

See demo here

Contributing

Contribution is welcomed!

We are working on extending Yor and adding more parsers (to support additional IaC frameworks) and more taggers (to tag using other contextual data).

To maintain our conventions, please run lint on your branch before opening a PR. To run lint:

golangci-lint run --fix --skip-dirs tests/yor_plugins

Support

For more support contact us at https://slack.bridgecrew.io/.

yor's People

Contributors

achiar99 avatar actions-user avatar alexjurkiewicz avatar arielkru avatar chanochshayner avatar dependabot[bot] avatar exe-r avatar giladmark avatar gruebel avatar guyeisenkot avatar hi-artem avatar ismailyenigul avatar jameswoolfenden avatar jasonnovich avatar jdorsman-bhvr avatar lonegunmanb avatar matansha avatar metahertz avatar mikeurbanski1 avatar miri-kaufman avatar miriam-slov avatar nimrodkor avatar omrymen avatar rabeazr avatar rotemavni avatar schosterbarak avatar tronxd avatar tsmithv11 avatar wanderer163 avatar yutachaos avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yor's Issues

Auto network tags

Is your feature request related to a problem? Please describe.
Today, I can use yor to add labels to my resources, however I am looking for ability to be able to add Network Tags (GCP) to my resources as well.

Describe the solution you'd like
Ability to add Network tags via yor

Support for non-recursive tagging

Is your feature request related to a problem? Please describe.
I'm toying with adding this to our Terraform module builds. The concept is interesting, but it chokes when I just point it to . because there are subdirectories that it attempts to scan (e.g., .vscode/settings.json, ./tests, ./examples). In order to keep the tagging at only that level, I have to ls to look up the directory names, and manually add them to the --skip-dirs parameter.

Describe the solution you'd like
A non-recursive option which limits the searching to only the current directory.

Describe alternatives you've considered
I've tried --skip-dirs *, --skip-dirs "*", and --skip-dirs "*,.*" to no avail.

Additional context
Using latest Terraform (1.0.10) and yor downloaded from Homebrew today.

tag_group level directory filters are allowed but not supported

Describe the bug
Based on the docs, directory filters for tag groups are seemingly only supported for each tag in the tag group. For example:

tag_groups:
  - name: devops
    tags:
      - name: group
        value:
          default: devops
        filters:
          directory: apps/datadog

We would like to be able to use the directory filter on the tag_group level instead of per tag. For example:

tag_groups:
  - name: datadog
    tags: 
      - name: group
        value:
          default: devops
      - name: env
        value:
          default: prod
    filters:
      directory: apps/datadog
  - name: jenkins
    tags: 
      - name: group
        value:
          default: infra
      - name: env
        value:
          default: dev
    filters:
      directory: apps/jenkins

While yor is able to run with this config the directory filters are not respected at all.
When run with the above config file there there are resources in the same file which have tags from each of the different groups.
For example:

resource "aws_instance" "test" {
  ....
  ....

  tags = {
    group                = "devops"
    env                   = "prod"
  }
}

resource "aws_eip" "test" {
  ....
  ....
  tags = {
    group                = "infra"
    env                   = "dev"
  }
}

Even files that do not correspond to the directory filters are tagged as well (files in the other subdirectories of apps/).

Expected behavior
It isn't clear if this is something that is currently supported. If not, yor shouldn't be able to run with an incorrect config file.

Desktop (please complete the following information):

  • OS: macOS Catalina 10.15.2
  • Yor Version 0.1.128

Module tags for Google (they use the word "labels")

Describe the bug
Looks like yor will not populate labels for GCP modules with "labels" for example:

module "project" {
  source = "[email protected]:my-org/terraform-google-project?ref=v2.1.3"

  prefix            = var.app_abbrev
  system            = var.system
  env               = var.env
  random_project_id = var.random_project_id
  labels = {
  }
}

Changing labels to tags populates fine. For consistency I'd like to see tags = labels with modules in Google.

Thanks!

support for source = "terraform-aws-modules/security-group/aws" syntax

git clone https://github.com/ismailyenigul/terraform-circleci-demo
cd terraform-circleci-demo
 yor tag --directory .
2021/05/19 15:21:47 [INFO] Setting up to tag the directory .

2021/05/19 15:21:47 [INFO] failed to parse extra tags from env: unexpected end of JSON input
2021/05/19 15:21:47 [WARNING] failed to parse hcl module in directory terraform-aws-modules/security-group/aws/terraform-aws-modules/security-group/aws because of errors Failed to read module directory: Module directory terraform-aws-modules/security-group/aws does not exist or cannot be read.
2021/05/19 15:21:47 [DEBUG] Mute logging

test code:

module "complete_sg" {
  source              = "terraform-aws-modules/security-group/aws"
  name                = "my-sg-test-ismail"
  vpc_id              = data.aws_vpc.default.id
  use_name_prefix     = true
  ingress_cidr_blocks = ["10.10.0.0/16"]
  ingress_rules       = ["https-443-tcp"]
}

it works! if I change source = "terraform-aws-modules/security-group/aws" to
source = "git::[email protected]:/terraform-aws-modules/terraform-aws-security-group.git?ref=v4.0.0"

source = "terraform-aws-modules/security-group/aws" should be supported.

External modules resources tagging

Is your feature request related to a problem? Please describe.
We are looking at using yor to auto tag terraform IAC files. One specific use case which doesn't seem to be supported at the moment is module level tagging.
If given a module defined as follows:

# Example module to tag
module "consul" {
  source = "hashicorp/consul/aws"
  version = "0.1.0"
}

Describe the solution you'd like
I would like yor to be able to tag all resources that will be created by the module. The reason we looking for something like this is we have internal terraform modules, using yor as it is we won't be able to tag them dynamically from repos where they are being used.
Possible option is:

  • I run terraform init which should download the relevant module
  • Running yor tag I am able to tag local files as well as modules I have downloaded from terraform init.

Provide a list of supported Terraform Provider.

Is your feature request related to a problem? Please describe.
I wanted to try Yor out for personal projects and found out, that the Terraform provider I use are as it seems not supported to run Yor against it.
For example I have a K3S Cluster at Civo or some development machines at Scaleway.
As it seems DigitalOcean is also not supported - at least I don't receive any errors or recommended tags by Yor.

Describe the solution you'd like
I would like to have some kind of overview about which Providers are supported or even in detail which terraform resources for the providers are supported. Maybe there is a way to add this to the Website or this GitHub Repository.

Describe alternatives you've considered
An additional idea would be to maybe autodetect if the current Terraform provider is support by Yor, so the CLI would just spit out "Yeah sorry I found that you want to create a DigitalOcean resource, but we don't support DigitalOcean yet".

Additional context

Here you can see, that my GCP project is the only one which finds resources which can be updated.
Output of a GCP project:

terraform providers && yor tag -d .

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/random]
└── provider[registry.terraform.io/hashicorp/google] 3.75.0

  __    __
  \ \  / /
   \ \/ /___  _  ____
    \  /  _ \| |/  __|
    | |  |_| |   /
    |_|\____/|__|v0.1.85
 Yor Findings Summary
 Scanned Resources:       7
 New Resources Traced:    2
 Updated Resources:       0

New Resources Traced (2):
[...]

Output of a Scaleway project:

terraform providers && yor tag -d .

Providers required by configuration:
.
├── provider[registry.terraform.io/scaleway/scaleway] ~> 2.1
└── provider[registry.terraform.io/integrations/github] ~> 4.0

Providers required by state:

    provider[registry.terraform.io/integrations/github]

    provider[registry.terraform.io/scaleway/scaleway]

2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_account_ssh_key
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_vpc_private_network
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_instance_ip
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_instance_ip_reverse_dns
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_instance_security_group
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_instance_security_group_rules
2021/07/28 12:08:55 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type scaleway_instance_server
  __    __
  \ \  / /
   \ \/ /___  _  ____
    \  /  _ \| |/  __|
    | |  |_| |   /
    |_|\____/|__|v0.1.85
 Yor Findings Summary
 Scanned Resources:       9
 New Resources Traced:    0
 Updated Resources:       0

Output of a DigitalOcean project:

terraform providers && yor tag -d .

Providers required by configuration:
.
└── provider[registry.terraform.io/digitalocean/digitalocean] 2.6.0

2021/07/28 12:04:08 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type digitalocean_droplet
2021/07/28 12:04:08 [WARNING] failed to parse terraform block because failed to find tags attribute name for resource type digitalocean_firewall
  __    __
  \ \  / /
   \ \/ /___  _  ____
    \  /  _ \| |/  __|
    | |  |_| |   /
    |_|\____/|__|v0.1.85
 Yor Findings Summary
 Scanned Resources:       5
 New Resources Traced:    0
 Updated Resources:       0

Can't run yor on MacOS

MacOS Version 10.14.6
Terraform v0.14.6
Go version go1.16.4 darwin/amd64

All yor commands fail to run, including yor --version. This is the same when installed through homebrew and when build from source.

Expected output:

yor version 0.1.59

Actual output:

./yor --version
runtime: netpoll: break fd ready for -2
fatal error: runtime: netpoll: break fd ready for something unexpected
runtime stack:
runtime.throw(0x5df86e7, 0x39)
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/panic.go:1117 +0x72
runtime.netpoll(0x16ea5e, 0x14bdedda9e06)
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/netpoll_kqueue.go:147 +0x33e
runtime.findrunnable(0xc000073800, 0x0)
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/proc.go:2923 +0x3ee
runtime.schedule()
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/proc.go:3169 +0x2d7
runtime.park_m(0xc000001b00)
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/proc.go:3318 +0x9d
runtime.mcall(0x80000)
	/usr/local/Cellar/go/1.16.4/libexec/src/runtime/asm_amd64.s:327 +0x5b
goroutine 1 [runnable, locked to thread]:
golang.org/x/text/internal/language.getLangID(0xc0005acb18, 0x2, 0x20, 0x80, 0x5bbf260, 0x5468617)
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/lookup.go:39 +0xbb
golang.org/x/text/internal/language.parseTag(0xc00051cbd8, 0x0, 0x0, 0x0, 0x5dab424)
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/parse.go:294 +0x6d
golang.org/x/text/internal/language.parse(0xc00051cbd8, 0x5dab424, 0xe, 0x0, 0x0, 0x0, 0x6169636e656c, 0x0)
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/parse.go:265 +0x236
golang.org/x/text/internal/language.Parse(0x5dab424, 0xe, 0x0, 0x0, 0x0, 0x1, 0xc00004e220)
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/parse.go:250 +0x215
golang.org/x/text/internal/language.MustParse(...)
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/tags.go:10
golang.org/x/text/internal/language/compact.init.0()
	/Users/blackdenc/go/pkg/mod/golang.org/x/[email protected]/internal/language/compact/compact.go:59 +0x13e
goroutine 19 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc00013de50)
	/Users/blackdenc/go/pkg/mod/[email protected]/stats/view/worker.go:154 +0xcd
created by go.opencensus.io/stats/view.init.0
	/Users/blackdenc/go/pkg/mod/[email protected]/stats/view/worker.go:32 +0x57

Add git tags to all scanned resources

Is your feature request related to a problem? Please describe.
I'm currently working on adding yor to an existing AWS terraform stack. When running yor tag --tag-groups git the git tags are not added to all resources. I noticed that if I update a resource in code and run yor again the git tags are added.

This behavior seems to indicate that git tags are only added to new/ updated resources.

Describe the solution you'd like
Ability to add git tags to all resources in a stack irrespective of if they were recently updated.

yor version : v0.1.122

list-tags and list-tag-groups to use config-file for dry-run

It would be helpful if a
yor list-tag-groups -d . --config-file myfile.yaml

Would also include the new tag-groups and tags from the config file. This would help to ensure the config file was written correctly prior to use. Same for list-tags.

Tag does not respect existing tags

Describe the bug

resource "aws_iam_role" "lambda_deploy_role" {
  count = length(var.deploy_user) > 0 ? 1 : 0
  name  = module.iam_deploy_label.id

  assume_role_policy = jsonencode(
    {
      "Version" : "2012-10-17",
      "Statement" : [
        {
          "Sid" : "assume",
          "Action" : "sts:AssumeRole",
          "Principal" : {
            "AWS" : var.deploy_user
          },
          "Effect" : "Allow"
        }
      ]
  })
  tags = module.iam_deploy_label.tags
}

when tagged creates

resource "aws_iam_role" "lambda_deploy_role" {
  count = length(var.deploy_user) > 0 ? 1 : 0
  name  = module.iam_deploy_label.id

  assume_role_policy = jsonencode(
    {
      "Version" : "2012-10-17",
      "Statement" : [
        {
          "Sid" : "assume",
          "Action" : "sts:AssumeRole",
          "Principal" : {
            "AWS" : var.deploy_user
          },
          "Effect" : "Allow"
        }
      ]
  })
  tags                 = module.iam_deploy_label.
  git_commit           = "c82ca7bedd825f8a76d762abafc9351bfa7a268f"
  git_file             = "iam.tf"
  git_last_modified_at = "2021-03-27 21:26:52"
  git_last_modified_by = "<REDACTED>"
  git_modifiers        = "<REDACTED>"
  git_org              = "<REDACTED>"
  git_repo             = "<REDACTED>"
  yor_trace            = "7b714242-de52-454e-9184-ce906e4c88b0"
  tags
}

which is not valid terraform

To Reproduce
Steps to reproduce the behavior:

  1. See above

Expected behavior

Tagging should use a strategy like merge(existing,{...})

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

list-tags should work in a non git directory

Running...
yor list-tags
...gives the error
<datetime> [ERROR] Failed to initialize git service for path .. Please ensure the provided root directory is initialized via the git init command.

After speaking with @schosterbarak it seems the available tags list is not dependent the git repo or local path. As such, this command should work outside of a git repo. Or at least show the defaults without any local saved state if that were to be the case.

Happy to work on this issue as well BTW.

Unable to fetch git blame info on specific files

@nimrodkor Most of my git tagging issues have been resolved after #147, however it seems like one of the files in the sub-dir I tested this on was still unable to fetch git blame info:

2021/06/18 09:01:16 [WARNING] Unable to get git blame for file envs/cetus/receipts/receipts.tf: failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error contents and commits have different length
2021/06/18 09:01:29 [WARNING] Failed to tag receipts_bucket with git tags, err: failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error failed to get blame for latest commit of file envs/cetus/receipts/receipts.tf because of error contents and commits have different length

You can see that one of the resources was only tagged with yor_trace:

New Resources Traced (4):
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
|                   FILE                   |            RESOURCE            |       TAG KEY        |                TAG VALUE                 |                YOR ID                |
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
| envs/cetus/receipts/receipts.tf          | receipts_bucket                | yor_trace            | 0a4a88ef-f713-4f9f-8e50-effadb1422b4     | 0a4a88ef-f713-4f9f-8e50-effadb1422b4 |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | yor_trace            | 0a4a88ef-f713-4f9f-8e50-effadb1422b4     |                                      |
+------------------------------------------+--------------------------------+----------------------+------------------------------------------+--------------------------------------+
| envs/cetus/receipts/secretsmgr.tf        | aws_iam_policy.secrets_manager | yor_trace            | 0b7c9745-0ded-438b-8b59-7876b1b4e712     | 0b7c9745-0ded-438b-8b59-7876b1b4e712 |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | git_org              | waveaccounting                           |                                      |
+                                          +                                +----------------------+------------------------------------------+                                      +
|                                          |                                | git_repo             | keller                                   |                                      |
+                                          +                                +----------------------+------------------------------------------+                                      +

Not sure what error contents and commits have different length means. From slack:

It seems like we're not handling git correctly for merge commits

Originally posted by @mwarkentin in #147 (comment)

Brew tap/install from fresh doesn't work

Describe the bug
The first time ever installing doesn't seem to work for me. The error is not particularly helpful.
Are there some missing dependencies (compilers, Xcode, etc.)?

To Reproduce

$ brew tap bridgecrewio/tap
brew install bridgecrewio/tap/yor
Updating Homebrew...
==> Auto-updated Homebrew!
Updated Homebrew from 3e52e8b48 to 89b861915.
Updated 4 taps (homebrew/cask-versions, homebrew/core, homebrew/cask and homebrew/cask-fonts).
==> New Formulae
umple
==> Updated Formulae
Updated 84 formulae.
==> Updated Casks
Updated 77 casks.
==> Deleted Casks
beautune blue-jeans-browser-plugin caramba-switcher

==> Tapping bridgecrewio/tap
Cloning into '/usr/local/Homebrew/Library/Taps/bridgecrewio/homebrew-tap'...
remote: Enumerating objects: 325, done.
remote: Counting objects: 100% (325/325), done.
remote: Compressing objects: 100% (214/214), done.
remote: Total 325 (delta 105), reused 20 (delta 4), pack-reused 0
Receiving objects: 100% (325/325), 31.70 KiB | 3.96 MiB/s, done.
Resolving deltas: 100% (105/105), done.
Tapped 1 formula (12 files, 45KB).
==> Installing yor from bridgecrewio/tap
==> Cloning https://github.com/bridgecrewio/yor.git
Cloning into '/Users/richardj/Library/Caches/Homebrew/yor--git'...
==> Checking out tag 0.1.85
HEAD is now at 4b31cf4 When adding tags to templates, validate modified template is acceptable (#167)
Error: An exception occurred within a child process:
Errno::ENOENT: No such file or directory - yor

Expected behaviour
Yor installed

Desktop (please complete the following information):

  • OS: Big Sur 11.4
  • Yor Version (see above)

Additional context
Add any other context about the problem here (e.g. code snippets).

Can .yor_plugins be at the users home directory vs. CWD?

I know I can easily add .yor_plugins to .gitignore however it seems unnecessary to copy the plugins all over the place. Minimally a single .yor_plugin for each repo would be nice, or just get it out of the repo all together and move to ~/.yor_plugins.

Thanks!

Unable to tag a module that has a required configuration alias (Terraform)

Describe the bug
When running yor tag in a directory with a configuration block that looks like this:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      configuration_aliases = [aws.route53_parent_zone_account]
    }
  }
  required_version = ">= 1.0"
}

Yor throws an error [ERROR] Variables not allowed: Variables may not be used here. and does not tag anything.

To Reproduce
See above, commenting out the configuration_aliases line like so:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      # configuration_aliases = [aws.route53_parent_zone_account]
    }
  }
  required_version = ">= 1.0"
}

allows yor to run as expected.

Expected behavior
Resources to be tagged, required configuration_aliases is valid terraform

Desktop (please complete the following information):

  • OS: Ubuntu 20.04 in WSL2
  • Yor Version: 0.1.104

CloudFormation: when no tags are present, Yor renders an incorrect template

Take this sample minimal S3 CloudFormation input template:


---
AWSTemplateFormatVersion: 2010-09-09
Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
    DeletionPolicy: Retain

when running yor tag -d ., yor adds the required tags, but doesn't add a 'Tags:' directive:

---
AWSTemplateFormatVersion: 2010-09-09
Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
    DeletionPolicy: Retain
    - Key: git_repo
      Value: yor-sandbox
    - Key: git_file
      Value: s3.yml
[...]

thus rendering the CloudFormation template invalid.
I would expect yor to issue an error saying it expects the 'Tags:' directive, or to add it itself.
When tagging a similar minimal Terraform s3.tf file, yor behaves correctly.

skip-resource-types does not accept a comma delimited list

Describe the bug
When a comma delimited list is passed to --skip-resource-types all resources in the list should be skipped. However, if more than one resource is passed to the command none of the resource are skipped. Currently the command only produces the desired result when individual resource types are passed.

To Reproduce
Steps to reproduce the behavior:

  1. Create terraform stack with resources aws_elb and aws_instance
  2. Run cli command yor tag --skip-resource-types aws_elb,aws_instance -d .
  3. See that tagging for neither resource has been skipped.

Note that using the arg twice (yor tag --skip-resource-types aws_elb --skip-resource-types aws_instance -d .) does produce the intended result. However, I believe this arg should follow the convention seen with the other flags such as --skip-tags, --skip-dirs, etc. and accept a comma delimited list.

Expected behavior
All resource types specified in the comma delimited list are skipped when tagging.

Desktop (please complete the following information):

  • OS: macOS Catalina
  • Yor Version: 0.1.122

Yor doesn't parse references in cloudformation

Describe the bug
When running yor tag -d . with yor v0.1.84 in a directory with cloudformation templates, any that use a parameter for EBS size return the below warning, and don't get tagged.

ERROR: json: cannot unmarshal string into Go struct field Instance_Ebs.Properties.BlockDeviceMappings.Ebs.VolumeSize of type int
2021/07/07 15:16:15 [WARNING] There was an error processing the cloudformation template template.yml: json: cannot unmarshal string into Go struct field Template.Resources of type int

  __    __
  \ \  / /
   \ \/ /___  _  ____
    \  /  _ \| |/  __|
    | |  |_| |   /
    |_|\____/|__|v0.1.84
 Yor Findings Summary
 Scanned Resources:       0
 New Resources Traced:    0
 Updated Resources:       0

To Reproduce
Steps to reproduce the behavior:

  1. Use the below cloudformation template:
Parameters:
  EbsSize:
    Description: Please provider the extra EBS size you need
    Type: Number
    Default: 20
Resources:
  Ec2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-0d57f0639aae57eb6
      InstanceType: t3.small
      KeyName: <Use your own key>
      SecurityGroupIds: [<Security Group>]
      SubnetId: <Subnet ID>
      BlockDeviceMappings:
        - DeviceName: /dev/xvdf
          Ebs:
            DeleteOnTermination: true
            Encrypted: true
            VolumeSize: !Ref EbsSize
            VolumeType: gp2
  1. Run cli command yor tag -d .
  2. See error
  3. Comment out the line VolumeSize: !Ref EbsSize
  4. Re-run cli command yor tag -d .
  5. See the command is successful

Expected behavior
Yor should add tags to cloudformation that use parameters

Desktop (please complete the following information):

  • OS: MacOS Mojave 10.14.6
  • Yor Version: v0.1.84

github-actions[bot] not valid for tag regex

When run as a github action, yor produces the following tags block on an aws_iam_policy resource

resource "aws_iam_policy" "test-policy" {
  <Resource configuration>

  tags = {
    git_last_modified_by = "41898282+github-actions[bot]@users.noreply.github.com"
    git_modifiers        = "41898282+github-actions[bot]/blackdenc"
  }
}

When terraform apply is run on this resource, the following error is thrown:

Error: error updating tags for IAM Policy (<ARN>): error tagging resource (<ARN>): ValidationError: 2 validation errors detected: Value '41898282+github-actions[bot]@users.noreply.github.com' at 'tags.1.member.value' failed to satisfy constraint: Member must satisfy regular expression pattern: [\p{L}\p{Z}\p{N}_.:/=+\-@]*; Value '41898282+github-actions[bot]/blackdenc' at 'tags.2.member.value' failed to satisfy constraint: Member must satisfy regular expression pattern: [\p{L}\p{Z}\p{N}_.:/=+\-@]*
        status code: 400, request id: 08e80e1e-8dd1-4bb3-b013-4df5b5ec576c

  on <FILENAME> line 53, in resource "aws_iam_policy" "test-policy":
  53: resource "aws_iam_policy" "test-policy" {

41898282+github-actions[bot] Was the user that added the tags on the initial yor workflow run on this resource. I removed the bot username from the tag and re-ran terraform apply and the resource deploy ed without issues.

Can we change the name of the the github actions user that makes the change in order to match regex?

Cloudformation: empty (null) tags cause stack "CREATE_FAILED"

Context: using yor (version 0.1.47) to tag cloudformation yaml file in local git repo (git init)

Tags are successfully created (see yaml snipped below), however, the empty tags (e.g. git_org) cause that cloudformation create stack to fail with:

CREATE_FAILED
Value ( git_org ) for parameter value is invalid.
Tag value cannot be null.
Use empty string instead. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 0ade36-f7ec-4716-b787-42fb7bf8b; Proxy: null)
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref 'InstanceType'
      SecurityGroups: [!Ref 'InstanceSecurityGroup']
      KeyName: !Ref 'KeyName'
      ImageId: !Ref 'LatestAmiId'
      Tags:
        - Key: yor_trace
          Value: 83d7be13-2c13-4f2d-92a0-5d88e44437c6
        - Key: Environment
          Value: rollwagen-dev
        - Key: git_commit
          Value: 54f7425be387138d3381c760d8d067b499bb8459
        - Key: git_modifiers
          Value: rollwagen
        - Key: git_last_modified_at
          Value: "2021-06-06 11:10:44"
        - Key: git_last_modified_by
          Value: [email protected]
        - Key: git_org
        - Key: git_repo
        - Key: git_file
          Value: EC2InstanceWithSecurityGroupSample.yaml

Warning - Unable to get git blame for file

I have tried this in a few different Git repositories I have cloned down and can reproduce it on all. It will add the yor_trace tags but none of the Git tags.

Command Run (from the root of the Git repo):

yor tag -d .\env\

Error:

2021/06/11 12:56:30 [WARNING] Unable to get git blame for file env\aks.tf: failed to get blame for latest commit of file env\aks.tf because of error file not found
2021/06/11 12:56:30 [WARNING] Failed to tag azurerm_kubernetes_cluster.aks with git tags, err: failed to get blame for latest commit of file env\aks.tf because of error failed to get blame for latest commit of file env\aks.tf because of error file not found

Environment information:

  • Yor Version: v0.1.61
  • Git Version: 2.31.0.windows.1
  • Terraform Version: 1.0.0

In all cases I am in a valid Git repository. All of the other Git command work (including git blame). Not sure if I am doing something or running into a bug on this one.

Oracle Cloud support

Hi,

I would be grateful if you could add the following feature: Use "freeform_tags" of OCI resources to add Yor tags.

Thank you
Jens

Hard to understand error "no provider exists with the given name"

I was trying to run yor for testing on one of our dev applications in our terraform monorepo. When I ran it I got the following error which doesn't really indicate what I can do to fix it. Eg. which provider is it having issues with, etc.

❯ yor tag --directory envs/cetus/receipts
2021/05/27 14:00:16 [ERROR] failed to install provider for directory envs/cetus/receipts because of errors no provider exists with the given name

Yor not handling malformed .gitconfig, breaking the terraform

Describe the bug
The auto-tagger is inserting stray quotes in the git_modifiers tag

To Reproduce
Steps to reproduce the behavior:

  1. Initial tag: git_modifiers = "company-johnsmith/john.smith"
  2. Modify resource to trigger Yor tag update
  3. Updated tag: git_modifiers = "bob.jackson/company-johnsmith/“john.smith"

Expected behavior
Expected tag value:
git_modifiers = "bob.jackson/company-johnsmith/john.smith"
or simply:
git_modifiers = "bob.jackson/john.smith"

Additional context
I have noticed that the user "John Smith" has several different identifiers appearing in the tags e.g.
[email protected]
[email protected]
[email protected]

Possibly to do with the way accounts are set up as part of our organization?

Random YAML files are detected as CloudFormation

Describe the bug
Files that are not CloudFormation templates cause errors to be generated like:

2021/08/17 06:45:32 [WARNING] There was an error processing the cloudformation template 
gcr-ls-docker-creds.yaml: failed to unmarshal YAML: error converting YAML to JSON: 
yaml: invalid map key: map[interface {}]interface {}{".Values.docker_secret_name":interface {}(nil)}

Expected behavior
Only YAML files that are CF templates should generate this error.

Additional context
Check YAML files for AWSTemplateFormatVersion and Resources top level keys, only files with both of these should be considered CloudFormation files and generate parsing errors if they can't be parsed.

[Question] Does yor support a --dry-run option

We avoid automation that changes the source tree without human feedback but this seems to be the model that yor currently uses (e.g. the GitHub action that adds tags to all the things and then commits them to the branch).

In our workflow (we do this with terraform fmt and black for python) we expect the branch as it is committed/pushed by the user to be correct and we have tests that run to confirm it is so (e.g., pytest-black or a script that runs terraform fmt -check).

I'd like to do something similar with yor; make it the responsibility of the dev to run yor and confirm that it did the right thing and write a test that calls yor --check or yor --dry-run and examine its exit code.

I can't see any yor cli flags that would make this possible.

Is it possible (am I missing something?)?

If not, how much work would it require (how big of a change is it?)?

Upgrade Github Issue template to Github Issue Forms

Is your feature request related to a problem? Please describe.
GitHub has recently rolled out a public beta for their issue forms feature. This would allow you to create interactive issue templates and validate them 🤯.

Describe the solution you'd like
This repository currently uses the older issue template format. Your task is to create GitHub issue forms for this repository. We can use this repo's issue templates as a reference for this PR.

Yor with Terragrunt

Is your feature request related to a problem? Please describe.

Is there guidance on using this with terragrunt repository?
This is an example of what those repository could look like https://github.com/gruntwork-io/terragrunt-infrastructure-live-example

Describe the solution you'd like

It would be nice if the yor tag -d . command would be able to parse the inputs looking for a tags key

Describe alternatives you've considered

I've considered locals that would get filled in by yor and then somehow import that into terragrunt, but yor won't populate a variable AFAIK, and terragrunt can't "input" a variable from standard terraform file. It can do envvar, but again it relies on yor to change its purpose.

Additional context
Add any other context or screenshots about the feature request here.

Smarter `git_org` when remote is a GitHub repo

Is your feature request related to a problem? Please describe.
My company uses GitHub Enterprise. As teams and people come and go, we will sometimes move repos around into different orgs or assign different names to match the current state of reality. GitHub automatically handles the redirects so that existing clones don't break even when the "remote" pointers are outdated.

Describe the solution you'd like
If the Git repo is a GitHub or GitHub Enterprise repository, follow the remote location to discover its current location before determining git_org or git_repo. Perhaps use a GitHub API token to query the API for this information.

Describe alternatives you've considered
N/A

Additional context
I don't have enough familiarity with GitLab or BitBucket to know if they work similarly, but I'm sure those users would appreciate this feature as well.

GCP requirements for labels

Label requirements for GCP are here

Trying to use yor in Google Cloud fails:

# Project Module
# https://github.com/my-org/terraform-google-project

module "project" {
  source = "[email protected]:my-org/terraform-google-project?ref=v2.1.3"

  cost_center       = "uZZZ"
  folder_id         = var.folder_id
  prefix            = "shared"
  system            = "services"
  env               = var.env
  random_project_id = var.random_project_id

  labels = {
    team                 = "ops"
    yor_trace            = "8019ee39-ca38-4a8d-8b7a-d50ffd353f5c"
    git_commit           = "c828864803f716011793b240628c4311330de7d8"
    git_file             = "global/main.tf"
    git_last_modified_at = "2021-11-03 13:13:21"
    git_last_modified_by = "[email protected]"
    git_modifiers        = "my-name"
    git_org              = "my-org"
    git_repo             = "my-gcp-repo"
  }
}

ERROR:

Error: Error updating project "my-services-tf21ee98-sb": googleapi: Error 400: field [project.labels] has issue [Label value 'global/main.tf' violates format constraints. The value can only contain lowercase letters, numeric characters, underscores and dashes. The value can be at most 63 characters long. International characters are allowed.], badRequest

In this case it's grumpy about the directory "/" but email etc will fail too.

Describe the solution you'd like
Honestly, I'd like to see Google do better here but I'm afraid that will be a challenge :D Maybe we ca use "-" everywhere for example:

    git_file             = "global-main.tf"
    git_last_modified_at = "2021-11-03-13-13-21"
    git_last_modified_by = "my-name-my-org-com"
    git_modifiers        = "my-name"
    git_org              = "my-org"

My current W/A is to skip those tags for now in pre-commit:

  - repo: git://github.com/bridgecrewio/yor
    rev: 0.1.128
    hooks:
      - id: yor
        name: yor
        entry: yor tag --directory
        args: ["global", "regional", "--skip-tags git_last_modified_by,git_last_modified_at,git_file"]
        language: golang
        types: [terraform]

CloudFormation: yor copies resources instead of modifying them

Take this sample CloudFormation template which defines a VPC with some additional resources:

---
AWSTemplateFormatVersion: 2010-09-09
Description:  This template deploys a VPC, with a pair of public and private subnets spread
  across two Availability Zones. It deploys an internet gateway, with a default
  route on the public subnets. It deploys a pair of NAT gateways (one in each AZ),
   and default routes for them in the private subnets.

Parameters:
  EnvironmentName:
    Description: An environment name that is prefixed to resource names
    Type: String

  VpcCIDR:
    Description: Please enter the IP range (CIDR notation) for this VPC
    Type: String
    Default: 10.192.0.0/16

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

When running 'yor tag -d .', yor will add a resource block for each defined resource :

$ yor tag -d .
$ cat vpc.yml
[...]
Resources:
 VPC:
   Type: AWS::EC2::VPC
   Properties:
     CidrBlock: !Ref VpcCIDR
     EnableDnsSupport: true
     EnableDnsHostnames: true
     Tags:
       - Key: Name
          Value: EnvironmentName
       - Key: git_file
         Value: vpc.yml
       - Key: git_commit
         Value: b9ea9bd11725f65d97ee897e8825214c7e182cc7
       - Key: git_modifiers
         Value: kristof.willen
       - Key: git_last_modified_at
         Value: "2021-06-01 12:31:19"
       - Key: yor_trace
         Value: 0d4a31e9-90ce-452e-b946-c731e3285024
[...]
 VPC:
   Type: AWS::EC2::VPC
   Properties:
     CidrBlock: !Ref VpcCIDR
     EnableDnsSupport: true
     EnableDnsHostnames: true
     Tags:
       - Key: Name
         Value: !Ref EnvironmentName

So I have now 2 VPC resource blocks defined, which breaks my CF template:

$ cfn-lint ./vpc.yml    
E0000 Duplicate resource found "VPC" (line 48)
./vpc.yml:48:3

Additional info:

$ yor --version                                                                                                                                                                                                                                                                                                                                     
yor version 0.1.38

yor adds extra comma to tags. It breaks terraform code.

Bug Description

yor add extra , without checking , is already exist or not.
It breaks terraform code and then terraform init fails.

How to reproduce

git clone https://github.com/terraform-aws-modules/terraform-aws-eks
cd terraform-aws-eks
yor tag --directory .

cd examples/basic/
terraform init
Initializing modules...
- eks in ../..
- eks.fargate in ../../modules/fargate
- eks.node_groups in ../../modules/node_groups
Downloading terraform-aws-modules/vpc/aws 2.78.0 for vpc...
- vpc in .terraform/modules/vpc

Error: Invalid expression

On ../../cluster.tf line 112: Expected the start of an expression, but found
an invalid expression token.


Error: Invalid expression

On ../../workers.tf line 355: Expected the start of an expression, but found
an invalid expression token.

original resource in cluster.tf

resource "aws_security_group" "cluster" {
  count       = var.cluster_create_security_group && var.create_eks ? 1 : 0
  name_prefix = var.cluster_name
  description = "EKS cluster security group."
  vpc_id      = var.vpc_id
  tags = merge(
    var.tags,
    {
      "Name" = "${var.cluster_name}-eks_cluster_sg"
    },
  )
}

as you see above, tags ends with extra comma }, without new block.
and yor add another , to merge it's new tags.
and it breaks terraform code (line 112)

102 resource "aws_security_group" "cluster" {
103   count       = var.cluster_create_security_group && var.create_eks ? 1 : 0
104   name_prefix = var.cluster_name
105   description = "EKS cluster security group."
106   vpc_id      = var.vpc_id
107   tags = merge(
108     var.tags,
109     {
110       "Name" = "${var.cluster_name}-eks_cluster_sg"
111     },
112     , {
113       git_commit           = "f2cc9f503919d5a0a4a16fe3c8336d6d155fe48f"
114       git_file             = "cluster.tf"
115       git_last_modified_at = "2020-02-27 10:23:41"
116       git_last_modified_by = "[email protected]"
117       git_modifiers        = "9570112+nauxliu/brandoconnor/brandon/ryan.ooi"
118       git_org              = "terraform-aws-modules"
119       git_repo             = "terraform-aws-eks"
120       yor_trace            = "8528df16-0b36-4e0f-a050-5891594b979a"
121   })
122 }

Repo does not have .pre-commit-hooks.yaml

I added yor to one of my repos' .pre-commit-config.yaml files with settings taken from the readme:

- repo: https://github.com/bridgecrewio/yor
    rev: 5d56b61447940e9928426e4df6b577f28000cd6d  # frozen: v0.1.65
    hooks:
      - id: yor
        name: yor
        entry: yor tag -d
        args: ["terraform"]
        language: golang
        types: [terraform]
        pass_filenames: false

Then added the config file to a git branch so that I could execute pre-commit run and try it out, but an exception was thrown during initialization:

$ pre-commit run
[INFO] Initializing environment for https://github.com/bridgecrewio/yor.
An error has occurred: InvalidManifestError:
=====> /Users/gwhorley/.cache/pre-commit/repoxhvlei6s/.pre-commit-hooks.yaml does not exist
Check the log at /Users/gwhorley/.cache/pre-commit/pre-commit.log

The log only provided a stacktrace along with the error shown above. The contents of that cached directory contains the cloned repo with no .pre-commit-hooks.yaml. What am I missing?

Custom tag group directory filter - support wildcards

Is your feature request related to a problem? Please describe.
When using the custom tag group with directory filter, today it supports exact prefix - i.e. path/to/dir-tree. I'd like to be able to tag with a specific tag all the directories which match a specific regex / joker. Example - tag everyhing that's in a directory named infra/networkign as Component: Networking.

Describe the solution you'd like
Extending the existing HasPrefix to support regex correctly.

if strings.HasPrefix(blockFP, p) {

Describe alternatives you've considered
Writing out all the possibilities is time consuming and not scalable.

skip_dirs doesn't function according to docs

Describe the bug
According to the README:

yor tag -d path/to/files --skip-dirs path/to/files/skip,path/to/files/another/skip2
Run yor on the directory path/to/files, skipping path/to/files/skip/ and path/to/files/another/skip2/

In reality the full path of the files should not be included in skip_dirs for it to work correctly.

To Reproduce
yor tag -d apps --skip-dirs buildkite works as intended.
yor tag -d apps --skip-dirs apps/buildkite tags all resources in apps/buildkite

Expected behavior
Based on the documentation I would expect skip_dirs to work opposite of how it is actually working.

Desktop (please complete the following information):

  • OS: macOS Catalina 10.15.2
  • Yor Version: 0.1.128

Need the ability to skip files and/or resources

Is your feature request related to a problem? Please describe.
I need a way to skip tagging individual files/resources instead of only directories.

Describe the solution you'd like
Add a new parameter or modify existing skip-dirs parameter to accept file names or terraform resource IDs.

tags are still generated when the directory is the same as skip_dirs

Describe the bug
When the directory is the same as skip_dirs, tags are still being generated even though a Warning is thrown to expect an empty set.

To Reproduce
yor tag -d modules --skip-dirs modules will produce the following warning:

[WARNING] Selected dir, modules, is skipped - expect an empty result

However tags are still generated for all resources inside the modules directory.

Expected behavior
No resources are tagged.

Desktop (please complete the following information):

  • OS: macOS Catalina 10.15.2
  • Yor Version: 0.1.128

Windows releases to .zip

Describe the bug
Recent releases for windows started to have "tar.gz", can this be reverted back to ".zip"?

There was also a recent change to use _ instead of -

Old artifact have the following convention:
yor-0.1.88-windows-amd64.zip

Recent ones:
yor_0.1.101_windows_amd64.tar.gz

Support git tags when --directory != git root

We have a github repo which looks like this:

root/
  .git/
  accounts/
    acc1/
    acc2/
    ...
  envs/
    env1/
      app1/
      app2/
      ...
    env2/
      app1/
      app2/
      ...
    ...

I would like to be able to run yor tag --directory envs/env1/app1 from the root directory. When I do so currently, yor doesn't seem to set the git tags. I assume this is because the .git repo is in the root and not the directory I've configured.

A few options that would resolve for me:

  • Pick up git metadata from the directory where yor is run, not from --directory setting
  • Search up in the tree from --directory if a .git dir is not found
  • Add a --git-dir option that lets me configure it directly

Cloudformation: Parsing errors

Yor v0.1.52

Run yor tag -d . in a folder containing the following three examples.

Mappings:
  MyMap:
    Key: "Value"

Resources:
  ExampleResource:
    Type: AWS::Lambda::Function
    Properties:
      Description: An example template
      Environment:
        Variables:
          BROKEN_FIND_IN_MAP: !FindInMap [Mappings, MyMap, Key]
Resources:
  ExampleResource:
    Type: AWS::Lambda::Function
    Properties:
      Description: An example template
      Environment:
        Variables:
          BROKEN_INT: 10
Resources:
  ExampleResource:
    Type: AWS::Lambda::Function
    Properties:
      Description: An example template
      Environment:
        Variables:
          BROKEN_JOIN: !Join ['-', ['a', 'b', 'c']]

Gives me the following errors:

ERROR: json: cannot unmarshal array into Go struct field Function_Environment.Properties.Environment.Variables of type string
2021/06/07 16:01:51 [WARNING] There was an error processing the cloudformation template example.yml: json: cannot unmarshal array into Go struct field Template.Resources of type string
ERROR: json: cannot unmarshal number into Go struct field Function_Environment.Properties.Environment.Variables of type string
2021/06/07 16:01:51 [WARNING] There was an error processing the cloudformation template example_2.yml: json: cannot unmarshal number into Go struct field Template.Resources of type string
ERROR: json: cannot unmarshal array into Go struct field Function_Environment.Properties.Environment.Variables of type string
2021/06/07 16:01:51 [WARNING] There was an error processing the cloudformation template example_3.yml: json: cannot unmarshal array into Go struct field Template.Resources of type string
  __    __
  \ \  / /
   \ \/ /___  _  ____
    \  /  _ \| |/  __|
    | |  |_| |   /
    |_|\____/|__|v0.1.52
 Yor Findings Summary
 Scanned Resources:	  0
 New Resources Traced: 	  0
 Updated Resources:	  0

Cloudformation: yor overwrites existing tag values

Take this sample VPC CloudFormation template:

---
AWSTemplateFormatVersion: 2010-09-09

Parameters:
  EnvironmentName:
    Description: An environment name that is prefixed to resource names
    Type: String

  VpcCIDR:
    Description: Please enter the IP range (CIDR notation) for this VPC
    Type: String
    Default: 10.192.0.0/16

Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName
 
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

When running yor, it correctly adds tags, but it changes the Name tag (the !Ref is removed) :

$ yor tag -d .
[...]
$ cat vpc.yml
[...]
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: EnvironmentName
        - Key: yor_trace
          Value: def86a84-eff9-4054-a7cb-1890919a6bb5
[...]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.