Giter VIP home page Giter VIP logo

ecspresso's Introduction

ecspresso

ecspresso is a deployment tool for Amazon ECS.

(pronounced the same as "espresso" ☕)

Documents

Install

Homebrew (macOS and Linux)

$ brew install kayac/tap/ecspresso

asdf (macOS and Linux)

$ asdf plugin add ecspresso
# or
$ asdf plugin add ecspresso https://github.com/kayac/asdf-ecspresso.git

$ asdf install ecspresso 2.3.0
$ asdf global ecspresso 2.3.0

aqua (macOS and Linux)

aqua is a CLI version manager.

$ aqua g -i kayac/ecspresso

Binary packages

Releases

CircleCI Orbs

https://circleci.com/orbs/registry/orb/fujiwara/ecspresso

version: 2.1
orbs:
  ecspresso: fujiwara/[email protected]
jobs:
  install:
    steps:
      - checkout
      - ecspresso/install:
          version: v2.3.0 # or latest
          # version-file: .ecspresso-version
          os: linux # or windows or darwin
          arch: amd64 # or arm64
      - run:
          command: |
            ecspresso version

version: latest installs different versions of ecspresso for each Orb version.

Note: version: latest is not recommended as it may cause unexpected behavior when a new version of ecspresso is released.

Orb fujiwara/[email protected] supports version-file: path/to/file, which installs the ecspresso version specified in the file. This version number does not have a v prefix, For example, 2.0.0.

GitHub Actions

Action kayac/ecspresso@v2 installs an ecspresso binary for Linux(x86_64) into /usr/local/bin. This action runs install only.

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: kayac/ecspresso@v2
        with:
          version: v2.3.0 # or latest
          # version-file: .ecspresso-version
      - run: |
          ecspresso deploy --config ecspresso.yml

To use the latest version of ecspresso, pass the parameter "latest".

      - uses: kayac/ecspresso@v2
        with:
          version: latest

version: latest installs different versions of ecspresso for each Action version.

  • kayac/ecspresso@v1
    • The latest version of v1.x
  • kayac/ecspresso@v2
    • The latest version of v2.x

Note: version: latest is not recommended as it may cause unexpected behavior when a new version of ecspresso is released.

Action kayac/ecspresso@v2 supports version-file: path/to/file, which installs the ecspresso version specified in the file. This version number does not have a v prefix, For example 2.3.0.

Usage

Usage: ecspresso <command>

Flags:
  -h, --help                      Show context-sensitive help.
      --envfile=ENVFILE,...       environment files ($ECSPRESSO_ENVFILE)
      --debug                     enable debug log ($ECSPRESSO_DEBUG)
      --ext-str=KEY=VALUE;...     external string values for Jsonnet ($ECSPRESSO_EXT_STR)
      --ext-code=KEY=VALUE;...    external code values for Jsonnet ($ECSPRESSO_EXT_CODE)
      --config="ecspresso.yml"    config file ($ECSPRESSO_CONFIG)
      --assume-role-arn=""        the ARN of the role to assume ($ECSPRESSO_ASSUME_ROLE_ARN)
      --timeout=TIMEOUT           timeout. Override in a configuration file ($ECSPRESSO_TIMEOUT).
      --filter-command=STRING     filter command ($ECSPRESSO_FILTER_COMMAND)
      --[no-]color                enable colorized output ($ECSPRESSO_COLOR)

Commands:
  appspec
    output AppSpec YAML for CodeDeploy to STDOUT

  delete
    delete service

  deploy
    deploy service

  deregister
    deregister task definition

  diff
    show diff between task definition, service definition with current running
    service and task definition

  exec
    execute command on task

  init --service=SERVICE
    create configuration files from existing ECS service

  refresh
    refresh service. equivalent to deploy --skip-task-definition
    --force-new-deployment --no-update-service

  register
    register task definition

  render <targets>
    render config, service definition or task definition file to STDOUT

  revisions
    show revisions of task definitions

  rollback
    rollback service

  run
    run task

  scale
    scale service. equivalent to deploy --skip-task-definition
    --no-update-service

  status
    show status of service

  tasks
    list tasks that are in a service or having the same family

  verify
    verify resources in configurations

  wait
    wait until service stable

  version
    show version

For more options for sub-commands, See ecspresso sub-command --help.

Quick Start

ecspresso allows you to easily manage your existing/running ECS services by code.

Try ecspresso init for your ECS service with option --region, --cluster and --service.

$ ecspresso init --region ap-northeast-1 --cluster default --service myservice --config ecspresso.yml
2019/10/12 01:31:48 myservice/default save service definition to ecs-service-def.json
2019/10/12 01:31:48 myservice/default save task definition to ecs-task-def.json
2019/10/12 01:31:48 myservice/default save config to ecspresso.yml

Review the generated files: ecspresso.yml, ecs-service-def.json, and ecs-task-def.json.

Now you can deploy the service using ecspresso!

$ ecspresso deploy --config ecspresso.yml

Next step

ecspresso can read service and task definition files as a template. A typical use case is to replace the image's tag in the task definition file.

Modify ecs-task-def.json as below.

-  "image": "nginx:latest",
+  "image": "nginx:{{ must_env `IMAGE_TAG` }}",

Then, deploy the service with environment variable IMAGE_TAG.

$ IMAGE_TAG=stable ecspresso deploy --config ecspresso.yml

For more information, refer to the Configuration file and Template syntax sections.

Configuration file

A configuration file for ecspresso (YAML, JSON, or Jsonnet format).

region: ap-northeast-1 # or AWS_REGION environment variable
cluster: default
service: myservice
task_definition: taskdef.json
timeout: 5m # default 10m
ignore:
  tags:
    - ecspresso:ignore # ignore tags of service and task definition

ecspresso deploy works as below.

  • Register a new task definition from task-definition file (JSON or Jsonnet).
    • Replace {{ env `FOO` `bar` }} syntax in the JSON file with environment variable "FOO".
      • If "FOO" is not defined, replaced by "bar"
    • Replace {{ must_env `FOO` }} syntax in the JSON file wth environment variable "FOO".
      • If "FOO" is not defined, abort immediately.
  • Update service tasks by the service_definition file (JSON or Jsonnet).
  • Wait for the service to be stable.

Configuration files and task/service definition files are read by go-config which provides template functions env, must_env and json_escape.

Template syntax

ecspresso uses the text/template standard package in Go to render template files, and parses them as YAML or JSON.

When using Jsonnet, ecspresso first renders the Jsonnet files and then parses them as text/template. As a result, template functions can only render string values using "{{ ... }}", since the template function syntax {{ }} conflicts with Jsonnet syntax. To render non-string values, consider using Jsonnet functions instead.

By default, ecspresso provides the following as template functions.

env

"{{ env `NAME` `default value` }}"

This replaces the placeholder with the value of the environment variable NAME. If not set, it defaults to "default value".

must_env

"{{ must_env `NAME` }}"

This replaces the placeholder with the value of environment variable NAME. If not set, ecspresso will panic and stop forcefully.

Defining critical values with must_env helps prevent unintended deployments by ensuring these values are set before execution.

json_escape

"{{ must_env `JSON_VALUE` | json_escape }}"

This escapes values as JSON strings, which is useful for embedding values as strings that require escaping, such as quotes.

Plugin provided template functions

ecspresso also adds some template functions via plugins. See the Plugins section.

Example of deployment

Rolling deployment

$ ecspresso deploy --config ecspresso.yml
2017/11/09 23:20:13 myService/default Starting deploy
Service: myService
Cluster: default
TaskDefinition: myService:3
Deployments:
    PRIMARY myService:3 desired:1 pending:0 running:1
Events:
2017/11/09 23:20:13 myService/default Creating a new task definition by myTask.json
2017/11/09 23:20:13 myService/default Registering a new task definition...
2017/11/09 23:20:13 myService/default Task definition is registered myService:4
2017/11/09 23:20:13 myService/default Updating service...
2017/11/09 23:20:13 myService/default Waiting for service stable...(it will take a few minutes)
2017/11/09 23:23:23 myService/default  PRIMARY myService:4 desired:1 pending:0 running:1
2017/11/09 23:23:29 myService/default Service is stable now. Completed!

Blue/Green deployment (with AWS CodeDeploy)

ecspresso deploy can deploy services using the CODE_DEPLOY deployment controller. Configure ecs-service-def.json as follows.

{
  "deploymentController": {
    "type": "CODE_DEPLOY"
  },
  // ...
}

Important notes:

  • ecspresso does not create or modify any CodeDeploy resources. You must separately create an application and deployment group for your ECS service in CodeDeploy.
  • ecspresso automatically detects CodeDeploy deployment settings for the ECS service.
  • If there are numerous CodeDeploy applications, the API calls during this detection process may cause throttling. To mitigate this, specify the CodeDeploy application_name and deployment_group_name in the config file:
# ecspresso.yml
codedeploy:
  application_name: myapp
  deployment_group_name: mydeployment

ecspresso deploy creates a new deployment for CodeDeploy, and it continues on CodeDeploy.

$ ecspresso deploy --config ecspresso.yml --rollback-events DEPLOYMENT_FAILURE
2019/10/15 22:47:07 myService/default Starting deploy
Service: myService
Cluster: default
TaskDefinition: myService:5
TaskSets:
   PRIMARY myService:5 desired:1 pending:0 running:1
Events:
2019/10/15 22:47:08 myService/default Creating a new task definition by ecs-task-def.json
2019/10/15 22:47:08 myService/default Registering a new task definition...
2019/10/15 22:47:08 myService/default Task definition is registered myService:6
2019/10/15 22:47:08 myService/default desired count: 1
2019/10/15 22:47:09 myService/default Deployment d-XXXXXXXXX is created on CodeDeploy
2019/10/15 22:47:09 myService/default https://ap-northeast-1.console.aws.amazon.com/codesuite/codedeploy/deployments/d-XXXXXXXXX?region=ap-northeast-1

CodeDeploy appspec hooks can be defined in a config file. ecspresso automatically creates Resources and version elements in appspec on deployment:

cluster: default
service: test
service_definition: ecs-service-def.json
task_definition: ecs-task-def.json
appspec:
  Hooks:
    - BeforeInstall: "LambdaFunctionToValidateBeforeInstall"
    - AfterInstall: "LambdaFunctionToValidateAfterTraffic"
    - AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficStarts"
    - BeforeAllowTraffic: "LambdaFunctionToValidateBeforeAllowingProductionTraffic"
    - AfterAllowTraffic: "LambdaFunctionToValidateAfterAllowingProductionTraffic"

Scale out/in

To change the desired count of a service, specify scale --tasks.

$ ecspresso scale --tasks 10

scale command is equivalent to deploy --skip-task-definition --no-update-service.

Example of deploy

ecspresso can deploy a service using a service_definition JSON file.

$ ecspresso deploy --config ecspresso.yml
...
# ecspresso.yml
service_definition: service.json

service.json example:

{
  "role": "ecsServiceRole",
  "desiredCount": 2,
  "loadBalancers": [
    {
      "containerName": "myLoadbalancer",
      "containerPort": 80,
      "targetGroupArn": "arn:aws:elasticloadbalancing:[region]:[account-id]:targetgroup/{target-name}/201ae83c14de522d"
    }
  ]
}

Keys are in the same format as aws ecs describe-services output.

  • deploymentConfiguration
  • launchType
  • loadBalancers
  • networkConfiguration
  • placementConstraint
  • placementStrategy
  • role
  • etc.

Example of run task

$ ecspresso run --config ecspresso.yml --task-def=db-migrate.json

If --task-def is not set, ecspresso will use the task definition included in the service.

Other options for RunTask API are set by service attributes (CapacityProviderStrategy, LaunchType, PlacementConstraints, PlacementStrategy and PlatformVersion).

Notes

Version constraint

required_version in the configuration file is for fixing the version of ecspresso.

required_version: ">= 2.0.0, < 3"

This allows ecspresso to execute if the version is greater than or equal to 2.0.0 and less than 3. If the version does not fall within this range, execution will fail.

This feature is implemented by go-version.

Manage Application Auto Scaling

For ECS services using Application Auto Scaling, adjusting the minimum and maximum auto-scaling settings with the ecspresso scale command is a breeze. Simply specify either scale --auto-scaling-min or scale --auto-scaling-max to modify the settings.

$ ecspresso scale --tasks 5 --auto-scaling-min 5 --auto-scaling-max 20

ecspresso deploy and scale can suspend and resume application auto scaling.

  • --suspend-auto-scaling sets suspended state to true.
  • --resume-auto-scaling sets suspended state to false.

To change the suspended state, simply use ecspresso scale --suspend-auto-scaling or ecspresso scale --resume-auto-scaling. These commands will only change the suspended state without affecting other settings.

Use Jsonnet instead of JSON and YAML.

ecspresso supports the Jsonnet file format.

  • v1.7 and later: Jsonnet support for service and task definitions
  • v2.0 and later: Jsonnet support for the configuration file
  • v2.4 and later: supports Jsonnet functions

If a file has the .jsonnet extension, ecspresso will proceed in the following order:

  1. process it as Jsonnet
  2. convert it to JSON
  3. load it with evaluation template syntax.

Using Template syntax in Jsonnet files may lead to syntax errors due to conflicts with Jsonnet syntax. In such cases, consider using Jsonnet functions instead.

{
  cluster: 'default',
  service: 'myservice',
  service_definition: 'ecs-service-def.jsonnet',
  task_definition: 'ecs-task-def.jsonnet',
}

ecspresso includes github.com/google/go-jsonnet as a library, so a separate installation of jsonnet is not needed.

--ext-str and --ext-code flag sets Jsonnet External Variables.

$ ecspresso --ext-str Foo=foo --ext-code "Bar=1+1" ...
{
  foo: std.extVar('Foo'), // = "foo"
  bar: std.extVar('Bar'), // = 2
}

Jsonnet functions

v2.4 and later supports Jsonnet native functions in Jsonnet files.

In the .jsonnet file,:

  1. Define local func = std.native('func');
  2. Use func()

Jsonnet functions are evaluated when rendering Jsonnet files, which helps avoid conflicts with template syntax.

env, must_env

env and must_env functions work the similary to template functions in JSON and YAML files. However, unlike template functions, Jsonnet functions are capable of rendering non-string values from environment variables using std.parseInt(), std.parseJson(), etc.

local env = std.native('env');
local must_env = std.native('must_env');
{
  foo: env('FOO', 'default value'),
  bar: must_env('BAR'),
  bazNumber: std.parseInt(env('BAZ_NUMBER', '0')),
  booBool: std.parseJson(env('BOO_BOOL', 'false')),
}

Other plugin-provided functions

See Plugins section.

Deploy to Fargate

When deploying services to Fargate, both task definitions and service definitions require specific settings.

For task definitions,

  • requiresCompatibilities (requires "FARGATE")
  • networkMode (requires "awsvpc")
  • cpu (required)
  • memory (required)
  • executionRoleArn (optional)
{
  "taskDefinition": {
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
      "FARGATE"
    ],
    "cpu": "1024",
    "memory": "2048",
    // ...
}

For service-definitions,

  • launchType (requires "FARGATE")
  • networkConfiguration (requires "awsvpcConfiguration")
{
  "launchType": "FARGATE",
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "subnets": [
        "subnet-aaaaaaaa",
        "subnet-bbbbbbbb"
      ],
      "securityGroups": [
        "sg-11111111"
      ],
      "assignPublicIp": "ENABLED"
    }
  },
  // ...
}

Fargate Spot support

  1. Set capacityProviders and defaultCapacityProviderStrategy for the ECS cluster.
  2. To migrate an existing service to use Fargate Spot, define capacityProviderStrategy in the service definition as shown below. Use ecspresso deploy --update-service to apply the settings to the service.
{
  "capacityProviderStrategy": [
    {
      "base": 1,
      "capacityProvider": "FARGATE",
      "weight": 1
    },
    {
      "base": 0,
      "capacityProvider": "FARGATE_SPOT",
      "weight": 1
    }
  ],
  // ...

ECS Service Connect support

ecspresso supports ECS Service Connect.

To configure, define serviceConnectConfiguration in service definitions and portMappings in task definitions.

For more details, see Service Connect parameters

EBS Volume support

ecspresso supports Amazon EBS Volumes.

To configure, define volumeConfigurations in service definitions, and mountPoints and volumes in task definitions.

// ecs-service-def.json
  "volumeConfigurations": [
    {
      "managedEBSVolume": {
        "filesystemType": "ext4",
        "roleArn": "arn:aws:iam::123456789012:role/ecsInfrastructureRole",
        "sizeInGiB": 10,
        "tagSpecifications": [
          {
            "propagateTags": "SERVICE",
            "resourceType": "volume"
          }
        ],
        "volumeType": "gp3"
      },
      "name": "ebs"
    }
  ]
// ecs-task-def.json
// containerDefinitions[].mountPoints
      "mountPoints": [
        {
          "containerPath": "/mnt/ebs",
          "sourceVolume": "ebs"
        }
      ]
// volumes
  "volumes": [
    {
      "name": "ebs",
      "configuredAtLaunch": true
    }
  ]

ecspresso run command supports EBS volumes too.

By default, EBS volumes attached to standalone tasks are deleted when the task stops. Use the --no-ebs-delete-on-termination option to preserve volumes.

$ ecspresso run --no-ebs-delete-on-termination

For tasks run by ECS services, EBS volumes are always deleted when the task stops. This is an ECS specification that ecspresso cannot override.

How to check diff and verify service/task definitions before deploy.

ecspresso supports diff and verify commands.

diff

Shows differences between local task/service definitions and remote (on ECS) definitions.

$ ecspresso diff
--- arn:aws:ecs:ap-northeast-1:123456789012:service/ecspresso-test/nginx-local
+++ ecs-service-def.json
@@ -38,5 +38,5 @@
   },
   "placementConstraints": [],
   "placementStrategy": [],
-  "platformVersion": "1.3.0"
+  "platformVersion": "LATEST"
 }
 
--- arn:aws:ecs:ap-northeast-1:123456789012:task-definition/ecspresso-test:202
+++ ecs-task-def.json
@@ -1,6 +1,10 @@
 {
   "containerDefinitions": [
     {
       "cpu": 0,
       "environment": [],
       "essential": true,
-      "image": "nginx:latest",
+      "image": "nginx:alpine",
       "logConfiguration": {
         "logDriver": "awslogs",
         "options": {

v2.4 or later, ecspresso diff --external can invoke an external command. You can use the "diff" command you like.

For example, use difftastic (difft) command.

$ ecspresso diff --external "difft --color=always"

$ ECSPRESSO_DIFF_COMMAND="difft --color=always" ecspresso diff

The command should exit with status 0. If it exits with a non-zero status when two files differ (for example, diff(1)), you need to write a wrapper command.

verify

Verify resources related with service/task definitions.

For example it checks if,

  • An ECS cluster exists.
  • The target groups in service definitions match the container name and port defined in the definitions.
  • A task role and a task execution role exist and can be assumed by ecs-tasks.amazonaws.com.
  • Container images exist at the URL defined in task definitions. (Checks only for ECR or DockerHub public images.)
  • Secrets in task definitions exist and are readable.
  • Log streams can be created and messages can be put into the specified CloudWatch log groups streams.

ecspresso verify tries to assume the task execution role defined in task definitions to verify these items. If it fails to assume the role, it continues to verify with the current session.

$ ecspresso verify
2020/12/08 11:43:10 nginx-local/ecspresso-test Starting verify
  TaskDefinition
    ExecutionRole[arn:aws:iam::123456789012:role/ecsTaskRole]
    --> [OK]
    TaskRole[arn:aws:iam::123456789012:role/ecsTaskRole]
    --> [OK]
    ContainerDefinition[nginx]
      Image[nginx:alpine]
      --> [OK]
      LogConfiguration[awslogs]
      --> [OK]
    --> [OK]
  --> [OK]
  ServiceDefinition
  --> [OK]
  Cluster
  --> [OK]
2020/12/08 11:43:14 nginx-local/ecspresso-test Verify OK!

Manipulate ECS tasks

ecspresso can manipulate ECS tasks using the tasks and exec commands.

After v2.0, These operations are provided by ecsta as a library. The ecsta CLI can manipulate any ECS tasks, not limited to those deployed by ecspresso.

Consider using ecsta as a CLI command.

tasks

The task command lists tasks run by a service or having the same family to a task definition.

Flags:
      --id=                       task ID
      --output=table              output format
      --find=false                find a task from tasks list and dump it as JSON
      --stop=false                stop the task
      --force=false               stop the task without confirmation
      --trace=false               trace the task

The --find option enables task selection rom a list and displays it as JSON.

The ECSPRESSO_FILTER_COMMAND environment variable can be set to specify a command for filtering tasks, such as peco, fzf, etc.

$ ECSPRESSO_FILTER_COMMAND=peco ecspresso tasks --find

The --stop option allows for task selecttion and stopping from a list.

exec

The exec command executes a command on a task.

session-manager-plugin is required in PATH.

Flags:
      --id=                       task ID
      --command=sh                command to execute
      --container=                container name
      --port-forward=false        enable port forward
      --local-port=0              local port number
      --port=0                    remote port number (required for --port-forward)
      --host=                     remote host (required for --port-forward)
      -L                          short expression of local-port:host:port

If --id is not set, the command shows a list of tasks to select a task to execute.

The ECSPRESSO_FILTER_COMMAND environment variable works the same as with the tasks command.

See also the official documentation Using Amazon ECS Exec for debugging.

port forwarding

The --port-forward option enables port forwarding from a local port to an ECS task's port.

$ ecspresso exec --port-forward --port 80 --local-port 8080
...

If --id is not set, the command shows a list of tasks to select for port forwarding.

When --local-port is not specified, an ephemeral port is used as the local port.

The -L option is a short expression for local-port:host:port. For example, -L 8080:example.com:80 is equivalent to --local-port 8080 --host example.com --port 80.

$ ecspresso exec --port-forward -L 8080:example.com:80

Plugins

ecspresso supports plugins to extend template functions and Jsonnet native functions.

tfstate

The tfstate plugin introduces the tfstate and tfstatef template functions.

ecspresso.yml

region: ap-northeast-1
cluster: default
service: test
service_definition: ecs-service-def.json
task_definition: ecs-task-def.json
plugins:
  - name: tfstate
    config:
      url: s3://my-bucket/terraform.tfstate
      # or path: terraform.tfstate    # path to local file

ecs-service-def.json

{
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "subnets": [
        "{{ tfstatef `aws_subnet.private['%s'].id` `az-a` }}"
      ],
      "securityGroups": [
        "{{ tfstate `data.aws_security_group.default.id` }}"
      ]
    }
  }
}

{{ tfstate "resource_type.resource_name.attr" }} will expand to the attribute value of the resource in tfstate.

{{ tfstatef "resource_type.resource_name['%s'].attr" "index" }} is similar to {{ tfstatef "resource_type.resource_name['index'].attr" }}. This function is useful for build a resource addresses with environment variables.

{{ tfstatef `aws_subnet.ecs['%s'].id` (must_env `SERVICE`) }}

tfstate Jsonnet function

tfstate Jsonnet function works the same as template function in JSON and YAML files. tfstatef Jsonnet function is not provided. Use std.format() or interpolation instead.

local tfstate = std.native('tfstate');
{
  networkConfiguration: {
    awsvpcConfiguration: {
      subnets: [
        tfstate('aws_subnet.private["%s"].id' % 'az-z'),
        tfstate(std.format('aws_subnet.private["%s"].id', 'az-b')),
      ],
      securityGroups: [
        tfstate('data.aws_security_group.default.id'),
      ]
    }
  }
}

Supported tfstate URL formats

  • Local file file://path/to/terraform.tfstate
  • HTTP/HTTPS https://example.com/terraform.tfstate
  • Amazon S3 s3://{bucket}/{key}
  • Terraform Cloud remote://api.terraform.io/{organization}/{workspaces}
    • TFE_TOKEN environment variable is required.
  • Google Cloud Storage gs://{bucket}/{key}
  • Azure Blog Storage azurerm://{resource_group_name}/{storage_account_name}/{container_name}/{blob_name}

This plugin uses tfstate-lookup to load tfstate.

Multiple tfstate support

func_prefix adds prefixes to template function names for each plugin configuration, enabling support for multiple tfstate files.

# ecspresso.yml
plugins:
   - name: tfstate
     config:
       url: s3://tfstate/first.tfstate
     func_prefix: first_
   - name: tfstate
     config:
       url: s3://tfstate/second.tfstate
     func_prefix: second_

In templates, functions are called with the specified prefixes.

[
  "{{ first_tfstate `aws_s3_bucket.main.arn` }}",
  "{{ second_tfstate `aws_s3_bucket.main.arn` }}"
]

Similar features are also supported for Jsonnet.

local first_tfstate = std.native('first_tfstate'); // func_prefix: first_
local second_tfstate = std.native('second_tfstate'); // func_prefix: second_
[
  first_tfstate('aws_s3_bucket.main.arn'),
  second_tfstate('aws_s3_bucket.main.arn'),
]

CloudFormation

The cloudformation plugin introduces the cfn_output and cfn_export template functions.

An example of a CloudFormation stack template defining Outputs and Exports.

# StackName: ECS-ecspresso
Outputs:
  SubnetAz1:
    Value: !Ref PublicSubnetAz1
  SubnetAz2:
    Value: !Ref PublicSubnetAz2
  EcsSecurityGroupId:
    Value: !Ref EcsSecurityGroup
    Export:
      Name: !Sub ${AWS::StackName}-EcsSecurityGroupId

Load the cloudformation plugin in a config file.

ecspresso.yml

# ...
plugins:
  - name: cloudformation

cfn_output StackName OutputKey looks up the OutputValue of OutputKey in the StackName. cfn_export ExportName looks up the exported value by name.

ecs-service-def.json

{
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "subnets": [
        "{{ cfn_output `ECS-ecspresso` `SubnetAz1` }}",
        "{{ cfn_output `ECS-ecspresso` `SubnetAz2` }}"
      ],
      "securityGroups": [
        "{{ cfn_export `ECS-ecspresso-EcsSecurityGroupId` }}"
      ]
    }
  }
}

Jsonnet functions cfn_output, cfn_export

Similar features are also supported for Jsonnet.

local cfn_output = std.native('cfn_output');
local cfn_export = std.native('cfn_export');
{
  subnets: [
    cfn_output('ECS-ecspresso', 'SubnetAz1'),
    cfn_output('ECS-ecspresso', 'SubnetAz2'),
  ],
  securityGroups: [
    cfn_export('ECS-ecspresso-EcsSecurityGroupId'),
  ],
}

SSM Parameter Store lookups

The ssm template function reads parameters from AWS Systems Manager (SSM) Parameter Store.

Given SSM Parameter Store has the following parameters:

  • name: '/path/to/string', type: String, value: "ImString"
  • name: '/path/to/stringlist', type: StringList, value: "ImStringList0,ImStringList1"
  • name: '/path/to/securestring', type: SecureString, value: "ImSecureString"

This template,

{
  "string": "{{ ssm `/path/to/string` }}",
  "stringlist": "{{ ssm `/path/to/stringlist` 1 }}",  *1
  "securestring": "{{ ssm `/path/to/securestring` }}"
}

will be rendered as:

{
  "string": "ImString",
  "stringlist": "ImStringList1",
  "securestring": "ImSecureString"
}

Jsonnet functions ssm, ssm_list

The ssm function works the same as template function. For string list parameters, use ssm_list to specify the index.

local ssm = std.native('ssm');
local ssm_list = std.native('ssm_list');
{
  string: ssm('/path/to/string'),
  stringlist: ssm_list('/path/to/stringlist', 1),
  securestring: ssm('/path/to/securestring'),
}

Resolve secretsmanager secret ARN

The secretsmanager_arn template function resolves the Secrets Manager secret ARN by secret name.

  "secrets": [
    {
      "name": "FOO",
      "valueFrom": "{{ secretsmanager_arn `foo` }}"
    }
  ]

will be rendered as:

  "secrets": [
    {
      "name": "FOO",
      "valueFrom": "arn:aws:secretsmanager:ap-northeast-1:123456789012:secret:foo-06XQOH"
    }
  ]

Jsonnet function secretsmanager_arn

The secretsmanager_arn function works the same as template function.

local secretsmanager_arn = std.native('secretsmanager_arn');
{
  secrets: [
    {
      name: "FOO",
      valueFrom: secretsmanager_arn('foo'),
    }
  ]
}

LICENSE

MIT

Author

KAYAC Inc.

ecspresso's People

Contributors

aereal avatar ch1aki avatar cohalz avatar dependabot-preview[bot] avatar dependabot[bot] avatar dmnlk avatar ebi-yade avatar enm10k avatar enokawa avatar fnaoto avatar fujiwara avatar goruha avatar handlename avatar hirofumi-narita avatar ijin avatar koluku avatar komapotter avatar konoui avatar m22r avatar mashiike avatar nao23 avatar shogo82148 avatar songmu avatar testwill avatar tksx1227 avatar tkuchiki avatar tomiyan avatar uchida avatar yowatari avatar yukitsugu-tsukamoto avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ecspresso's Issues

Your .dependabot/config.yml contained invalid details

Dependabot encountered the following error when parsing your .dependabot/config.yml:

The property '#/update_configs/0/package_manager' value "go" did not match one of the following values: javascript, ruby:bundler, php:composer, java:maven, elixir:hex, rust:cargo, java:gradle, dotnet:nuget, go:dep, go:modules, docker, elm, submodules, github_actions, python, terraform

Please update the config file to conform with Dependabot's specification using our docs and online validator.

Feature Request: verify container image platform

Thank you for making a great tool!
I would like to propose a platform verify feature.

Background

Currently, ECS provides several runtimes.
e.g.
Fargate/EC2 with AMD64/Linux
Fargate/EC2 with ARM64/Linux
Fargate/EC2 with AMD64/Windows

The feature is to verify whether images in task definition support target platform or not.
For example, there is a following task definition.

{
    "containerDefinitions": [
        {
            "name": "custom-image"
        }
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "networkMode": "awsvpc",
    "runtimePlatform": {
        "operatingSystemFamily": "LINUX",
        "cpuArchitecture": "ARM64"
    }
}

When custom-image does not support arm64/linux platform, ECS task will fail to start due to incompatible platform.
We need to start tasks and check container logs to be aware of failures.

I will try to implement it and create PR.

timeoutを設定したが必ず10分で失敗してしまう

config fileでtimeout: 15mを設定したが、10分後にerrorになって失敗してしまうのですが、なぜでしょうか。error文は下記になります。

2019/08/28 06:32:18 deploy FAILED. deploy failed: ResourceNotReady: exceeded wait attempts
Build step 'Execute shell' marked build as failure

`ecspresso deploy` does not seem to succeed in v1.7.4

ecspresso deploy, which was successful in v1.7.3, does not seem to succeed in v1.7.4 with the following error:

2021/12/25 18:35:43 my-fargate-service/my-fargate Starting deploy
Service: my-fargate-service
Cluster: my-fargate
TaskDefinition: my-task:77
Deployments:
   PRIMARY my-task:77 desired:3 pending:0 running:3
Events:
2021/12/25 18:35:44 my-fargate-service/my-fargate Registering a new task definition...
2021/12/25 18:35:44 my-fargate-service/my-fargate Task definition is registered my-task:78
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1b71cc4]

goroutine 1 [running]:
github.com/kayac/ecspresso.(*App).Deploy(0xc000580c00, 0xc000037ab5, 0xc000037ab8, 0xc000037ac0, 0xc000037ac1, 0xc000037ac2, 0x0, 0xc00020ae80, 0xc000037ac4, 0xc000037ac5, ...)
        /home/runner/work/ecspresso/ecspresso/deploy.go:81 +0xa24
main._main(0xc000100058)
        /home/runner/work/ecspresso/ecspresso/cmd/ecspresso/main.go:250 +0x8e6e
main.main()
        /home/runner/work/ecspresso/ecspresso/cmd/ecspresso/main.go:18 +0x25

Probably due to #365.

rollback - support rollback to older tasks definitions

It would be very nice to have the ability to a specific version and not just for the last one.
the reason is that if you know the last version also has issued and you wish to rollback to version X-5 you will have to run rollback 5 times.

Amazon ECR public image [NG] verify with 401 Unauthorized.

前日リリースされたAmazon ECR Public Galleryのイメージを指定した task definition を、ECRに未ログインで verify したところ 401 Unauthorized で NG となりました。

期待する動作はECRへの未ログイン時にECR Publicイメージへは匿名アクセスで存在確認が行われる、または verifySkipErr などでverify NGとはならない形になります。

NG時の出力:

TaskDefinition [NG] verify ContainerDefinition[mackerel-container-agent] failed: verify Image[public.ecr.aws/mackerel/mackerel-container-agent:plugins] failed: 401 Unauthorized

イメージをDocker Hubのものに変えただけの差分でのOK出力:

ContainerDefinition[mackerel-container-agent]
Image[mackerel/mackerel-container-agent:plugins]
--> [OK]

public.ecr.aws/mackerel/mackerel-container-agent:plugins は手元のマシンで未認証で docker pull 可能なことを確認しました。

Upgrade AWS-SDK-Go version to support SSO Login.

Please consider to upgrade aws-sdk-go version.
AWS Single Sign-On (SSO) service was added to v1.37.0.
aws/aws-sdk-go#3755

Currently ecspresso cannot get credentials from aws sso login command.

command example.

$ ecspresso version
ecspresso v1.3.2

$ aws sso login --profile my-sso-profile
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:

https://device.sso.us-east-1.amazonaws.com/

Then enter the code:

ABCD-EFGH
Successully logged into Start URL: https://example.com/start

$ export AWS_PROFILE=my-sso-profile
$ ecspresso init --config config.yml --region ap-northeast-1 --cluster hoge-cluster --service hoge-service
2021/02/02 16:15:34 init FAILED. failed to describe service: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Feature Request: Support --envfile option

I have multiple deploy destinations.

I have prepared one ecspresso configuration file group.
When run ecspresso xxx, I use the environment variables for each environment.

like this:

# for env HOGE
source HOGE.env
ecspresso verify --config=config.yaml && ecspresso diff --config=config.yaml
ecspreoss deploy --config=config.yaml

# for env FUGA
source FUGA.env
ecspresso verify --config=config.yaml && ecspresso diff --config=config.yaml
ecspreoss deploy --config=config.yaml

However, the above command overwrites the login shell environment variables.
If ecspresso supports the --envfile option, it will prevent overwriting them.

# for env HOGE
ecspresso verify --envfile=HOGE.env --config=config.yaml && ecspresso diff --envfile=HOGE.env --config=config.yaml
ecspreoss deploy --envfile=HOGE.env --config=config.yaml

There are workarounds such as the following, but this is a little complicated.

# for env HOGE
(source HOGE.env && ecspresso verify --config=config.yaml && ecspresso diff --config=config.yaml)
(source HOGE.env && ecspreoss deploy --config=config.yaml)

Furthermore, if supports multiple --envfile options, it will be more useful.

ecspreoss deploy --envfile=HOGE_MAIN.env --envfile=HOGE_SIDECAR.env --config=config.yaml

wait command does not supported when the deploymentController is CODE_DEPLOY?

I tried ecspresso wait command but doesn't work.
Code Deploy Deployment was Succeeded.

$ ecspresso deploy --config config.yaml --no-wait
2020/04/01 21:13:12 sample/sample-cluster Starting deploy
Service: sample
Cluster: sample-cluster
TaskDefinition: sample-app:98
TaskSets:
   PRIMARY sample-app:98 desired:2 pending:0 running:2
Events:
2020/04/01 21:13:12 sample/sample-cluster Creating a new task definition by ecs-task-def.json
2020/04/01 21:13:12 sample/sample-cluster Registering a new task definition...
2020/04/01 21:13:12 sample/sample-cluster Task definition is registered sample-app:101
2020/04/01 21:13:13 sample/sample-cluster Deployment d-XXXXXXXXX is created on CodeDeploy:
2020/04/01 21:13:13 sample/sample-cluster https://ap-northeast-1.console.aws.amazon.com/codesuite/codedeploy/deployments/d-XXXXXXXXX?region=ap-northeast-1
$
$ ecspresso wait --config config.yaml
2020/04/01 21:13:23 sample/sample-cluster Waiting for the service stable
2020/04/01 21:13:23 sample/sample-cluster Waiting for service stable...(it will take a few minutes)
2020/04/01 21:16:18 (service sample) has reached a steady state.27906806) updated state to
2020/04/01 21:16:08 (service sample, taskSet ecs-svc/9666680786127906806) updated state to
 STEADY_STATE.
2020/04/01 21:15:58 (service sample, taskSet ecs-svc/9666680786127906806) has stopped 2 ru
nning tasks: (task 496fcdf75dd84b86a2151d5879d89fe0) (task 45c3d01e80a24f51baeda1f51d25049
d).
2020/04/01 21:15:58 (service sample, taskSet ecs-svc/1043283654285067039) updated state to
 STEADY_STATE.
2020/04/01 21:15:48 (service sample, taskSet ecs-svc/9666680786127906806) has begun draini
ng connections on 2 tasks.
2020/04/01 21:15:48 (service sample) deregistered 2 targets in (target-group arn:aws:elast
icloadbalancing:ap-northeast-1:012345678901:targetgroup/sample-alb-01-tg-02/4e53119caa016b
82)
2020/04/01 21:15:48 (service sample, taskSet ecs-svc/1043283654285067039) updated state to
 STABILIZING.
2020/04/01 21:15:48 (service sample) updated computedDesiredCount for taskSet ecs-svc/9666
680786127906806 to 0.
2020/04/01 21:14:25 (service sample) has reached a steady state.
2020/04/01 21:23:23 wait FAILED. the service still unstable: RequestCanceled: waiter context canceled

It worked fine when the deploymentController is ECS.

$ ecspresso deploy --config config.yaml --no-wait
2020/04/01 21:06:00 sample-app/sample-cluster Starting deploy
Service: sample-app
Cluster: sample-cluster
TaskDefinition: sample-app:99
Deployments:
   PRIMARY sample-app:99 desired:1 pending:0 running:1
Events:
2020/04/01 21:06:02 sample-app/sample-cluster Creating a new task definition by ecs-task-def.json
2020/04/01 21:06:02 sample-app/sample-cluster Registering a new task definition...
2020/04/01 21:06:02 sample-app/sample-cluster Task definition is registered sample-app:100
2020/04/01 21:06:02 sample-app/sample-cluster Updating service tasks...
2020/04/01 21:06:05 sample-app/sample-cluster Service is deployed.
$
$ ecspresso wait --config config.yaml
2020/04/01 21:06:15 sample-app/sample-cluster Waiting for the service stable
2020/04/01 21:06:15 sample-app/sample-cluster Waiting for service stable...(it will take a few minutes)
2020/04/01 21:08:16 sample-app/sample-cluster  PRIMARY sample-app:100 desired:1 pending:0 running:1
2020/04/01 21:07:54 (service sample-app) has stopped 1 running tasks: (task c073aeb0f71148
969b477faf66faed83).
2020/04/01 21:08:20 sample-app/sample-cluster Service is stable now. Completed!

proposal: enable to deploy new service

Thank you for great tool!

Background

Being able to manage an existing app by a code base is very useful and easily, but on the other hand,
I often want to deploy new service from existing conf files with same tool (= ecspresso).

For example, we have a situation where we want to create a dev, stg, prd env based on the conf files which we we have used in PoC env.

Suggestion

I want to make ecspresso deploy enable the deployment of new services in addition to existing features.
Alternatively, this could be accomplished by passing a flag (such as --new) to deploy subcommand.


I apologize if it's already possible or if I've made suggestions that are outside of the ecspresso's philosophy.

[Feature Request] Unlimited terminal width

My problem

Copying the event log from Terminal is inconvenient because the Terminal width is fixed at 90.
スクリーンショット 2021-10-16 14 00 47

Feature Request

Would you please add "Unlimited terminal width mode" with no line breaks in the event message?

emit warning messages into stderr instead of stdout to make JSON/TSV outputs parseable

Currently, ecspresso emits warning messages into stdout such as:
https://github.com/kayac/ecspresso/blob/v1/verify.go#L226-L230

This causes breaking structured outputs.

For example, running ecspresso tasks --output=json and gets the below outputs from stdout:

# ecspresso tasks --output=json
WARNING: json: unknown field "$comment" in etc/ecspresso/production/task-definition.json
{
  "attachments": [
    {
      "details": [
      # snip

But those are invalid as either JSON or JSON lines.

ecspresso using fmt.Print* functions to emit warning messages but those functions use os.Stdout to output destination.

When emitting warning or error messages, the stderr may be preferred.

When run `deploy` command, throttling exception happened

Hi, I use ecspresso with Codedeploy.
My environments have many deployment group, applications.

When deploying multiple services at the same time, throttling exception happend.

I'd like to fix this issue.

version

$ ecspresso version
ecspresso v1.7.7

problem

I think it's root cause .

ecspresso/deploy.go

Lines 242 to 305 in 7ec5449

func (d *App) findDeploymentInfo() (*codedeploy.DeploymentInfo, error) {
// search deploymentGroup in CodeDeploy
d.DebugLog("find all applications in CodeDeploy")
la, err := d.codedeploy.ListApplications(&codedeploy.ListApplicationsInput{})
if err != nil {
return nil, err
}
if len(la.Applications) == 0 {
return nil, errors.New("no any applications in CodeDeploy")
}
// BatchGetApplications accepts applications less than 100
for i := 0; i < len(la.Applications); i += 100 {
end := i + 100
if end > len(la.Applications) {
end = len(la.Applications)
}
apps, err := d.codedeploy.BatchGetApplications(&codedeploy.BatchGetApplicationsInput{
ApplicationNames: la.Applications[i:end],
})
if err != nil {
return nil, err
}
for _, info := range apps.ApplicationsInfo {
d.DebugLog("application", info.String())
if *info.ComputePlatform != "ECS" {
continue
}
lg, err := d.codedeploy.ListDeploymentGroups(&codedeploy.ListDeploymentGroupsInput{
ApplicationName: info.ApplicationName,
})
if err != nil {
return nil, err
}
if len(lg.DeploymentGroups) == 0 {
d.DebugLog("no deploymentGroups in application", *info.ApplicationName)
continue
}
groups, err := d.codedeploy.BatchGetDeploymentGroups(&codedeploy.BatchGetDeploymentGroupsInput{
ApplicationName: info.ApplicationName,
DeploymentGroupNames: lg.DeploymentGroups,
})
if err != nil {
return nil, err
}
for _, dg := range groups.DeploymentGroupsInfo {
d.DebugLog("deploymentGroup", dg.String())
for _, ecsService := range dg.EcsServices {
if *ecsService.ClusterName == d.config.Cluster && *ecsService.ServiceName == d.config.Service {
return &codedeploy.DeploymentInfo{
ApplicationName: aws.String(*info.ApplicationName),
DeploymentGroupName: aws.String(*dg.DeploymentGroupName),
DeploymentConfigName: aws.String(*dg.DeploymentConfigName),
}, nil
}
}
}
}
}
return nil, fmt.Errorf(
"failed to find CodeDeploy Application/DeploymentGroup for ECS service %s on cluster %s",
d.config.Service,
d.config.Cluster,
)
}

findDeploymentInfo() take applicationName/deploymentconfigName/deploymentGroupName dynamically.

Therefore, I think that it call a lot of APIs in my environment and exceeded the API rate limit.

Below when error happened.

# log
2022/04/13 09:10:14 test-ecsporesso/test-ecspresso-cluster Starting deploy
Service: test-ecsporesso
Cluster: test-ecspresso-cluster
TaskDefinition: test-ecsporesso-api:85
TaskSets:
   PRIMARY test-ecsporesso-api:85 desired:1 pending:0 running:0
    ACTIVE test-ecsporesso-api:86 desired:1 pending:0 running:0
AutoScaling:
  Capacity min:1 max:1
  Suspended in:false out:false scheduled:false
  Policy name:test-ecsporesso-policy type:TargetTrackingScaling
Events:
2022/04/13 09:10:14 test-ecsporesso/test-ecspresso-cluster Registering a new task definition...
2022/04/13 09:10:14 test-ecsporesso/test-ecspresso-cluster Task definition is registered test-ecsporesso-api:88
2022/04/13 09:10:14 test-ecsporesso/test-ecspresso-cluster Updating service attributes...
2022/04/13 09:10:17 test-ecsporesso/test-ecspresso-cluster desired count: unchanged
2022/04/13 09:10:34 deploy FAILED. ThrottlingException: Rate exceeded

My environments

  • Applications have 70 over
  • Deployment Group have one every application(Applicatons : Deployment Group = 1:1 )

Proposal

  • Add new deployment_definition .
cluster: default
service: test
service_definition: ecs-service-def.json
task_definition: ecs-task-def.json
deployment_definition: ecs-deployment-def.json
{
  "applicationName": "test-app",
  "deploymentConfigName": "CodeDeployDefault.ECSAllAtOnce",
  "deploymentGroupName": "test-deployment-group"
}

I think applicationName/deploymentconfigName/deploymentGroupName take deployment_definition statically.

What do you think?

[v0.99.6] --tasks を 1 -> 2 に増やす deploy が Fail (管理コンソール上はOK)

"schedulingStrategy": "REPLICA" (アプリケーション)と "DAEMON" (Datadog) のサービスが
同居(同時稼働)する EC2 インスタンス 2台(サービス2つずつ)に対して、

  1. DAEMON 側へ ecspresso deploy --tasks 1 を実行 -> 正常終了するが desiredCount 2 のまま
  2. REPLICA 側へ ecspresso deploy --tasks 1 を実行 -> 正常終了して desiredCount 1 になった(DAEMON側は2のまま)
  3. REPLICA 側へ ecspresso deploy --tasks 2 を実行 -> 画面には Fail したと表示されるが、管理コンソール上では desiredCount が 2 となっていて期待結果になったように見える

画面に表示されたメッセージを可能な限り記述します。
使用が推奨できない環境に該当するようにも思われるのですが、今回の状況が
表示のみの問題なのか、何か確認しなければならない懸念事項が残っている状態なのかを知りたいです。

❯ ecspresso deploy --config app-green/app-green/app-green-config.yaml --tasks 2 --debug

2020/09/28 18:40:01 app-green/app-green Starting deploy
Service: app-green
Cluster: app-green
TaskDefinition: app-green:13
Deployments:
   PRIMARY app-green:13 desired:1 pending:0 running:1
Events:
2020/09/28 18:40:02 app-green/app-green Registering a new task definition...
2020/09/28 18:40:02 app-green/app-green Task definition is registered app-green:14
2020/09/28 18:40:02 app-green/app-green service attributes will not change
2020/09/28 18:40:02 app-green/app-green desired count: 2
2020/09/28 18:40:02 app-green/app-green Updating service tasks...
2020/09/28 18:40:02 app-green/app-green {
  Cluster: "app-green/app-green",
  DesiredCount: 2,
  ForceNewDeployment: false,
  Service: "app-green/app-green",
  TaskDefinition: "arn:aws:ecs:ap-southeast-1:XXXXXXXXXXX:task-definition/app-green:14"
}
2020/09/28 18:40:06 app-green/app-green Waiting for service stable...(it will take a few minutes)
2020/09/28 18:49:56 app-green/app-green  PRIMARY app-green:14 desired:2 pending:0 running:1
2020/09/28 18:49:56 app-green/app-green   ACTIVE app-green:13 desired:1 pending:0 running:1
2020/09/28 18:40:38 (service app-green) was unable to place a task because no container instance met all of its requirements. The closest matching (container-instance xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx) has insufficient CPU units available. For more information, see the Troubleshooting section of the Amazon ECS Developer Guide.
2020/09/28 18:40:29 (service app-green) has started 1 tasks: (task xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).
2020/09/28 18:50:01 deploy FAILED. failed to wait service stable: RequestCanceled: waiter context canceled
caused by: context deadline exceeded

~/src/.../infra/ecspresso/green feat/ecspresso-greenReduce* 10m 0s

extVar support for jsonnet files

Hi,
I'm really excited with the jsonnet support. I can remove jsonnet install step 🙂
I tried it and found currently there's no way to pass --ext-str a=b options.
Do you have plans to support extVar related options?

must_env in task definition causes exec command to fail

Environment

ecspresso v1.7.8

Incident

When executing ecspresso exec to a task definition that contains must_env environment variables, panic occurs due to undefined environment variables.

This did not happen in v1.5.0.

Expected behavior

Even if must_env is used in a task definition, it should be possible to ecspresso exec without setting environment variables at runtime.

Log

$ DEPLOY_TARGET_ENV=prod ecspresso exec --config=.ecspresso/application-config.yml --command="/bin/bash"
panic: template attach failed: template: conf:12:77: executing "conf" at <must_env `DEPLOY_IMAGE_TAG`>: error calling must_env: environment variable DEPLOY_IMAGE_TAG is not defined

goroutine 1 [running]:
github.com/kayac/go-config.readConfigBytes(0xc000372600, 0x5b9, 0x5ba, 0xc00083ed60, 0x5ba, 0x0, 0x0, 0x0, 0x0)
        /home/runner/go/pkg/mod/github.com/kayac/[email protected]/config.go:149 +0x18e
github.com/kayac/go-config.(*Loader).ReadWithEnv(0xc0003d8180, 0xc0003680f0, 0x24, 0x3a7c600, 0x0, 0x23d, 0xc00083ee08, 0x10187d3)
        /home/runner/go/pkg/mod/github.com/kayac/[email protected]/config.go:322 +0xc6
github.com/kayac/ecspresso.(*App).readDefinitionFile(0xc0000bac80, 0xc0003680f0, 0x24, 0x10c1245, 0x10c1d00, 0xc0003685a0, 0xc000293628, 0x0)
        /home/runner/work/ecspresso/ecspresso/util.go:68 +0x6b9
github.com/kayac/ecspresso.(*App).LoadTaskDefinition(0xc0000bac80, 0xc0003680f0, 0x24, 0x25, 0x2374b08, 0xc0002935f0)
        /home/runner/work/ecspresso/ecspresso/ecspresso.go:498 +0x5a
github.com/kayac/ecspresso.(*App).listTasks(0xc0000bac80, 0x236f1a0, 0xc00071d680, 0xc000719d20, 0xc00083f2f8, 0x1, 0x1, 0xc0007143c0, 0x1, 0x1, ...)
        /home/runner/work/ecspresso/ecspresso/tasks.go:64 +0x48b
github.com/kayac/ecspresso.(*App).Exec(0xc0000bac80, 0xc000719d20, 0xc000719d50, 0xc000719d60, 0xc00045f7a0, 0xc00045f790, 0xc00045f798, 0x0, 0x0)
        /home/runner/work/ecspresso/ecspresso/exec.go:53 +0x145
main._main(0xc000094058)
        /home/runner/work/ecspresso/ecspresso/cmd/ecspresso/main.go:293 +0x8cde
main.main()
        /home/runner/work/ecspresso/ecspresso/cmd/ecspresso/main.go:18 +0x25

At this time, in our task definition, we set the family, arn, and image URLs as follows. The reason we specify must_env is that this is mainly for CD.

"family": “my_{{ must_env `DEPLOY_TARGET_ENV` }}_application”,
"taskRoleArn": "arn:aws:iam::hogefuga/components/my_{{ must_env `DEPLOY_TARGET_ENV` }}_application_task_role",

"containerDefinitions": [
  {
    "name": “hoge”,
    "image": “hogehoge.dkr.ecr.ap-northeast-1.amazonaws.com/hoge:{{ must_env `DEPLOY_IMAGE_TAG` }}",

…

Also, we have confirmed that if must_env is replaced with env, it can be executed without any problem.

This also happens when the environment variable used by the family is left undefined, so it does not seem to depend on which element uses must_env.

Workaround

Define a must_env environment variable (in our case, DEPLOY_IMAGE_TAG) at ecspresso exec runtime.

Possible causes

Probably because the change in v1.6 to allow command execution without ECS service refers to the task definition to find the same family.

What does --force-new-deployment option do?

First of all I'd like to say thank you for such a great tool, and keep up a good work! 👍

It's not really clear to me at first glance so I would be really thankful if someone could explain it to me what does --force-new-deployment option do?
ecspresso deploy --force-new-deployment

Is this option perhaps going to force tasks to be stopped, skip the draining part or it is something completely different?

Thanks in advance for your answer! 👋

Feature suggestion: SSM parameter store plugin

Is there a plan to add plugin support for ssm parameter store? If not, are you willing to accept the PR if there's one?
I'm imagining something like below:

# ecspresso.yml

...
plugins:
  - name: ssm
# anywhere in ecs-service-def.json or task-def.json

{
   ...
   "some_key": "{{ ssm `/path/to/parameter` }}"
}

Thanks.

Unable to update platformVersion and networkConfiguration for CodeDeploy services.

ecspresso appspec creates an AppSpec file based on current ECS service attributes (not based on service definition file).

Therefore we cannot update platformVersion and networkConfiguration for services that have deploymentController "CODE_DEPLOY".

Need to create an AppSpec file based on service definition file when ecspresso deploy --update-service.

Feature Request: Support a flag of the 'scale' command that specifies whether to suspend AutoScaling

I'm sorry, but I'm making this issue in Japanese for ease and speed. Please translate comments with services such as DeepL If you are not a Japanese speaker.

先日 Twitter にて AutoScaling 管理下の Desired Count を変更した際に起こる競合について質問をした者です。その際はありがとうございました。

(ちゃんとタスクがスケールされたことの確認のポーリングを自前で実装するのは面倒だったこともあり、) ecspresso scale でのスケールを検討しているのですが、どうせなら ecspresso deploy コマンドのように --suspend-auto-scaling および --no-suspend-auto-scaling もセットになっていたら嬉しいなぁ...と思いました。

自分でも手伝えるような内容に思えるので、今夜PRの作成トライしてみますが、シンプルさなどの方針から取り込みたくないということであれば Issue 自体 Reject していただいて大丈夫です!

[v1.5.2] MissingRegion: could not find region configuration になる(v1.5.1 では起きない)

https://twitter.com/sogaoh/status/1390165192357466112 でお伝えした問題です

ecspresso --config config.yaml verify
2021/05/06 04:34:22 failed to read tfstate: s3://XXXXXXXXXX/YYYYY/ZZZZZ/terraform.tfstate: failed to read tfstate from s3://XXXXXXXXXX/YYYYY/ZZZZZ/terraform.tfstate: MissingRegion: could not find region configuration

状況として、AWS_DEFAULT_REGION="ap-southeast-1" を direnv で設定してますが、AWS_REGION は設定していませんでした。

他に確認事項あれば随時ご連絡ください。 @fujiwara

ECS Exec 時、ログに KMS key (KMS key) を使用した暗号化を有効にしていた場合にセッションに失敗する

問題

例えば、CDK から s3EncryptionEnabled を true にしたようなクラスタに対して exec すると、以下のようなエラーがでます。

SessionId: ******: 
----------ERROR-------
Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error: Failed to process action KMSEncryption: Installed version of AWS CLI does not support Session Manager encryption feature. Please upgrade to latest version of AWS CLI

原因

session-manager-plugin を呼ぶ際、引数を 4 つ渡していますが、
https://github.com/kayac/ecspresso/blob/v1/exec.go#L93

これは session-manager-plugin 上 LegacyArgumentLength に該当するようです。
https://github.com/aws/session-manager-plugin/blob/e4ad4017df969b298c3e263f0c8ce9881c0b0a79/src/sessionmanagerplugin/session/session.go#L139

また、IsAwsCliUpgradeNeeded な場合、レガシー判定を受けて ProcessKMSEncryptionHandshakeAction を行うことができません。
https://github.com/aws/session-manager-plugin/blob/65933d1adf368d1efde7380380a19a7a691340c1/src/datachannel/streaming.go#L832

最新の awscli が扱っているようにあるいは、コメントにあるように引数を増やしてあげる必要がありそうです。
https://github.com/aws/aws-cli/blob/45b0063b2d0b245b17a57fd9eebd9fcc87c4426a/awscli/customizations/sessionmanager.py#L83
https://github.com/aws/session-manager-plugin/blob/e4ad4017df969b298c3e263f0c8ce9881c0b0a79/src/sessionmanagerplugin/session/session.go#L101

feature request: resolve definition files path relatively from config.yml

Motivation

ecspresso cli resolves definition files ( service_definition and task_definition ) path from current working directory.
So we need to run ecspresso cli on exact directory .
This causes some confusion when config files put nested directory structure.

Expected behavior

ecspresso cli resolves definition files path from config.yml file relatively.

Actual behavior example

In this directory structure, I need to configure and run escpresso cli in 2 ways.

.
└── deploy
   ├── config.yaml
   ├── ecs-service-def.json
   └── ecs-task-def.json
  1. change directory deploy way

Configure config.yml 's definition files following example, and run ecspresso cli after change directory to deploy

service_definition: ecs-service-def.json
task_definition: ecs-task-def.json
  1. specify definition files path from root directory way

Configure config.yml 's definition files following example, and run ecspresso cli in root directory.

service_definition: ./deploy/ecs-service-def.json
task_definition: ./deploy/ecs-task-def.json

DeploymentConfigName is not used correctly with CODE_DEPLOY

ecspresso version: v0.17.3
Fargate : 1.3.0

CodeDeployDefault.ECSAllAtOnce is always used instead of the DeploymentConfigName set in the DeploymentGroup.
If you have any solutions, please let me know.

DeploymentGroup

aws deploy get-deployment-group --application-name cluster --deployment-group-name cluster | jq -r '.[].deploymentConfigName'

CodeDeployDefault.ECSCanary10Percent5Minutes

Results of running ecspresso (cluster name,service name,sg, subnet are masked)

ecspresso deploy --config config.yml --debug --tasks=1 --suspend-auto-scaling --force-new-deployment


2020/08/27 23:38:27 service-name/cluster Starting deploy
Service: service-name
Cluster: cluster
TaskDefinition: cluster:817
TaskSets:
   PRIMARY cluster:817 desired:1 pending:0 running:1
    ACTIVE cluster:818 desired:1 pending:0 running:1
Events:
2020/08/27 23:38:28 service-name/cluster Creating a new task definition by ../task_definition.json
2020/08/27 23:38:28 service-name/cluster Registering a new task definition...
2020/08/27 23:38:28 service-name/cluster Task definition is registered cluster:819
2020/08/27 23:38:28 service-name/cluster desired count: 1
2020/08/27 23:38:28 service-name/cluster No scalable target for service/cluster/service-name
2020/08/27 23:38:28 service-name/cluster appSpecContent: version: "0.0"
Resources:
- TargetService:
    Type: AWS::ECS::Service
    Properties:
      TaskDefinition: arn:aws:ecs:ap-northeast-1:123456789012:task-definition/cluster:819
      LoadBalancerInfo:
        ContainerName: application
        ContainerPort: 8080
      PlatformVersion: 1.3.0
      NetworkConfiguration:
        awsvpcconfiguration:
          assignpublicip: DISABLED
          securitygroups:
          - sg-1234567890abcd
          subnets:
          - subnet-1234567890abcd
          - subnet-1234567890abcd
          - subnet-1234567890abcd

2020/08/27 23:38:28 service-name/cluster depoymentInfo {
  ApplicationName: "cluster",
  AutoRollbackConfiguration: {
    Enabled: true,
    Events: ["DEPLOYMENT_FAILURE"]
  },
  BlueGreenDeploymentConfiguration: {
    DeploymentReadyOption: {
      ActionOnTimeout: "STOP_DEPLOYMENT",
      WaitTimeInMinutes: 5
    },
    TerminateBlueInstancesOnDeploymentSuccess: {
      Action: "TERMINATE",
      TerminationWaitTimeInMinutes: 5
    }
  },
  CompleteTime: 2020-08-27 14:16:47 +0000 UTC,
  ComputePlatform: "ECS",
  CreateTime: 2020-08-27 14:13:47 +0000 UTC,
  Creator: "user",
  DeploymentConfigName: "CodeDeployDefault.ECSAllAtOnce",
  DeploymentGroupName: "cluster",
  DeploymentId: "d-HT2P9ZZR5",
  DeploymentOverview: {
    Failed: 0,
    InProgress: 0,
    Pending: 0,
    Ready: 0,
    Skipped: 0,
    Succeeded: 1
  },
  DeploymentStatusMessages: [],
  DeploymentStyle: {
    DeploymentOption: "WITH_TRAFFIC_CONTROL",
    DeploymentType: "BLUE_GREEN"
  },
  FileExistsBehavior: "DISALLOW",
  IgnoreApplicationStopFailures: false,
  InstanceTerminationWaitTimeStarted: true,
  LoadBalancerInfo: {
    TargetGroupPairInfoList: [{
        ProdTrafficRoute: {
          ListenerArns: ["arn:aws:elasticloadbalancing:ap-northeast-1:123456789012:listener/app/cluster-external/e4a3a44e88770f9c/8c707a19857045b2"]
        },
        TargetGroups: [{
            Name: "cluster-external-8080-blue"
          },{
            Name: "cluster-external-8080-green"
          }]
      }]
  },
  PreviousRevision: {
    AppSpecContent: {
      Sha256: "4b3a0a34efd879534fb5074d1c9a89dcd2bed4468bd615f11d3f64a4615e5748"
    },
    RevisionType: "AppSpecContent"
  },
  Revision: {
    AppSpecContent: {
      Sha256: "3469252eba2fe38ab9eb15bd4f97d3f38928d584f47f1d7757669aeff953cf1e"
    },
    RevisionType: "AppSpecContent"
  },
  Status: "Succeeded",
  UpdateOutdatedInstancesOnly: false
}
2020/08/27 23:38:28 service-name/cluster creating a deployment to CodeDeploy {
  ApplicationName: "cluster",
  DeploymentConfigName: "CodeDeployDefault.ECSAllAtOnce",
  DeploymentGroupName: "cluster",
  Revision: {
    AppSpecContent: {
      Content: "version: \"0.0\"\nResources:\n- TargetService:\n    Type: AWS::ECS::Service\n    Properties:\n      TaskDefinition: arn:aws:ecs:ap-northeast-1:123456789012:task-definition/cluster:819\n      LoadBalancerInfo:\n        ContainerName: application\n        ContainerPort: 8080\n      PlatformVersion: 1.3.0\n      NetworkConfiguration:\n        awsvpcconfiguration:\n          assignpublicip: DISABLED\n          securitygroups:\n          - sg-1234567890abcd\n          subnets:\n          - subnet-1234567890abcd\n          - subnet-1234567890abcd\n          - subnet-1234567890abcd\n"
    },
    RevisionType: "AppSpecContent"
  }
}

CodeDeploy利用時にwaitコマンドでタイムアウトの設定が反映されない

CodeDeploy利用時のwaitコマンドでconfigで30分以上の値を設定してもSDKのデフォルト値(120 * 15)の30分でタイムアウトします。
https://github.com/aws/aws-sdk-go/blob/v1.42.23/service/codedeploy/waiters.go#L31-L32

WaitUntilDeploymentSuccessfulWithContext でも WaitUntilServicesStableWithContext と同様にwaiterでdelay、attemptsを渡してあげる必要がある。

ecspresso/ecspresso.go

Lines 646 to 649 in 354ba4a

return d.codedeploy.WaitUntilDeploymentSuccessfulWithContext(
ctx,
&codedeploy.GetDeploymentInput{DeploymentId: dpID},
)

ecspresso/ecspresso.go

Lines 484 to 488 in 354ba4a

return d.ecs.WaitUntilServicesStableWithContext(
ctx, d.DescribeServicesInput(),
request.WithWaiterDelay(request.ConstantWaiterDelay(delay)),
request.WithWaiterMaxAttempts(attempts),
)

fail to find corresponding CodeDeploy deployment group if cluster's full ARN is specified at cluster's value in config.yml

Way to reproduce

  • Specify full ARN of the cluster at config.yml such as cluster: arn:aws:ecs:ap-northeast-1:1234567890:cluster/my-cluster
  • Setup service's deployment controller with CodeDeploy
  • Run ecspresso deploy

Expected behavior

No errors happen.

Actual Behavior

Got failed to find CodeDeploy Application/DeploymentGroup error:

2021/07/23 19:17:02 deploy FAILED. failed to find CodeDeploy Application/DeploymentGroup for ECS service <my-service-name> on cluster arn:aws:ecs:ap-northeast-1:1234567890:cluster/my-cluster

Possible causes

The cluster value in config.yml compared with possible deployment group's ECS cluster value, but we can specify cluster's full ARN as cluster value in config.yml and ecspresso works well except running ecspresso deploy.

So if we specified the full ARN, it is never matched with the corresponding deployment group's cluster specification.

I have some ideas about this problem:

ecspresso run fails with "ResourceNotReady" when the command finishes in a very short time

Summary

When running a command in a container that takes less than 1 second to complete, escpresso run frequently determines that the task has failed, even though the command completed successfully.

Error output example

$ ecspresso run --config ecspresso.yml 
2022/04/12 15:22:02 ecspresso-test/api Running task
2022/04/12 15:22:02 ecspresso-test/api Registering a new task definition...
2022/04/12 15:22:04 ecspresso-test/api Task definition is registered ecspresso-test:22
2022/04/12 15:22:04 ecspresso-test/api Task definition ARN: arn:aws:ecs:us-east-1:509803364528:task-definition/ecspresso-test:22
2022/04/12 15:22:04 ecspresso-test/api Watch container: hello
2022/04/12 15:22:04 ecspresso-test/api Running task with arn:aws:ecs:us-east-1:509803364528:task-definition/ecspresso-test:22
2022/04/12 15:22:04 ecspresso-test/api Task ARN: arn:aws:ecs:us-east-1:509803364528:task/api/3c84bfc97964403cae75ec093ceb200e
2022/04/12 15:22:04 ecspresso-test/api Waiting for run task...(it may take a while)
2022/04/12 15:22:04 ecspresso-test/api Watching container: hello
2022/04/12 15:22:04 ecspresso-test/api logGroup: ecspresso-test
2022/04/12 15:22:04 ecspresso-test/api logStream: test/hello/3c84bfc97964403cae75ec093ceb200e
2022/04/12 15:22:07 ecspresso-test/api Waiting for task ID 3c84bfc97964403cae75ec093ceb200e until running
2022/04/12 15:22:07 run FAILED. failed to run task: failed to run task: ResourceNotReady: failed waiting for successful resource state

Possible Cause

ecspresso run waits for the task to reach the RUNNNIG state with WaitUntilTasksRunningWithContext after starting the task.

if err := d.ecs.WaitUntilTasksRunningWithContext(

However, if the container/task has completed in a very short time, the state of the task will already moved to STOPPED.
In that case, WaitUntilTasksRunningWithContext will return it as failure.
https://github.com/aws/aws-sdk-go/blob/4829c1f1e069367524159c9051fbea670ecc36f5/service/ecs/waiters.go#L145-L161

Reproduction code

Here is the task definition I used.

{
  family: 'ecspresso-test',
  containerDefinitions: [
    {
      name: 'hello',
      image: 'alpine:latest',
      essential: true,
      cpu: 10,
      memory: 64,
      command: [
        'sh',
        '-c',
        'sleep 1 && echo hello',
      ],
      logConfiguration: {
        logDriver: 'awslogs',
        options: {
          'awslogs-group': 'ecspresso-test',
          'awslogs-region': 'us-east-1',
          'awslogs-stream-prefix': 'test',
        },
      },
    },
  ],
}

`ecspresso verify` cannot get from secrets manager

When taskDefinition defines secrets that stored in secrets manager,ecspresso verify always shows error.

example definition

      "secrets": [
        {
          "name": "MY_SECRET",
          "valueFrom": "arn:aws:secretsmanager:ap-northeast-1:1234567890:secret:foo/bar/my-secret-abcdef"
        }
      ]

error message

[NG] verify Secret[MY_SECRET] failed: ValidationException: Invalid parameter name. Please use correct syntax for referencing a version/label  <name>:<version/label>

The cause is that all secrets are retrieved for the parameter store in the following:

ecspresso/verify.go

Lines 327 to 330 in 3af4584

_, err := d.verifier.ssm.GetParameterWithContext(ctx, &ssm.GetParameterInput{
Name: secret.ValueFrom,
WithDecryption: aws.Bool(true),
})

If valueFrom starts with arn:aws:secretsmanager, the secret should be retrieved from secrets manager.

ecspresso verify should check log driver.

task definitions require a valid log driver.

failed to register task definition: ClientException: awsvpc is not a valid log driver. Must be one of [awslogs,splunk,awsfirelens]

Failed to verify for non tagged image URL.

When an Amazon ECR image URL has not a tag (for example 123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/myimage), ecspresso fails to verify the URL.

[NG] verify Image[123456789012.dkr.ecr.ap-northeast-1.amazonaws.com/myimage] failed: aborted

Feature Request: Support specifying a remote state file by s3 URL.

日本語で失礼致します:bowing_man:
先日Twitterにてリプライした内容について、おこがましいですがIssueを作成致します。

Use-cases

s3にあるtfstateを参照するとき、 s3://<bucket-name>/<key-of-file> 形式で指定できると直感的でわかりやすいと思います。
また同時に、 Workspaces に関して現在 ecspresso ではサポートがないものと存じますが、これステージ分離等に活用している場合も(configファイルを複数設けるなどハードコーディングで)対応できるように感じます。(URLを s3://<bucket-name>/env:/${terraform.workspace}/<key-of-project> 形式にする)

Proposal

これは完全に余談ですが、ユースケースによってはtfstate自体が分離されている可能性もあって、tfstateを複数指定できたら更に良いなとも思います。ただこれは単に配列で取得してJSONをマージしても事故が起こると思われるので、名前をつけてimportする必要がありますが、そこで付与した名前をserviceなどの定義ファイル内で識別子として認識するところも実装する必要がありそうで大変そうなので...自分がまともにGoのコードを開発できるようになってからPR飛ばします...:crying_cat_face:

Feature Request: Support add Tags to TaskDefinition

Hello.

I want to add tag to taskdefinition.
But ecspresso doesn't support to add tags.

Because there are no tags in the argument.

ecspresso/diff.go

Lines 108 to 122 in 53d3c28

func tdToRegisterTaskDefinitionInput(td *ecs.TaskDefinition) *ecs.RegisterTaskDefinitionInput {
return &ecs.RegisterTaskDefinitionInput{
ContainerDefinitions: td.ContainerDefinitions,
Cpu: td.Cpu,
ExecutionRoleArn: td.ExecutionRoleArn,
Family: td.Family,
Memory: td.Memory,
NetworkMode: td.NetworkMode,
PlacementConstraints: td.PlacementConstraints,
RequiresCompatibilities: td.RequiresCompatibilities,
TaskRoleArn: td.TaskRoleArn,
ProxyConfiguration: td.ProxyConfiguration,
Volumes: td.Volumes,
}
}

The AWS SDK RegisterTaskDefinitionInput supports adding tags.
https://docs.aws.amazon.com/sdk-for-go/api/service/ecs/#RegisterTaskDefinitionInput

Tags []*Tag `locationName:"tags" type:"list"`

In my project, I use tags to use billing and AWS Resource Groups services.
So I want this feature.

Thank you.

Does not `desiredCount` field work?

I tried to change desiredCount field in service.json and run ecspresso deploy --update-service --skip-task-definition , but a count of tasks was not changed.

   {
-     "desiredCount": 1,
+     "desiredCount": 0,

I confirmed that --tasks option works fine to scale-in/out, so does not ecspresso support to change desiredCount field in JSON for scale-in/out?

Version: 0.16.0

Run with --latest-task-definition behavior

Hi and thanks for your work 👍

I've read your Advent calendar on zenn and tried ecspresso run --latest-task-definition. I thought it would use the latest task definition but actually it uses the definition used by the service. Is that an intended behavior? If so, maybe reflect it in docs?

My use case is as follows:

  • register new task definition (e.g. def:1 -> def:2)
  • run task with the new definition (def:2)
    currently ecspresso runs task with def:1 here
  • update service with the new definition (def:2)

It would be nice if you could specify task revision (e.g. 2) for the run command.

Roadmap to v2

  • Switch to aws-sdk-go-v2.
  • rollback --deregister-task-definition true by default.
    • Raise an error when rollback --deregister-task-definition with --no-wait.
  • #344
  • #45
  • Abolish create subcommand. use deploy subcommand simply.
    #435
  • No require service_definition if run tasks only.
  • render command accepts render target by CLI args instead of flags (e.g. --task-definition)
    #444
  • Remove fallback to executionRole.
    #269
  • diff --unified by default.
  • init --jsonnet by default.
    #447
  • Progress bar for codedeploy #412
  • Switch CLI parser from kingpin.
    #455

...and so on

feature request: can use only `--command` flag to execute command

When I want to run any command without changing the task or service definition, I have to use --overrides flag at run, and have to prepare json string for it.

So, I would like to add the flag --command and be able to execute any command as follows.

ecspresso run --config config.yaml --command "echo hoge"

Feature Request: Support diff of desiredCount

Currently,diff command does not cover desiredCount.
I want to target desiredCount as well as other parameters so that we can manage changes in ecs-service-def.json.
We can also achieve this with init --force-overwrite, but this method will not work if you are using templates or jsonnet.

Please add an option like --enable-desired-count.

forgetting to pass --config argument causes strange error

Way to reproduce

Run ecspresso init with required arguments except --config.

Expected behavior

No errors happen or enough information to fix it such as --config required but not passed.

Actual Behavior

Got failed to write file: open : no such file or directory error.

Full error messages are below:

[2021-07-21 13:55:58] ✘╹◡╹✘ < ecspresso init --cluster $cluster_arn --service $service_arn
2021/07/21 13:56:06 <snip> save service definition to ecs-service-def.json
2021/07/21 13:56:06 <snip> save task definition to ecs-task-def.json
2021/07/21 13:56:06 <snip> save config to
2021/07/21 13:56:06 init FAILED. failed to write file: open : no such file or directory

Way to fix

I think --config argument is actually a required parameter so it must be marked as a required parameter using .Required(), or should we set a default value such as config.yml?

Feature Request: Support force override for init sub-command.

In my use-case, deployment workflow updates multiple services and runs task (unrelated to those services).
So, I want to run ecspress init command multiple times in one work directory.
This produces overwrite prompt like below.

021/02/02 12:01:38 xxxx/xxxx save service definition to ecs-service-def.json
Overwrite existing file ecs-service-def.json? (y/n) [n]: y
2021/02/02 12:01:44  xxxx/xxxx  save task definition to ecs-task-def.json
Overwrite existing file ecs-task-def.json? (y/n) [n]: y

Or, should I create config.yml files for each ECS Services I want to update?

v1.7.9 fails deployment app, which using B/G deployment with CodeDeploy

We tried to deploy an ECS application using B/G deployment with CodeDeploy, by ecspresso v1.7.9, but it was failed with events log below.

Events:
2022/03/10 06:20:44 ... Registering a new task definition...
2022/03/10 06:20:45 ... Task definition is registered ...:33
2022/03/10 06:20:45 ... Updating service attributes...
2022/03/10 06:20:45 deploy FAILED. failed to update service attributes: InvalidParameterException: Unable to update load balancers on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.

By ecspresso v1.7.8, we could deploy the application successfully.


FYI: ecs-service-def.json like below.
And we didn't update the file, so it should not updates ECS service I think.

{
  "deploymentConfiguration": {
    "maximumPercent": 200,
    "minimumHealthyPercent": 100
  },
  "deploymentController": {
    "type": "CODE_DEPLOY"
  },
  "desiredCount": 1,
  "enableECSManagedTags": true,
  "enableExecuteCommand": false,
  "healthCheckGracePeriodSeconds": 0,
  "launchType": "FARGATE",
  "loadBalancers": [
    {
      "containerName": "app",
      "containerPort": 80,
      "targetGroupArn": "..."
    }
  ],
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "assignPublicIp": "DISABLED",
      "securityGroups": ["..."],
      "subnets": ["...", "..."]
    }
  },
  "placementConstraints": [],
  "placementStrategy": [],
  "platformFamily": "Linux",
  "platformVersion": "1.4.0",
  "schedulingStrategy": "REPLICA",
  "serviceRegistries": [],
  "tags": []
}

Specifying multiple tfstates

Can we use values from multiple tfstate files?
I tried the following, but seems like only the last one is loaded.

# ecspresso.yml
...
plugins:
   - name: tfstate
    config:
      url: s3://tfstate1
  - name: tfstate
    config:
      url: s3://tfstate2

Thanks for the great tool.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.