devopsartfactory / goployer Goto Github PK

New volume type of EBS, io2, was released.
This has better i/o performance than io1 with no additional cost.

Goployer should support this volume type for users to use this type with autoscaling group.

https://aws.amazon.com/blogs/aws/new-ebs-volume-type-io2-more-iops-gib-higher-durability/

when ami base arch is x86_64, user using override spot types arm64 instance type.

then it work.

i think this case failed deployment.

Is your feature request related to a problem? Please describe.
There is no test code before checking goployer works in local system. Also if someone ones to deploy with goployer in the AWS environment, ALB, TGs are needed before testing.

Describe the solution you'd like
Terraform code is needed

Additional context
Before considering local test, we should have pre-test environment for contributing.
@jupitersong

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
AWS autoscaling lifecycle feature.
https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroupLifecycle.html

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

New EBS volume type GP3 is released!!
Goployer should support all ebs volume types~
https://aws.amazon.com/ko/blogs/korea/new-amazon-ebs-gp3-volume-lets-you-provision-performance-separate-from-capacity-and-offers-20-lower-price/

Is your feature request related to a problem? Please describe.
Change goployer as command line interface. We could add many features if we could use CLI format.

example.

• goployer deploy -> run deployment
• goployer init -> initialize goployer project ( creating yaml file )
• goployer server -> run goployer server etc..
• goployer version -> check goployer version

Is your feature request related to a problem? Please describe.
Controlling polling time when doing healthcheck.

Describe the solution you'd like
There are two ways to implement this.

1. send polling-duration argument from command line
2. specify polling_duration configuration in manifest file

Additional context
Polling time means the duration of seconds for doing healthchecking process for newly created autoscaling groups and previous version of autoscaling group. Default polling time is 60 seconds.
https://github.com/DevopsArtFactory/goployer/blob/master/pkg/tool/common.go#L14

It would be better if we could control the capacity of changing autoscaling group capacity.

Expected

## single change
$ goployer update hello --stack=artd --region=ap-northeast-2 --min=1
$ goployer update hello --stack=artd --region=ap-northeast-2 --desired=3
$ goployer update hello --stack=artd --region=ap-northeast-2 --max=10 

## change all capacity
$ goployer update hello --stack=artd --region=ap-northeast-2 --min=1 --desired=3 --max=10 

Describe the bug
Goployer cannot find the ami id with ami name.

To Reproduce
Steps to reproduce the behavior:

1. Deploy with --ami=<name of AMI>

Expected behavior
Goployer should find the ami-id with the name

Additional context
If there are more than one AMIs with the name provided in the account, then it should give the latest AMI ID.

Canary Deployment should be supported in order for users to deploy A/B features.

For canary deployment, goployer will do these jobs

• Create new version autoscaling group
• Add tag to both of autoscaling group
• Attach new version autoscaling group to existing target group
• Create copy of LB and target group
• Attach net version to new LB
• --canary-confirm will replace all instances to new version

change sample code

add golangcli-lint for clean code
https://github.com/golangci/golangci-lint

Currently, goployer status shows information like this

$ goployer status hello
? Choose autoscaling group: hello-dev_apnortheast2-v003
Name:           hello-dev_apnortheast2-v003
Created Time:   2020-09-16 10:29:21.169 +0000 UTC

📦 Capacity
MINIMUM    DESIRED    MAXIMUM
1          1          2

🖥 Instance Statistics
 ∙ t3.medium: 1

⚓ Tags
 ∙ Name=hello-dev_apnortheast2-v003
 ∙ ansible-tags=all
 ∙ app=hello
 ∙ project=test
 ∙ repo=hello-deploy
 ∙ stack=_apnortheast2
 ∙ stack-name=artd
 ∙ test=test

But I think it would be better if we could find security group information so that user does not need to check all ingress and egress every time with console.

We need to gather several metrics when new deployment process occur.
This could help users to find information of previous deployments which could be useful when we want to add more features with these data.

Metrics we could gather are...

• uptime,
• start time,
• all configurations applied at the moment of deployment including tags and userdata.
  Please comment if there are more metrics to gather.

For the database, we could support dynamo DB first. Later, if possible, we could support RDBMS or S3(Text file).
Thanks for suggestion, @jupitersong

Currently, only one stack should be chose from the command line (--stack).

But I think many users want to deploy multiple stacks at the same time.
This might not be urgent right now because user can run multiple goployer command, even though these commands will run sequentially.

(cc @jupitersong @asbubam)

We need to support detailed monitoring for EC2 instance. This is controlled via launch template.
Detailed monitoring should be specified in the manifest file.

Configuration key will be detailed_monitoring_enabled.

Is your feature request related to a problem? Please describe.
We could provide CLI option for specifying instance type. It would be possible only if user wants to set exact same instance types for all regions in a stack.

This will not be available if mixed_instances_policy.enabled is true. It is better if we could raise error when these two values are provided for alerting.

This feature would be helpful if a user wants to change instance type without updating manifest file.

Describe the solution you'd like
For parameter name, --override-instance-type would be nice. --instance-type could be confused with instance type in the manifest file. And if this argument is set, it will override all instance type when creating launch template.

We need DEVELOPMENT.md guide file for future contributors

Instance refresh is a new feature of autoscaling group which changes instances with new ones without changing autoscaling group.

Further information:
https://aws.amazon.com/blogs/compute/introducing-instance-refresh-for-ec2-auto-scaling/

Current status format is now shown well if the shell is dark mode..

Currently goployer provides single api test configuration in the manifest which results in continuous changes for different test configurations.

I think it would be better if we could support multiple definitions for api test and make users select the specific api calls for the stack.

Current configuration is like this

api_test_template:
  name: api-test
  duration: 5s
  request_per_second: 10
  apis:
    - method: GET
      url: https://hello-artd.devopsartfactory.com
    - method: POST
      url: https://hello-artd.devopsartfactory.com/post
      body:
        - id=198355
        - username=gslee
        - test=test

This will be changed like below

stacks:
  - stack: wed
    env: wed
    api_test_enabled: true
    api_test_template: api-test-1

api_test_templates:
  - name: api-test-1
    duration: 5s
    request_per_second: 10
    apis:
      - method: GET
        url: https://hello-artd.devopsartfactory.com
      - method: POST
        url: https://hello-artd.devopsartfactory.com/post
        body:
          - id=198355
          - username=gslee
          - test=test

  - name: api-test-2
    duration: 5s
    request_per_second: 10
    apis:
      - method: GET
        url: https://hello-artd.devopsartfactory.com
      - method: POST
        url: https://hello-artd.devopsartfactory.com/post
        body:
          - id=198355
          - username=gslee
          - test=test
``

Currently goployer select subnets with name prefix.
If use_public_subnets is true, then it selects subnets with prefix public, otherwise 'private'.

This is name dependent so that users cannot select subnets to use per application.

Adding subnets option in manifest file like serverless manifest file can be helpful

When users want to deploy AWS resources, it would be better if they could choose profile among many accounts.

Documentation for goployer.
This could be done by hugo, or any other tools for generating document.

In the website, detailed explanation for how goployer works will be included.

We need to provide required IAM Policy so that users can use goployer as soon as they downloaded it.

This could be provided via terraform, or IAM Policy format. (Or we could just list all the permissions in README.md file)

In 2.0.5 version, goployer status command shows security group information.

But if source is another security group id, then it is hard for users to figure out what it is and which rules it has.

So, it would be better if goployer shows security group information more in detail or support another command to inspect security group id.

If there is no target group for the instance like bastion server, we should pass it without error by creating autoscaling group without healthcheck.

For each stack, slack alarm configuration is needed because some of stacks are really critical to the service so that failure deployment should be alarm in a special way.

Currently, goployer sends the process of deployment with slack message but this is not enough to check failure if it is timeout failure ( timeout is , by default, 1 hour )

I think only two more configurations needed for this feature

stack:
  slack_url: 
  notification_level: failure / all 

Add tag configuration to each stack.

If mixed_instances_policy option is enabled, then override_instance_types is working.

but override_instance_types cannot dynamic change value.

In this case, add override option for multiple override instance type for spot instance.

And user can Input multiple instances but only same instance type can use. (ex. t6g type(arm64) cannot use t3 type(x86_64))

AWS cli uses region configuration in ~/.aws/config file. So if --region is not specified then we can use the same region.
This could improve user experience

Currently goployer does not support termination policy setting.
Now all autoscaling group is supposed to use Default termination policy.

termination_policies option could be suitable for this configuration and this should be regional based. (per autoscaling group)

regions:
  - region: ap-northeast-2
    termination_policies: 
      - Default
      - OldestInstance

If termination_policies is not specified, then it should include Default only.

https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.html#custom-termination-policy

Provide official docker image which contains latest-released goployer.

This could be helpful when using goployer with codebuild or jenkins.

Opinion needed :) @jupitersong

Get current status of application like current capacity, healthy information, deployment date, spot percentage. release note etc...

Expected

$ goployer status hello --stack=artd --region=ap-northeast-2

For documentation, we are using docker with make preview-docs so that we don't need to download theme and manage it in the local file.

We can remove docs/themes/docsy and git submodule

New feature for autoscaling group came out.

Instance rebalancing occurs before 2-minutes termination notification and autoscaling will change the target spot instance with the new one available. This helps users that they no longer need to concern about the sudden termination of spot.

Further information:
https://aws.amazon.com/about-aws/whats-new/2020/11/introducing-ec2-instance-rebalance-recommendation-for-ec2-spot-instances/

Is your feature request related to a problem? Please describe.
I cannot set scheduled scaling event for autoscaling group. Scheduling feature could be used when users want to scale out during 9-18 or scale in during weekend in order to maximize the efficiency.

Describe the solution you'd like
Add scheduling event configuration to manifest file. I think this configuration should be in a different block like autoscaling policy. And in the region configuration, users could specify multiple scheduling events referring the name of event in the scheduling_events

scheduling_events: &scheduling_events
  - name: scale_in_during_weekend
     capacity: 
         min: 0
         desired: 0 
         max: 0
  - name: start_9am_weekday
     capacity: 
         min: 1
         desired: 3
         max: 5

( ... skipped...)
stack:
  ... 
  regions:
      - region: ap-northeast-2
        instance_type: t3.medium
        ssh_key: test-master-key
        scheduling_events:
            - scale_in_during_weekend
            - start_9am_weekday

Goployer as a server need to be added. This could be integrated with goployer dashboard in the future. If dashboard becomes available, then goployer could be more fascinating because user could manage deployment with dashboard without setting other tools like jenkins or codebuild.

This could be done via docker-compose and we could support more framework to use for goployer server.