Create services with a load balancer about amazon-ecs-cli HOT 56 CLOSED

@kaihendry I am using following script to do what @geowa4 mentioned in the description

ecs-cli configure --region us-east-1 --cluster $CLUSTER_NAME

# create ecs cluster of ec2 instances
ecs-cli up --keypair $KEY_PAIR --capability-iam --size $CLUSTER_SIZE --security-group $SSH_SECURITY_GROUP --vpc $VPC_ID --subnets $SUBNET_ID --image-id $AMI_ID --instance-type $INSTANCE_TYPE --verbose

# create task definition for a docker container
ecs-cli compose --file compose.yml --project-name $CLUSTER_NAME --verbose create

# create elb & add a dns CNAME for the elb dns
aws elb create-load-balancer --load-balancer-name "$CLUSTER_NAME" --listeners Protocol="TCP,LoadBalancerPort=8080,InstanceProtocol=TCP,InstancePort=8080" --subnets "$SUBNET_ID" --security-groups "$ECS_SECURITY_GROUP" --scheme internal

# create service with above created task definition & elb
aws ecs create-service --service-name "ecscompose-service-$CLUSTER_NAME" --cluster "$CLUSTER_NAME" --task-definition "ecscompose-$CLUSTER_NAME" --load-balancers "loadBalancerName=$CLUSTER_NAME,containerName=demo-service,containerPort=8080" --desired-count 1 --deployment-configuration "maximumPercent=200,minimumHealthyPercent=50" --role ecsServiceRole

and my compose.yml is as follows

demo-service:
    image: ubuntu
    ports:
        - "8080:8080"
    cpu_shares: 512
    mem_limit: 1024

from amazon-ecs-cli.

FYI Support for assigning ELB/ALB is now on master, the ELB should exist before creating. #211

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

PSA: use the subscribe button on the right to indicate "+1".

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

@sunghoaraipark @vincetse There is a built in voting system now using reactions. it's better to add a thumbs up reaction to the first comment

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

Thanks for the starting point @ankitcha

I maintain a set of scripts to setup the load balancer with ECS here: https://github.com/kaihendry/count

I really really hope ecs-cli gets some love and fixes this standard use case.

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

@kaihendry I get a 404 on your link

from amazon-ecs-cli.

I wrote this in the meantime: https://github.com/lox/ecsy

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

@LeandroCR please use the built-in reaction system instead of a "+1" comment. GitHub finally implemented it (and you can sort issues by them) so let's use them! 😄

from amazon-ecs-cli.

@dcworldwide Yes, it does work for scaling-in / out for containers.

from amazon-ecs-cli.

@Victorcoder, I've downloaded v0.5.0 (who's releast notes specify the availability of this feature) but can't seem to figure out where to call '--target-group-arn'. The help section doesn't have it listed as one of the options for the 'up' command either. Also, the main docs weren't updated either. Are you sure that it was merged with v0.5.0?

from amazon-ecs-cli.

Yes, this is merged with v0.5.0. Please look at ecs-cli compose service up --help and http://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service.html for reference

from amazon-ecs-cli.

I can't seem to get this to work. I checked and double-checked that I specified --target-group-arn, --container-name, --container-port, and --role properly, but I can't get ecs-cli to add my service to my target group (the target group is empty and in the same VPC as the ECS cluster and the role is configured properly with AmazonEC2ContainerServiceRole).

I ran it with --debug but it just seems like the load balancer info isn't being added to the service definition at all. Any tips on why this might be not working or getting more verbose log output from the up command?

(For reference I'm on ecs-cli version 0.5.0 (1aa8028))

Thanks so much. :)

from amazon-ecs-cli.

Ahh, I see. The behavior is expected, see documentation:

After you create a service, the target group ARN or load balancer name, container name, and container port specified in the service definition are immutable. You cannot add, remove, or change the load balancer configuration of an existing service. If you update the task definition for the service, the container name and container port that were specified when the service was created must remain in the task definition.

To avoid confusions here, we should add validations on cli when the service already exists.

from amazon-ecs-cli.

Sorry for the confusion, we definitely need better error handling here. I will create a new issue to follow up on this :) Thanks!

from amazon-ecs-cli.

@shatil Currently, we only support using existing ELB in Create Service. We are planning on supporting the ELB creation possibly using the CloudFormation, but I do not have an ETA yet.

from amazon-ecs-cli.

@jordantomax If you wanted to create a load balancer in the same workflow as creating an ECS service (rather than specifying one that was created separately), you might be interested in checking out the CDK! https://awslabs.github.io/aws-cdk/refs/_aws-cdk_aws-ecs.html#include-a-load-balancer

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

+1

from amazon-ecs-cli.

Has there been any traction on this?

from amazon-ecs-cli.

Hi, do we have any idea of an ETA of when / if this might be available?

from amazon-ecs-cli.

As there has been no updates on this, I've been using Cloudformation for few months now for managing ECS. After recent updates, it supports all paramaters I need. Might work for some one else too.

from amazon-ecs-cli.

@geowa4 @mfonsen Would be good if you iterate the steps you use on the cli to create a cluster with a load balancer.

My current work around is to create the service manually in the console but that can fraught with error and confusion. 😁

from amazon-ecs-cli.

New to ecs, dissapointed to see this use case isn't supported OOTB.

Thanks for sharing your workaround @ankitcha, I'm just wondering, if you subsequently scale out/in your containers, will ELB automatically update to reflect the changes?

from amazon-ecs-cli.

Ok, I think I've isolated the problem. I brought up a second cluster and LB and tried to just bring up an entirely new instance of my service on that cluster. It failed with error:

ERRO[0000] Error creating service                        error="InvalidParameterException: Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions.\n\tstatus code: 400, request id: <request-id>" service=<servicename>
FATA[0000] InvalidParameterException: Unable to assume role and validate the specified targetGroupArn. Please verify that the ECS service role being passed has the proper permissions.

Then, when I run ecs-cli compose --project-name <project-name> service up ----target-group-arn <arn> --container-name nginx --container-port 80 --role ecsServiceRole again, the command succeeds, but fails to actually add the service to the LB. Seems like there is a bug where if the role requests 400, ecs-cli doesn't properly indicate an error.

As an aside, the IAM error itself is really bizarre, given that I've created and am using IAM with effectively wide open permissions. Although, perhaps, this thread isn't the proper place to discuss issues with IAM and ECS themselves.

from amazon-ecs-cli.

Can you explain step by step how you run into this problem when service is not created with the LB? What commands did you run?

ecs-cli compose --project-name <project-name> service up --target-group-arn <arn> --container-name nginx --container-port 80

This should throw you an IAM error because role is required.

from amazon-ecs-cli.

Excuse me, I just neglected to include the role flag, I did include it the the command. However, this does seem to be broken in that when the service has already been created, it doesn't fail without a role flag.

I'm not sure what you mean by "when the service is not created with the LB". I created the ELB and target groups from within the console and specified the ARN that I created in the command above. Do I need to do something other than just run the ecs-cli compose command?

from amazon-ecs-cli.

Oh, huh. I was wondering why it wasn't updating when I updated the service definition, but it seems like there's no way to update the load balancer a service uses?

It looks like it's missing from UpdateServiceInput which Up() uses under the hood. Not sure how I missed this. I see it's in the docs now. So sorry. :(

from amazon-ecs-cli.

Yeah, this was really confusing and spent about an hour on it last night. An error message would be nice. :)

Cool if I submit a PR to this effect?

from amazon-ecs-cli.

@yinshiua, any idea whether this will be created using Cloudformation or via API?

from amazon-ecs-cli.

Since I struggle with this every week, I thought I would share the script I use to deploy services with.
https://github.com/unee-t/bugzilla/blob/master/deploy.sh

Prerequisites:

1. Create the ALB & targets manually in the console
2. I assume you've setup ecsServiceRole with the AmazonEC2ContainerServiceRole policy

So I wish there was something to quickly create the ALBs, not easy thanks to a certificate dance.

And not sure what or where the role was supposed to be created. I do it manually on new accounts for want of a better way.

from amazon-ecs-cli.

Oops, @mhaggerty042
https://github.com/unee-t/bugzilla-customisation/blob/master/deploy.sh

from amazon-ecs-cli.

I see that the latest docs have already been updated with support for adding a load balancer to a service -- https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service.html

Any idea when this version will be released? Are there instructions with how to build from source?

from amazon-ecs-cli.

@SoManyHs Is what @jasonrhaas says true? Or is the best way still to generate tasks using ecs-cli compose and use aws ecs create-service with the new task definition?

from amazon-ecs-cli.

@jordantomax Yes, what @jasonrhaas mentions is correct. The docs link has changed: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cmd-ecs-cli-compose-service-up.html

Here's an example of creating a service with an ALB (target group and ALB must already exist):

ecs-cli compose service up  \
--target-group-arn <your target group ARN> \
--container-name <your container name> \
--container-port <container port to route traffic too>

We're going to close this issue; for any new requests, please open a new issue and we'll respond accordingly. Thanks everyone for the feedback, we use this to help prioritize features for the ECS CLI!

from amazon-ecs-cli.

@PettitWesley I'm happy to create a new issue if you think that's the best thing to do. However, the command you listed still isn't working for me :(. The docs you sent are for create not for up. Was that your intention? Thanks in advance.

from amazon-ecs-cli.

@jordantomax Yeah, please create a new issue if you want the ECS CLI to support creating a load balancer, as opposed to merely creating an ECS Service with an existing load balancer.

Sorry about the docs link- I've edited it to include the link for up. However, up works the same as create, it just additionally sets the desired count of the service to 1.

Also, for your problem with the command- can you create a new issue and show the exact command that you ran, etc? That would help us determine your problem. Right now, our best guess is that you might not be using the correct ARN with the target group flag- that's something we've seen before.

from amazon-ecs-cli.

@PettitWesley Got it. Actually looks like my problem was that I was specifying the flag on the compose command instead of on the up command. All working now, thank you!

from amazon-ecs-cli.

Will this parameter be added to the ecs-params.yml?

from amazon-ecs-cli.

@pecigonzalo The parameter can definitely be added to the ECS Params yaml file! Can you create an issue so that we can track that request? Also, we always welcome pull requests, especially for small improvements like this :)

from amazon-ecs-cli.

from amazon-ecs-cli.

@pecigonzalo Awesome!

from amazon-ecs-cli.