aws / elastic-load-balancing-tools Goto Github PK

Hi there,

I'm getting an error telling me that nlb is an unknown service when trying to run ./copy_network_load_balancer.py.

I'm using...
boto3 (1.4.7)
botocore (1.7.7)

The --dry-run option doesn't appear to work with the Ireland region.

user$ python copy_classic_load_balancer.py --name WINDOWS-TEST-ELB -–region eu-west-1 --dry-run
usage: copy_classic_load_balancer.py --name --region
copy_classic_load_balancer.py: error: argument --region is required
user$ python copy_classic_load_balancer.py --name WINDOWS-TEST-ELB --dry-run
usage: copy_classic_load_balancer.py --name --region
copy_classic_load_balancer.py: error: argument --region is required
user$ python copy_classic_load_balancer.py --name WINDOWS-TEST-ELB
usage: copy_classic_load_balancer.py --name --region
copy_classic_load_balancer.py: error: argument --region is required
user$ python copy_classic_load_balancer.py --name WINDOWS-TEST-ELB --region eu-west-1
Your Application Load Balancer is ready!
Application Load Balancer ARN:
arn:aws:elasticloadbalancing:eu-west-1:12345678910:loadbalancer/app/WINDOWS-TEST-ELB/2245666360f9c5ce
Target group ARNs:
Considerations:

1. If your Classic load balancer is attached to an Auto Scaling group, attach the target groups to the Auto Scaling group.
2. All HTTPS listeners use the predefined security policy.
3. To use Amazon EC2 Container Service (Amazon ECS), register your containers as targets.
   user$

I am getting an error "TypeError: a bytes-like object is required, not 'str'" when running this script from cloudshell in AWS. Should I have to modify something?

[cloudshell-user@ip-10-0-110-49 classic-load-balancer-consolelink-utility]$ python3 consolelink_classic_load_balancer.py --region us-west-2 --format html
Traceback (most recent call last):
File "consolelink_classic_load_balancer.py", line 202, in
main()
File "consolelink_classic_load_balancer.py", line 200, in main
get_html(elb_data)
File "consolelink_classic_load_balancer.py", line 161, in get_html
html_file.write(html)
TypeError: a bytes-like object is required, not 'str'
[cloudshell-user@ip-10-0-110-49 classic-load-balancer-consolelink-utility]$

We use ALB-load-balancer-controller's helm chart 1.2.7 with AWS EKS to manage our ALB.
Now we want to redirect our site from test.example.com to app.example.io but because AWS doesn't support the 308 status code to redirect the domain, all POST calls are automatically converted into the GET endpoint and because of that POST endpoints of our services start failing.
Is there any way in ingress or the load balancer to redirect to another domain with the same HTTP method that came at the source endpoint?

I don't see how this script can know which ELB should be converted to the ALB type. Maybe I'm missing something, but we have several ELBs in our account and being able to choose the one I want to convert, should be a flag where I can give the script the name of that ELB. Am I missing something?

My ELB Listener with 443 has set to ELBSecurityPolicy-TLS-1-2-2017-01. When I run copy_classic_load_balancer.py, SecurityPolicy is reset to ELBSecurityPolicy-2016-08.

I see you mentioned below in README.

Addition considerations and best practices:
2.All HTTPS listeners created will be utilizing the AWS pre-defined cipher security policy. Please modify the HTTPS listener configurations if you need to use custom policies.

ELBSecurityPolicy-TLS-1-2-2017-01 is a pre-defined policy and it's not copied.
Can this be copied as well?

Thanks,
Kay

print(botocore.version)
1.7.48
print(boto3.version)
1.4.7

client = boto3.client('comprehend')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/boto3/init.py", line 83, in client
return _get_default_session().client(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/boto3/session.py", line 263, in client
aws_session_token=aws_session_token, config=config)
File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 861, in create_client
client_config=config, api_version=api_version)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 66, in create_client
service_model = self._load_service_model(service_name, api_version)
File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 99, in _load_service_model
api_version=api_version)
File "/usr/local/lib/python3.6/site-packages/botocore/loaders.py", line 132, in _wrapper
data = func(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/loaders.py", line 378, in load_service_model
known_service_names=', '.join(sorted(known_services)))
botocore.exceptions.UnknownServiceError: Unknown service: 'comprehend'. Valid service names are: acm, apigateway, application-autoscaling, appstream, athena, autoscaling, batch, budgets, ce, clouddirectory, cloudformation, cloudfront, cloudhsm, cloudhsmv2, cloudsearch, cloudsearchdomain, cloudtrail, cloudwatch, codebuild, codecommit, codedeploy, codepipeline, codestar, cognito-identity, cognito-idp, cognito-sync, config, cur, datapipeline, dax, devicefarm, directconnect, discovery, dms, ds, dynamodb, dynamodbstreams, ec2, ecr, ecs, efs, elasticache, elasticbeanstalk, elastictranscoder, elb, elbv2, emr, es, events, firehose, gamelift, glacier, glue, greengrass, health, iam, importexport, inspector, iot, iot-data, kinesis, kinesisanalytics, kms, lambda, lex-models, lex-runtime, lightsail, logs, machinelearning, marketplace-entitlement, marketplacecommerceanalytics, meteringmarketplace, mgh, mobile, mturk, opsworks, opsworkscm, organizations, pinpoint, polly, pricing, rds, redshift, rekognition, resourcegroupstaggingapi, route53, route53domains, s3, sdb, servicecatalog, ses, shield, sms, snowball, sns, sqs, ssm, stepfunctions, storagegateway, sts, support, swf, waf, waf-regional, workdocs, workspaces, xray

Is it possible to create the load balancer that forwards to the Lambda with Cloudformation?

Thanks

I doubt this'll be a surprise, but where do I get a version of boto that supports ALBs?

File ".../elastic-load-balancing-tools/venv/local/lib/python2.7/site-packages/botocore/loaders.py", line 353, in load_service_model
known_service_names=', '.join(sorted(known_services)))
botocore.exceptions.UnknownServiceError: Unknown service: 'elbv2'. Valid service names are: acm, apigateway, application-autoscaling, autoscaling, cloudformation, cloudfront, cloudhsm, cloudsearch, cloudsearchdomain, cloudtrail, cloudwatch, codecommit, codedeploy, codepipeline, cognito-identity, cognito-idp, cognito-sync, config, datapipeline, devicefarm, directconnect, discovery, dms, ds, dynamodb, dynamodbstreams, ec2, ecr, ecs, efs, elasticache, elasticbeanstalk, elastictranscoder, elb, emr, es, events, firehose, gamelift, glacier, iam, importexport, inspector, iot, iot-data, kinesis, kms, lambda, logs, machinelearning, marketplacecommerceanalytics, meteringmarketplace, opsworks, rds, redshift, route53, route53domains, s3, sdb, servicecatalog, ses, sns, sqs, ssm, storagegateway, sts, support, swf, waf, workspaces

copy_classic_load_balancer.py --name my-load-balancer –region us-west-2 --dry-run

should two dash '--' before region parameter

I cannot get the script to work.

First it had an unstated dependency (that I was able to resolve). You need to pip install boto3.

But then it won't accept my input this is what's happening:

> ./copy_classic_load_balancer.py --name=lor-backend-LB -region=eu-east-1 --dry-run
usage: copy_classic_load_balancer.py --elb-name <elb name> --region
copy_classic_load_balancer.py: error: argument --region is required

PS. There's a typo on

When I try to use the script to copy my ELB (which was created by ElasticBeanstalk), I get the following exception:

Traceback (most recent call last): File "./copy_classic_load_balancer.py", line 466, in <module> main() File "./copy_classic_load_balancer.py", line 443, in main alb_arn = create_alb(alb_data) File "./copy_classic_load_balancer.py", line 291, in create_alb response = client.create_load_balancer(**request) File "/Users/bward/.local/lib/aws/lib/python2.7/site-packages/botocore/client.py", line 159, in _api_call return self._make_api_call(operation_name, kwargs) File "/Users/bward/.local/lib/aws/lib/python2.7/site-packages/botocore/client.py", line 494, in _make_api_call raise ClientError(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the CreateLoadBalancer operation: Tag key 'aws:cloudformation:stack-name' cannot start with 'aws:'

My classic load balancer does have several keys that begin with aws:, but they were all created by ElasticBeanstalk and I don't seem to be able to edit them.

What should I do differently to get the copy script to work?

Hi Team ,

I am looking out to implement API level throttling in ALB.
As per documents I couldn't find anything related to API throttling implementation in ALB.

could you please assist on how to implement API Throttling in ALB.

Thanks,

HI,

I use this script by this
./copy_network_load_balancer.py --name demoCLB --region ap-southeast-1 --register-targets

and the script respond following error
2017-10-20 15:45:39,965 ERROR Parameter validation failed: Unknown parameter in input: "Type", must be one of: Name, Subnets, SecurityGroups, Scheme, Tags, IpAddressType

User does not get useful information by this error message. It took a long time , I figure out that my AWS CLI is too old that it does not support aws elbv2 create-load-balancer with type network.

In order not to waste anyone life, I suggest to highlight "Upgrade your AWS CLI first!!!!" in Readme.md

Hi Team,

Is there any plan to implement Load balancer logs to cloudwatch loggroup, pushing to s3 logs is something that is outdated.

Hello,
I install python 3.6 e boto3;
i try the command "copy_classic_load_balancer.py --name elb-classic --region us-east-1 --dry-run"
and dont have success.
Debug result:
checking if ALB exists
2017-11-14 16:37:17,726 INFO Starting new HTTPS connection (1): elasticloadbalancing.us-east-1.amazonaws.com
Checking hard failure detector
Traceback (most recent call last):
File "C:\Users\salvaitor\AppData\Local\Programs\Python\Python36-32\Scripts\copy_classic_load_balancer.py", line 595, in
main()
File "C:\Users\salvaitor\AppData\Local\Programs\Python\Python36-32\Scripts\copy_classic_load_balancer.py", line 562, in main
if passed_hardfailure_detector(elb_data):
File "C:\Users\salvaitor\AppData\Local\Programs\Python\Python36-32\Scripts\copy_classic_load_balancer.py", line 178, in passed_hardfailure_detector
if elb_data['LoadBalancerDescriptions'][0]['Policies']['OtherPolicies'] > 0:
TypeError: '>' not supported between instances of 'list' and 'int'

any tips?

I am running Python 3.6.2 with boto3 version 1.4.7 and am getting the error

botocore.exceptions.UnknownServiceError: Unknown service: 'glue'

I note that the closed issue boto/boto3#1242 mentions upgrading Boto3 however 1.4.7 is the latest version

I have been investigating recently WAF/ALB logs and according to [this official doc] (https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html) the lambda_error_reason field does not exist. But instead error_reason it does?

Then all the Athena tables setup and sql queries relay on lambda_error_reason but when chatting with AWS Support they asked me to verify error_reason field? Is this the same field or am I missing here something?

Hello,

When running: ./copy_classic_load_balancer.py --name <alb name> --region us-west-2 --register-targets -- I am seeing the below error. Could this be a syntax issue with the script itself?

Traceback (most recent call last):
  File "./copy_classic_load_balancer.py", line 595, in <module>
    main()
  File "./copy_classic_load_balancer.py", line 569, in main
    alb_data = get_alb_data(elb_data, region, load_balancer_name)
  File "./copy_classic_load_balancer.py", line 299, in get_alb_data
    alb_data['listeners'].append(listener)
UnboundLocalError: local variable 'listener' referenced before assignment

I need to deploy ingress-nginx with static IP, as per their doc,

Optionally you can set controller.service.loadBalancerIP if you need a static IP for the resulting LoadBalancer.

I've tried it but it doesn't work.

Traceback (most recent call last):
File "./copy_classic_load_balancer.py", line 17, in
import boto3
ImportError: No module named boto3

To fix, I simply did
pip install boto3

But those less familiar with python might not know to take that step.

Hi,

I have to run this command.

$ ./copy_classic_load_balancer.py --name test --region ap-northeast-1 --register-targets

But, target group is not created.

I think, copy_classic_load_balancer.py#L394 is not execute.

This is the debug logs.

$ ./copy_classic_load_balancer.py --name test --region ap-northeast-1 --register-targets --debug
2016-09-25 11:01:18,441 INFO Found credentials in shared credentials file: ~/.aws/credentials
Getting existing Classic ELB data
2016-09-25 11:01:18,557 INFO Found credentials in shared credentials file: ~/.aws/credentials
2016-09-25 11:01:18,620 INFO Starting new HTTPS connection (1): elasticloadbalancing.ap-northeast-1.amazonaws.com
elb data:

(snip)

checking if ALB exists
2016-09-25 11:01:18,917 INFO Starting new HTTPS connection (1): elasticloadbalancing.ap-northeast-1.amazonaws.com
Checking hard failure detector
Checking soft failure detector
building the Application Load Balancer data structure
alb_data:

(snip)

Creating the Application Load Balancer
2016-09-25 11:01:18,993 INFO Resetting dropped connection: elasticloadbalancing.ap-northeast-1.amazonaws.com
Create Application Load Balancer response:

(snip)

Creating the target groups
Getting listeners
Adding Application Load Balancer attributes
Modify load balancer attributes response:

(snip)

Tagging the Application Load Balancer and target groups
Registering targets with the Application Load Balancer
Your Application Load Balancer is ready!
Application Load Balancer ARN:
arn:aws:elasticloadbalancing:ap-northeast-1:xxxxxxxxxxxx:loadbalancer/app/test/bc7163b71c54ee74
Target group ARNs:
Considerations:
1. If your Classic load balancer is attached to an Auto Scaling group, attach the target groups to the Auto Scaling group.
2. All HTTPS listeners use the predefined security policy.
3. To use Amazon EC2 Container Service (Amazon ECS), register your containers as targets.

Hello,

I used the tool to copy CLB to ALB, but the tool work without create one.
My CLb have two Listner, one https to http and http to http, the last one the tool forgot.

@renlon Discovered that it will duplicate a listener and not support Policies if a Policy is specified that does not have LBCookieStickinessPolicy key.

Example, classic ELB with 2 listeners

2018-09-18 10:34:00,335 INFO Starting new HTTPS connection (1): elasticloadbalancing.us-west-2.amazonaws.com
elb data:
{u'LoadBalancerAttributes': {u'AccessLog': {u'Enabled': False},
                             u'ConnectionDraining': {u'Enabled': True,
                                                     u'Timeout': 2},
                             u'ConnectionSettings': {u'IdleTimeout': 60},
                             u'CrossZoneLoadBalancing': {u'Enabled': True}},
 u'LoadBalancerDescriptions': [{u'AvailabilityZones': ['us-west-2a',
                                                       'us-west-2c'],
                                u'BackendServerDescriptions': [],
                                u'CanonicalHostedZoneName': 'xxx.us-west-2.elb.amazonaws.com',
                                u'CanonicalHostedZoneNameID': 'xxx',
                                u'CreatedTime': datetime.datetime(2018, 9, 18, 1, 41, 15, 480000, tzinfo=tzutc()),
                                u'DNSName': 'xxx.us-west-2.elb.amazonaws.com',
                                u'HealthCheck': {u'HealthyThreshold': 2,
                                                 u'Interval': 5,
                                                 u'Target': 'HTTP:80/health',
                                                 u'Timeout': 3,
                                                 u'UnhealthyThreshold': 5},
                                u'Instances': [{u'InstanceId': 'i-xxx'},
                                               {u'InstanceId': 'i-xxx'}],
                                u'ListenerDescriptions': [{u'Listener': {u'InstancePort': 80,
                                                                         u'InstanceProtocol': 'HTTP',
                                                                         u'LoadBalancerPort': 80,
                                                                         u'Protocol': 'HTTP'},
                                                           u'PolicyNames': []},
                                                          {u'Listener': {u'InstancePort': 81,
                                                                         u'InstanceProtocol': 'HTTP',
                                                                         u'LoadBalancerPort': 443,
                                                                         u'Protocol': 'HTTPS',
                                                                         u'SSLCertificateId': 'xxx'},
                                                           u'PolicyNames': ['ELBSecurityPolicy-2016-08']}],
...
}

Resultant: alb_data

alb_data:
{'Alb_name': 'xxx',
 'Region': 'xxx',
 'Scheme': 'internet-facing',
 'Security_groups': ['xxx'],
 'Subnets': ['subnet-xxx', 'subnet-xxx'],
 'Tags': [],
 'VpcId': 'vpc-xxx',
 'attributes': [{'Key': 'idle_timeout.timeout_seconds', 'Value': '60'}],
 'instanceIds': ['i-xxx', 'i-xxx'],
 'listeners': [{'Port': 80,
                'Protocol': 'HTTP',
                'TargetGroup_Port': 80,
                'TargetGroup_Protocol': 'HTTP'},
               {'Port': 80,
                'Protocol': 'HTTP',
                'TargetGroup_Port': 80,
                'TargetGroup_Protocol': 'HTTP'}],
...
}

The below diff is a refactor that handles the Policy Name and presence of the LBCookieStickinessPolicy key and prevents duplicate listeners due to listener previously not being initialized at the beginning of the loop for absent else condition handling.

diff --git a/application-load-balancer-copy-utility/copy_classic_load_balancer.py b/application-load-balancer-copy-utility/copy_classic_load_balancer.py
index eb9abec..5018dad 100755
--- a/application-load-balancer-copy-utility/copy_classic_load_balancer.py
+++ b/application-load-balancer-copy-utility/copy_classic_load_balancer.py
@@ -262,15 +262,15 @@ def get_alb_data(elb_data, region, load_balancer_name):
 
     # this is used for building the listeners specs
     for elb_listener in elb_data['LoadBalancerDescriptions'][0]['ListenerDescriptions']:
-        # If there is a LBCookieStickinessPolicy, append TG attriubtes
+        TargetGroup_Attribute = {
+            'dereg_timeout_seconds_delay': str(elb_data['LoadBalancerAttributes']['ConnectionDraining']['Timeout']),
+            'TargetGroup_Port': elb_listener['Listener']['InstancePort']
+        }
         if len(elb_listener['PolicyNames']) > 0:
-            if 'LBCookieStickinessPolicy' in elb_listener['PolicyNames'][0]:
-                for policy in elb_data['PolicyDescriptions']:
-                    if elb_listener['PolicyNames'][0] == policy['PolicyName']:
-                        listener = {'Protocol': elb_listener['Listener']['Protocol'],
-                                    'Port': elb_listener['Listener']['LoadBalancerPort'],
-                                    'TargetGroup_Port': elb_listener['Listener']['InstancePort'],
-                                    'TargetGroup_Protocol': elb_listener['Listener']['InstanceProtocol']}
+            for policy in elb_data['PolicyDescriptions']:
+                if elb_listener['PolicyNames'][0] == policy['PolicyName']:
+                    # If there is a LBCookieStickinessPolicy, append TG attriubtes
+                    if 'LBCookieStickinessPolicy' in elb_listener['PolicyNames'][0]:
                         TargetGroup_Attribute = {
                             'dereg_timeout_seconds_delay': str(elb_data['LoadBalancerAttributes']['ConnectionDraining']['Timeout']),
                             'stickiness.enabled': 'true',
@@ -279,21 +279,14 @@ def get_alb_data(elb_data, region, load_balancer_name):
                             'stickiness.lb_cookie.duration_seconds': policy['PolicyAttributeDescriptions'][0]['AttributeValue'],
                             'TargetGroup_Port': elb_listener['Listener']['InstancePort']
                         }
-                        if listener['Protocol'] == "HTTPS":
-                            listener['Certificates'] = [
-                                {'CertificateArn': elb_listener['Listener']['SSLCertificateId']}]
-        else:
-            listener = {'Protocol': elb_listener['Listener']['Protocol'],
-                        'Port': elb_listener['Listener']['LoadBalancerPort'],
-                        'TargetGroup_Port': elb_listener['Listener']['InstancePort'],
-                        'TargetGroup_Protocol': elb_listener['Listener']['InstanceProtocol']}
-            TargetGroup_Attribute = {
-                'dereg_timeout_seconds_delay': str(elb_data['LoadBalancerAttributes']['ConnectionDraining']['Timeout']),
-                'TargetGroup_Port': elb_listener['Listener']['InstancePort']
-            }
-            if listener['Protocol'] == "HTTPS":
-                listener['Certificates'] = [
-                    {'CertificateArn': elb_listener['Listener']['SSLCertificateId']}]
+                        break
+        listener = {'Protocol': elb_listener['Listener']['Protocol'],
+                    'Port': elb_listener['Listener']['LoadBalancerPort'],
+                    'TargetGroup_Port': elb_listener['Listener']['InstancePort'],
+                    'TargetGroup_Protocol': elb_listener['Listener']['InstanceProtocol']}
+        if listener['Protocol'] == "HTTPS":
+            listener['Certificates'] = [
+                {'CertificateArn': elb_listener['Listener']['SSLCertificateId']}]
         # TGs is not per unique backend port as two TGs might have two
         # different stickiness policy
         alb_data['listeners'].append(listener)