amazon-archives / aws-serverlessrepo-python Goto Github PK
View Code? Open in Web Editor NEWA Python library with convenience helpers for working with the AWS Serverless Application Repository.
License: Apache License 2.0
A Python library with convenience helpers for working with the AWS Serverless Application Repository.
License: Apache License 2.0
Hi, I find the last version of serverlessrepo 0.1.6 using pipenv package, but If I using pip install that it will conflict with other tools like localstack.
serverlessrepo 0.1.6 has requirement six~=1.11.0, but you'll have six 1.12.0 which is incompatible. *********error here***********
Installing collected packages: six, airspeed, localstack
Found existing installation: six 1.10.0
Uninstalling six-1.10.0:
Successfully uninstalled six-1.10.0
Found existing installation: airspeed 0.5.5.dev20160812
Uninstalling airspeed-0.5.5.dev20160812:
Successfully uninstalled airspeed-0.5.5.dev20160812
Successfully installed airspeed-0.5.10 localstack-0.8.10 six-1.12.0
the last version localstack require six 1.12.0 while the serverlessrepo still lock the six version 1.11.0. Can you make your Pipfile.lock file using
"six": {
"hashes": [
"sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
"sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
],
"version": "==1.11.0"
},
to
"version": ">=1.11.0"
since it will block six or other dependence upgrade later on.
$ pip install serverlessrepo
Collecting serverlessrepo
Could not find a version that satisfies the requirement serverlessrepo (from versions: )
No matching distribution found for serverlessrepo
$ python --version
Python 3.5.2
First started to fail 2 days ago: https://travis-ci.org/sblask/dotfiles/builds
Seems to be a sudden incompatibility to Python 3.5 which is the standard Python in Ubuntu 16.04 (LTS)
When using sam publish command to publish a new version of a SAR app, if I forget to update the semantic version in the template metadata, it succeeds leading the user to think they published a new version.
We use this script when publishing to SAR:
https://github.com/awslabs/aws-athena-query-federation/blob/master/tools/publish.sh
It has the exact commands that get run. The steps to reproduce are:
You'll find that step #4 doesn't throw and error but that after step 5 which also won't error, the deployed app will still be running the code from step 1.
Step #4 should somehow throw an error or helpful information to indicate a new version is not published so that we can stop before running step 5.
When trying to run sam publish -t packaged.yaml
to update an application that already exists I get the following error:
Traceback (most recent call last):
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/serverlessrepo/publish.py", line 45, in publish_application
response = sar_client.create_application(**request)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 635, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ConflictException: An error occurred (ConflictException) when calling the CreateApplication operation: Application with name NewRelic-log-ingestion already exists
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/kolanos/.virtualenvs/aws/bin/sam", line 8, in <module>
sys.exit(cli())
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 96, in wrapped
raise exception # pylint: disable=raising-bad-type
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
return_value = func(*args, **kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 54, in cli
do_cli(ctx, template_file, semantic_version) # pragma: no cover
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 77, in do_cli
publish_output = publish_application(template_data)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/serverlessrepo/publish.py", line 57, in publish_application
sar_client.update_application(**request)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 316, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 608, in _make_api_call
api_params, operation_model, context=request_context)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/client.py", line 656, in _convert_to_request_dict
api_params, operation_model)
File "/Users/kolanos/.virtualenvs/aws/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Missing required parameter in input: "ApplicationId"
I'm running serverlessrepo==0.1.10
. But tried with 0.1.9
and got the same error.
The following line appears to fail to parse an application ID in serverlessrepo.publish:publish_application
:
application_id = parse_application_id(error_message)
Description:
Don't know if this is a CloudFormation bug or a SAM bug but since the error appears to be coming from the SAM CLI stack I'm making it here.
I'm unable to utilize CloudFormation functions for parameters in my template under the AWS::ServerlessRepo::Application Metadata section to deploy my application to the Serverless Application Repo (SAR).
Steps to reproduce the issue:
Here's my template.yaml
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: Lambda layer to house all shared code with backend apps
Parameters:
DeploymentEnvironment:
Type: String
Default: dev
Resources:
NodeModulesLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: !Sub 'serverless-shared-node-modules-${DeploymentEnvironment}-grant'
ContentUri: ./
CompatibleRuntimes:
- nodejs8.10
- nodejs10.x
# could specify Delete if we want to clean up layers?
RetentionPolicy: Retain
Outputs:
LayerVersion:
Description: Node Modules Layer ARN Reference
Value: !Ref NodeModulesLayer
Metadata:
AWS::ServerlessRepo::Application:
Name: !Sub serverless-shared-node-modules-${DeploymentEnvironment}-grant
Description: >
Lambda layer to house all Node-based shared code with backend/serverless apps
Author: grant
SemanticVersion: 1.0.0
ReadmeUrl: README.md
SourceCodeUrl: <url>
lambda-layers-dev
sam package --template-file template.yaml --s3-bucket lambda-layers-dev --output-template-file packaged.yaml
sam publish --template packaged.yaml
Observed result:
I get the following errors stack trace:
Uploading to d533bb334dcca7b4904d6e56c7e319a3 107675 / 107675.0 (100.00%)
Successfully packaged artifacts and wrote output template to file packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file <path>lambda-layers/packaged.yaml --stack-name <YOUR STACK NAME>
$ sam publish --template packaged.yaml
2019-10-25 10:24:56 Found credentials in shared credentials file: ~/.aws/credentials
Traceback (most recent call last):
File "/usr/local/bin/sam", line 11, in <module>
load_entry_point('aws-sam-cli==0.23.0', 'console_scripts', 'sam')()
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 93, in wrapped
raise exception # pylint: disable=raising-bad-type
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
return_value = func(*args, **kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 52, in cli
do_cli(ctx, template, semantic_version) # pragma: no cover
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/samcli/commands/publish/command.py", line 75, in do_cli
publish_output = publish_application(template_data)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/serverlessrepo/publish.py", line 45, in publish_application
response = sar_client.create_application(**request)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 634, in _make_api_call
api_params, operation_model, context=request_context)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/client.py", line 682, in _convert_to_request_dict
api_params, operation_model)
File "/usr/local/Cellar/aws-sam-cli/0.23.0/libexec/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Name, value: OrderedDict([('Fn::Sub', 'serverless-shared-node-modules-${DeploymentEnvironment}-ga892')]), type: <class 'collections.OrderedDict'>, valid types: <class 'str'>
Fn::Sub:
, Fn::Join:
and !Sub
. All throw the same error.Expected result:
When I change in my template the Name
property under the ServerlessRep to just a string with no CF functions the layer publishes fine.
I referenced this page for the params for that section:
https://docs.aws.amazon.com/en_pv/serverless-application-model/latest/developerguide/serverless-sam-template-publishing-applications-metadata-properties.html
Sorry if this is me not understanding/finding some of the documentation and if so let me know what I missed. Thanks!
SAM CLI was cut this issue, which outputs a stack trace when Unsupported Resources are within the template that is being published.
I think it makes sense to include some additional error handling within this library that provides a better error that the boto3 error that gets returned. This is similar to wrapping of the Botocore ClientError that this library already wraps.
Add a UnsupportedResourcesError
in exceptions.py
. We can then catch botocore.errorfactory.BadRequestException
in this try: except:
block and wrap it in the UnsupportedResourcesError with a message of "The template contains unsupported resource types. [XX::XX::XX]", which can be parsed out of the exception.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.