localstack / localstack Goto Github PK

💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline

License: Other

Makefile 0.31% Python 98.88% JavaScript 0.17% Java 0.10% Shell 0.08% C# 0.03% Dockerfile 0.07% Ruby 0.01% Batchfile 0.01% HCL 0.05% Go 0.01% HTML 0.05% Rust 0.01% ANTLR 0.15% Jinja 0.08%

aws localstack testing continuous-integration developer-tools python cloud

localstack's Introduction

⚡ We are thrilled to announce the release of LocalStack 3.4 ⚡

LocalStack is a cloud software development framework to develop and test your AWS applications locally.

Overview • Install • Quickstart • Run • Usage • Releases • Contributing
📖 Docs • 💻 Pro version • ☑️ LocalStack coverage

Overview

LocalStack is a cloud service emulator that runs in a single container on your laptop or in your CI environment. With LocalStack, you can run your AWS applications or Lambdas entirely on your local machine without connecting to a remote cloud provider! Whether you are testing complex CDK applications or Terraform configurations, or just beginning to learn about AWS services, LocalStack helps speed up and simplify your testing and development workflow.

LocalStack supports a growing number of AWS services, like AWS Lambda, S3, Dynamodb, Kinesis, SQS, SNS, and many more! The Pro version of LocalStack supports additional APIs and advanced features. You can find a comprehensive list of supported APIs on our ☑️ Feature Coverage page.

LocalStack also provides additional features to make your life as a cloud developer easier! Check out LocalStack's User Guides for more information.

Installation

The quickest way get started with LocalStack is by using the LocalStack CLI. It enables you to start and manage the LocalStack Docker container directly through your command line. Ensure that your machine has a functional docker environment installed before proceeding.

Brew (macOS or Linux with Homebrew)

Install the LocalStack CLI through our official LocalStack Brew Tap:

brew install localstack/tap/localstack-cli

Binary download (MacOS, Linux, Windows)

If Brew is not installed on your machine, you can download the pre-built LocalStack CLI binary directly:

Visit localstack/localstack-cli and download the latest release for your platform.
Extract the downloaded archive to a directory included in your PATH variable:
- For MacOS/Linux, use the command: sudo tar xvzf ~/Downloads/localstack-cli-*-darwin-*-onefile.tar.gz -C /usr/local/bin

PyPI (MacOS, Linux, Windows)

LocalStack is developed using Python. To install the LocalStack CLI using pip, run the following command:

python3 -m pip install localstack

The localstack-cli installation enables you to run the Docker image containing the LocalStack runtime. To interact with the local AWS services, you need to install the awslocal CLI separately. For installation guidelines, refer to the awslocal documentation.

Important: Do not use sudo or run as root user. LocalStack must be installed and started entirely under a local non-root user. If you have problems with permissions in macOS High Sierra, install with pip install --user localstack

Quickstart

Start LocalStack inside a Docker container by running:

 % localstack start -d

     __                     _______ __             __
    / /   ____  _________ _/ / ___// /_____ ______/ /__
   / /   / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
  / /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
 /_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|

 💻 LocalStack CLI 3.4.0
 👤 Profile: default

[12:47:13] starting LocalStack in Docker mode 🐳                       localstack.py:494
           preparing environment                                       bootstrap.py:1240
           configuring container                                       bootstrap.py:1248
           starting container                                          bootstrap.py:1258
[12:47:15] detaching                                                   bootstrap.py:1262

You can query the status of respective services on LocalStack by running:

% localstack status services
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Service                  ┃ Status      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ acm                      │ ✔ available │
│ apigateway               │ ✔ available │
│ cloudformation           │ ✔ available │
│ cloudwatch               │ ✔ available │
│ config                   │ ✔ available │
│ dynamodb                 │ ✔ available │
...

To use SQS, a fully managed distributed message queuing service, on LocalStack, run:

% awslocal sqs create-queue --queue-name sample-queue
{
    "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/sample-queue"
}

Learn more about LocalStack AWS services and using them with LocalStack's awslocal CLI.

Running

You can run LocalStack through the following options:

Usage

To start using LocalStack, check out our documentation.

To use LocalStack with a graphical user interface, you can use the following UI clients:

Releases

Please refer to GitHub releases to see the complete list of changes for each release. For extended release notes, please refer to the LocalStack Discuss.

Contributing

If you are interested in contributing to LocalStack:

Start by reading our contributing guide.
Check out our developer guide.
Navigate our codebase and open issues.

We are thankful for all the contributions and feedback we receive.

Get in touch

Get in touch with the LocalStack Team to report 🐞 issues, upvote 👍 feature requests, 🙋🏽 ask support questions, or 🗣️ discuss local cloud development:

Contributors

We are thankful to all the people who have contributed to this project.

Backers

We are also grateful to all our backers who have donated to the project. You can become a backer on Open Collective.

License

This version of LocalStack is released under the Apache License, Version 2.0 (see LICENSE). By downloading and using this software you agree to the End-User License Agreement (EULA). To know about the external software we use, look at our third party software tools page.

localstack's People

Contributors

Stargazers

Watchers

Forkers

msempere hemel-cse bladym pierreliddle sqlheisenberg accesso simplehq c00ler dtransmission terefds robotzero beejhuff nagyistge ashwiniverma mukesh23singh harendranathvegi9 the-interesting-times-gang codefresh-io cloud-devops costingalan rezhajulio kstdennis pmmarq cfirmo33 rkhozinov montazze grassato augmify prateeknayak dionisiolopez maniacs-ops siddharthgopi dyhpoon hkachetel cooganb anhntbk08 code-monkeys jpdro anderson4u2 joshuaroot codesurf42 timnee arunsanna juwlee bibhor cloudnuts supaflysnooka tamiral mauricionr user09110 manjufy xmanju dkokoska cloudacademy mrsinguyen hbcbh1999 nangal denistsoi paddymahoney moandcompany arrmac aleksandergondek michft curtiszimmerman prayagverma yunleiz stenioaraujo chosen1 vdt stvhanna rayning0 wlmgithub chungnt201 suyear superpan languageplayground xennygrimmato bdjibril cvesh swachalit hafeez3000 nocrates helols eywalker opsguru jgasmi unws lyuggang iervolino chinavolvocars reyadrahman khorkov odolhado abhishekgahlot zmilan lemonhall alexsjones redavis22 neo4reo bitroniq

localstack's Issues

Support for redshift

Not sure if it's possible but it would be great to have redshift support

Create simple JUnit test to demonstrate integration with Java

We should provide a seamless integration of LocalStack with other languages apart from Python.

Let's start with Java - add an exemplary JUnit test class that spins up local cloud services as part of a setup method, runs some simple tests, and finally terminates the services on teardown.

Add feature to inject ProvisionedThroughputExceededException for testing

Making application code resilient to Kinesis' ProvisionedThroughputExceededException is critical.
We should allow users to inject this kind of exception in their integration tests.

We can re-use the Kinesis proxy mechanism to inject this exception here:
https://github.com/atlassian/localstack/blob/master/localstack/mock/infra.py#L368

create-queue fails from aws-cli 1.11.74

Calling create-queue from the aws-cli (v1.11.74) on Mac OSX Sierra (10.12.4) responds with 500 error.

Docker command:
osxuk52234:localstack rday$ docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack

Response:

Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
Ready.

AWS CLI Command:
osxuk52234:~ rday$ aws --endpoint-url=http://localhost:4576 sqs create-queue --queue-name QueueName

Response:

Unable to parse response (syntax error: line 1, column 54), invalid XML received:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request.  Either the server is overloaded or there is an error in the application.</p>

This occurs with both the docker image (current and latest: docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack, docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack:0.3.4) and running from source with make infra.

Any idea on " Timeout when waiting for KCL initialization. "

I install follow the step. And run the tests.test_integration get error:


# python -m tests.test_integration

Starting local Elasticsearch (port 4571)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
Ready.
Creating stream...
Shutdown
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/ubuntu/localstack/tests/test_integration.py", line 191, in <module>
    start_test(env=env)
  File "/home/ubuntu//localstack/tests/test_integration.py", line 137, in start_test
    wait_until_started=True)
  File "localstack/utils/kinesis/kinesis_connector.py", line 424, in listen_to_kinesis
    raise Exception('Timeout when waiting for KCL initialization.')
Exception: Timeout when waiting for KCL initialization.](url)

Question: Advice on integrating aws-mock into localstack?

Background: https://github.com/treelogic-swe/aws-mock has good support for mocking EC2, which I don't see is available in localstack.

How would one go about integrating the EC2 mocking support in aws-mock with localstack? Where is the integration point for a new AWS mock service, and what is its API?

Cloudformation

Are there any plans to support Cloudformation at any point?

Intercept SNS calls to decouple SNS/SQS and forward messages to separate SQS process

The principle of LocalStack is to use isolated services (in separate processes). Moto currently has strong coupling between SNS and SQS when using SQS as a consumer of an SNS topic.

We need to intercept SNS calls to provide full decoupling between SNS/SQS. The idea is to keep a hashmap of SNS subscriptions and forward intercepted messages to separate SQS process via HTTP.

LocalstackTestRunner: getEndpointSQS() points to "sns"

I was trying out the LocalstackTestRunner in JUnit and was unable to get SQS client working against the localstack endpoint. Turns out the getEndpointSQS() method references "sns". I made the change to "sqs", rebuilt the artifact, and I get the expected results. Can you please provide a quick fix for this? I have the code change including a unit test if you want it. Thanks!

S3 Create Bucket

Stumped and fishing here...getting an UnknownHostException when I try to create a bucket. I can return a list of empty buckets as shown below, but when I s3.createBucket("hellobucket") it can't resolve the host: "hellobucket.localhost". I've looked through both the localstack project (and the virtual env folder created .pyenv) and can't find any mention where I could manually make a bucket.

@org.junit.Test
public void Test1() {
    String s = "http://localhost:4572/";
    AmazonS3 s3 = new AmazonS3Client(TEST_CREDENTIALS);
    s3.setEndpoint(s);
    System.out.println(s3.listBuckets().toString());

    s3.createBucket("hellobucket");

}

[]

com.amazonaws.SdkClientException: Unable to execute HTTP request: hellobucket.localhost

S3 whether to support ssl？

There is no configuration can let s3 support ssl or use proxy？

Creating DB and tables in Redshift

I am still new to localstack and was wondering if we will be able to create DB/tables in redshift mock service and execute sql commands ?.

SQS receive-message delays/sleeps

I am observing that calls to receive-message on SQS seem to take quite a long time.

Here are the details -

C02STG51GTFM:localstack mpandit$  aws --endpoint-url=http://localhost:4576 sqs send-message --queue-url http://localhost:4576/123456789012/test_queue --message-body 'Test Message!'
{
    "MD5OfMessageBody": "df69267381a60e476252c989db9ac8ad", 
    "MessageId": "dc8cd4d1-53ec-b8a3-63d2-457fb4ac4019"
}
C02STG51GTFM:localstack mpandit$ time aws --endpoint-url=http://localhost:4576 sqs receive-message --queue-url http://localhost:4576/123456789012/test_queue
{
    "Messages": [
        {
            "Body": "Test Message!", 
            "Attributes": {
                "ApproximateFirstReceiveTimestamp": "1.49138201347e+12", 
                "SenderId": "AIDAIT2UOQQY3AUEKVGXU", 
                "ApproximateReceiveCount": "2", 
                "SentTimestamp": "1.49138201346e+12"
            }, 
            "ReceiptHandle": "guoazsjkomdujlaegxlqzauynkketuxvcomewlvhbdkjyifrgirnobfxifjvkuztoeeuwkkmnxkbelyiyhbrldqgfpydmywtmoykjrimkhiezejsbxvotivjvinudnxxyjltnaoecaduxplgpyrigeqwgsdbhoqtsotysguttwcpmzonqmhnemuuc", 
            "MD5OfBody": "df69267381a60e476252c989db9ac8ad", 
            "MessageId": "dc8cd4d1-53ec-b8a3-63d2-457fb4ac4019"
        }
    ]
}

real	0m26.498s
user	0m0.309s
sys	0m0.168s
C02STG51GTFM:localstack mpandit$ time aws --endpoint-url=http://localhost:4576 sqs receive-message --queue-url http://localhost:4576/123456789012/test_queue
{
    "Messages": [
        {
            "Body": "Test Message!", 
            "Attributes": {
                "ApproximateFirstReceiveTimestamp": "1.49138201347e+12", 
                "SenderId": "AIDAIT2UOQQY3AUEKVGXU", 
                "ApproximateReceiveCount": "3", 
                "SentTimestamp": "1.49138201346e+12"
            }, 
            "ReceiptHandle": "xsxjduahofaqqisgjybfpfzzqgceiaiduscqnygkhalvzzfqrsltbbavhopjbqptrazwdjriuahtlswdezpnlflvxqaualqmabwiobecdttrzwpnjjvzekmbbgnjuvwvfwwoipeqbcctlcujtlzvxdvoqszmmhkrrxfhdfuzbfgebxugcihezefjv", 
            "MD5OfBody": "df69267381a60e476252c989db9ac8ad", 
            "MessageId": "dc8cd4d1-53ec-b8a3-63d2-457fb4ac4019"
        }
    ]
}

real	0m19.425s
user	0m0.281s
sys	0m0.162s

docker error:docker is looping through it self because of an error

docker is looping through it self,
tried docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack
and make docker-run

both giving me this result
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:43:27,737 INFO exited: infra (exit status 2; expected)
2017-04-14 04:43:28,762 INFO spawned: 'infra' with pid 586
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:43:30,541 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

ERROR checking state of local environment (after some retries): Traceback (most recent call last):
File "localstack/mock/infra.py", line 272, in check_infra
check_infra_elasticsearch(expect_shutdown=expect_shutdown, print_error=print_error)
File "localstack/mock/infra.py", line 255, in check_infra_elasticsearch
assert isinstance(out, basestring)
AssertionError

Traceback (most recent call last):
File "localstack/mock/infra.py", line 377, in
start_infra()
File "localstack/mock/infra.py", line 353, in start_infra
check_infra(apis=apis)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 278, in check_infra
raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:43:57,034 INFO exited: infra (exit status 2; expected)
2017-04-14 04:43:58,052 INFO spawned: 'infra' with pid 756
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:43:59,586 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

Traceback (most recent call last):
File "localstack/mock/infra.py", line 377, in
start_infra()
File "localstack/mock/infra.py", line 353, in start_infra
check_infra(apis=apis)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 278, in check_infra
raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:44:26,244 INFO exited: infra (exit status 2; expected)
2017-04-14 04:44:27,263 INFO spawned: 'infra' with pid 926
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:44:28,786 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
^C2017-04-14 04:44:31,338 WARN received SIGINT indicating exit request
2017-04-14 04:44:31,339 INFO waiting for dashboard, infra to die
2017-04-14 04:44:32,346 INFO stopped: infra (terminated by SIGTERM)
make: *** [Makefile:63: web] Terminated
2017-04-14 04:44:32,348 INFO stopped: dashboard (terminated by SIGTERM)
slmbp1607:localstack apurvakulkarni$ make docker-run
port_mappings="";
docker run -it -e DEBUG= -e SERVICES= -e KINESIS_ERROR_PROBABILITY= -e SERVICES= -p 4567-4578:4567-4578 -p 8080:8080 $port_mappings atlassianlabs/localstack
2017-04-14 04:44:57,556 INFO supervisord started with pid 1
2017-04-14 04:44:58,559 INFO spawned: 'dashboard' with pid 9
2017-04-14 04:44:58,561 INFO spawned: 'infra' with pid 10
(. .venv/bin/activate; bin/localstack web --port=8080)
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.

Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
Restarting with stat
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
Debugger is active!
Debugger PIN: 135-536-443
2017-04-14 04:45:00,336 INFO success: dashboard entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-14 04:45:00,337 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

Traceback (most recent call last):
File "localstack/mock/infra.py", line 377, in
start_infra()
File "localstack/mock/infra.py", line 353, in start_infra
check_infra(apis=apis)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 278, in check_infra
raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:45:27,085 INFO exited: infra (exit status 2; expected)
2017-04-14 04:45:28,103 INFO spawned: 'infra' with pid 191
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:45:29,864 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

Traceback (most recent call last):
File "localstack/mock/infra.py", line 377, in
start_infra()
File "localstack/mock/infra.py", line 353, in start_infra
check_infra(apis=apis)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 278, in check_infra
raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:45:56,171 INFO exited: infra (exit status 2; expected)
2017-04-14 04:45:57,191 INFO spawned: 'infra' with pid 417
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:45:58,696 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
^C2017-04-14 04:45:59,811 WARN received SIGINT indicating exit request
2017-04-14 04:45:59,813 INFO waiting for dashboard, infra to die
^C2017-04-14 04:46:00,007 WARN received SIGINT indicating exit request
^C2017-04-14 04:46:00,199 WARN received SIGINT indicating exit request
^C2017-04-14 04:46:00,363 WARN received SIGINT indicating exit request
^C2017-04-14 04:46:00,543 WARN received SIGINT indicating exit request
^C2017-04-14 04:46:00,719 WARN received SIGINT indicating exit request
2017-04-14 04:46:00,821 INFO stopped: infra (terminated by SIGTERM)
make: *** [Makefile:63: web] Terminated
2017-04-14 04:46:00,823 INFO stopped: dashboard (terminated by SIGTERM)
slmbp1607:localstack apurvakulkarni$ make infra
. .venv/bin/activate; exec localstack/mock/infra.py
/bin/sh: .venv/bin/activate: No such file or directory
make: *** [infra] Error 1
slmbp1607:localstack apurvakulkarni$ docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack
2017-04-14 04:47:08,404 INFO supervisord started with pid 1
2017-04-14 04:47:09,408 INFO spawned: 'dashboard' with pid 8
2017-04-14 04:47:09,410 INFO spawned: 'infra' with pid 9
(. .venv/bin/activate; bin/localstack web --port=8080)
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.

Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
Restarting with stat
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
Debugger is active!
Debugger PIN: 135-536-443
2017-04-14 04:47:11,126 INFO success: dashboard entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-14 04:47:11,127 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

Traceback (most recent call last):
File "localstack/mock/infra.py", line 377, in
start_infra()
File "localstack/mock/infra.py", line 353, in start_infra
check_infra(apis=apis)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 280, in check_infra
check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
File "localstack/mock/infra.py", line 278, in check_infra
raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-14 04:47:37,906 INFO exited: infra (exit status 2; expected)
2017-04-14 04:47:38,924 INFO spawned: 'infra' with pid 211
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-14 04:47:40,431 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
^C2017-04-14 04:47:59,485 WARN received SIGINT indicating exit request
2017-04-14 04:47:59,486 INFO waiting for dashboard, infra to die
^C2017-04-14 04:47:59,640 WARN received SIGINT indicating exit request
^C2017-04-14 04:47:59,817 WARN received SIGINT indicating exit request
2017-04-14 04:48:00,495 INFO stopped: infra (terminated by SIGTERM)
make: *** [Makefile:63: web] Terminated
2017-04-14 04:48:00,498 INFO stopped: dashboard (terminated by SIGTERM)

CORS-support in s3

Looks like, the current implementation of S3 module does not support CORS in any way. I've tried to set CORS on a bucket using cli, it responds w/o error, but no CORS are actually set -- i can't get them via get-bucket-cors and i see js-exceptions when i'm trying to use signed urls from aws API. The same behaviour holds when i'm trying to do this via java API.

Are there any plans to add CORS support?

Make startup services and ports configurable via Docker

Currently the Docker container spins up the default configuration which covers the entire set of services.

We should add a configuration option (e.g., environment variable) that can be passed to the container to customize the set of services to be spun up. Also the local port range should be configurable.

SyntaxError: Missing parentheses in call to 'print'

Collecting localstack
Downloading localstack-0.3.2.tar.gz (282kB)
100% |################################| 286kB 677kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users#####\AppData\Local\Temp\pip-build-ei7ef_g0\localstack\setup.py", line 40
print e.output
^
SyntaxError: Missing parentheses in call to 'print'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in *C:\Users#####\AppData\Local\Temp\pip-build-ei7ef_g0\localstack*

Using Python 3.5.2

Randomized errors for dynamodb mock

Add a way to throw errors randomly for ddb read/writes for testing purposes

See #13
The same feature should also be applied for simulating DynamoDB read/write throttles.
In both cases, we can introduce ERROR_PROBABILITY config variables in the range [0,1] which define the probability for the framework to inject throughput exceeded exceptions.

ElasticSearch ConnectionError

After building the Docker image by myself, I try to launch the whole stack and ElasticSearch gives me the following error:

('check_infra_elasticsearch exception', ConnectionError('N/A', "HTTPConnectionPool(host='localhost', port=4571): Max retries exceeded with url: /_aliases (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f64882a74d0>: Failed to establish a new connection: [Errno 111] Connection refused',))", ConnectionError(MaxRetryError("HTTPConnectionPool(host='localhost', port=4571): Max retries exceeded with url: /_aliases (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f64882a74d0>: Failed to establish a new connection: [Errno 111] Connection refused',))",),)))

If I exclude ES to be provisioned in the stack, eveything works fine.
The entire stack (ES included) works if I pull the image from the Docker hub.

Any guess what may cause it?

Setting up credentials with localstack

Hi,

We are trialing Atlassian localstack for mocking out Amazon Kinesis in our application.
I can start up the amazon services locally using the provided docker image.

I create a stream on my mocked Kinesis using the following command
aws --endpoint-url=http://localhost:4568 kinesis create-stream --stream-name "integration-test" --shard-count 1
I see the stream was created using the following command:
aws --endpoint-url=http://localhost:4568 kinesis describe-stream --stream-name "integration-test"
"StreamDescription": {
"StreamName": "integration-test",
"StreamARN": "arn:aws:kinesis:us-east-1:000000000000:stream/integration-test",
"StreamStatus": "ACTIVE"
}

The arn is arn:aws:kinesis:us-east-1:000000000000:stream/integration-test

I couldn't find any information in the wiki/readme how I go about setting up my credentials to point at this test stream?
In my tests I can configure the AWS_DEFAULT_REGION. I'm just wondering does anyone know how to configure a global account to match 000000000000 above? It keeps picking up my real amazon account from my credentials.
Also, I specify --endpoint-url in the amazon cli command above. Is it possible to set this globally so that it can be picked up by my aws sdk?

Its probably obvious to most but maybe updating the readme with some instructions how to create these to point at the mocked service

Cheers

Cognito emulation

Hi,

In our tests we use a large number of made-up users.
It would be nice if we could have a local user pool (to which we could add these users programmatically) and authentication service.

Best regards,
Csaba

┆Issue is synchronized with this Jira Bug by Unito

Use lambci/docker-lambda for local lambda execution?

Feel free to close this but might be worth considering using https://github.com/lambci/docker-lambda to execute lambdas. Seems they dumped the filesystem of a live lambda and made a container out of it. Neat.

Crash, when trying to create bucket on s3

With the latest docker localstack and Amazon S3 PHP libary

 docker run -it -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack

Error log:

Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
  File "localstack/mock/infra.py", line 248, in check_infra_elasticsearch
    out = es.cat.aliases()
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
    'aliases', name), params=params)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
    raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

ERROR checking state of local environment (after some retries): Traceback (most recent call last):
  File "localstack/mock/infra.py", line 272, in check_infra
    check_infra_elasticsearch(expect_shutdown=expect_shutdown, print_error=print_error)
  File "localstack/mock/infra.py", line 255, in check_infra_elasticsearch
    assert isinstance(out, basestring)
AssertionError

Traceback (most recent call last):
  File "localstack/mock/infra.py", line 377, in 
    start_infra()
  File "localstack/mock/infra.py", line 353, in start_infra
    check_infra(apis=apis)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 280, in check_infra
    check_infra(retries - 1, expect_shutdown=expect_shutdown, apis=apis, additional_checks=additional_checks)
  File "localstack/mock/infra.py", line 278, in check_infra
    raise e
AssertionError
make: *** [Makefile:49: infra] Error 1
2017-04-20 11:27:27,937 INFO exited: infra (exit status 2; expected)
2017-04-20 11:27:28,945 INFO spawned: 'infra' with pid 190

how to connect to s3 from presto

in presto:
create schema fake with (location='s3://fake/org/')
return error:
Query 20170408_181852_00108_w3kra failed: Got exception: java.io.IOException No FileSystem for scheme: s3

so , how to add --endpoint-url='http://localhost:4572'?

how to config s3 in luigi

luigi code:

import luigi
from luigi.contrib.s3 import S3Target, S3Client


class A(luigi.Task):
    def output(self):
        client = S3Client('aws_access_key_id', 'aws_secret_access_key', host='localhost', port=4572)
        return S3Target('s3://fake/test', client=client)

    def run(self):
        with self.output().open('w') as f:
            f.write('test')

luigi log:

2017-03-26 16:03:13,543 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.57ms
2017-03-26 16:03:14,552 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.58ms
2017-03-26 16:03:15,557 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.57ms
2017-03-26 16:03:16,568 tornado.access[58422] INFO: 200 GET /api/ping (::1) 1.37ms
2017-03-26 16:03:17,576 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.66ms
2017-03-26 16:03:18,587 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.79ms
2017-03-26 16:03:19,598 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.69ms
2017-03-26 16:03:20,607 tornado.access[58422] INFO: 200 GET /api/ping (::1) 0.56ms

so，i can not access to s3 by the config.（我不能在luigi里面使用这些代码和配置连接到s3）

I know that is a question of luigi configuration，because I can be connected by a command aws（我知道这是我配置的问题，因为在命令行里面，使用aws是可以连接到s3的）

So, I want to ask, how in luigi configuration to connect to the s3？？（所以，我想问一下，在luigi里面配置哪些可以连接到fake-s3）

Thank you very much.（非常感谢，祝你天天开心）

因为我通过aws命令是可以连接的

API Gateway get-api-keys return 404

Hello Guys,
I am trying to create an api gateway endpoint using localstack. I am using aws-cli and I have successfully executed commands - create-rest-api, get-resources and put-method.

When I try to execute the command: get-api-keys, the server returns:

aws apigateway get-api-keys --endpoint http://localhost:4567 --profile my_aws_profile

An error occurred (404) when calling the GetApiKeys operation: <!DOCTYPE HTML PUBLIC"-//W3C//DTD HTML 3.2 Final//EN"><title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>

My question is if all aws-cli commands for api gataway service are supported in localstack?

Lazy load dependencies to reduce installation time

Currently we are fetching all required dependencies as part of setup.py, which results in long installation times and potentially unnecessary downloads when users run pip install localstack.

We should change the download mechanism to fetch libs only when they are first used/accessed and do not yet exist on the system.

Issue with notifications on S3

Hello,

I'm trying to configure S3 to SQS notifications. As per AWS configuration, this is as simple as:

aws s3api put-bucket-notification-configuration --bucket my-bucket --notification-configuration file://notification.json

So I've created a bucket:

aws --endpoint-url=http://localhost:4572 s3 mb s3://my-bucket

And an SQS queue:

aws --endpoint-url=http://localhost:4576 sqs create-queue --queue-name my-bucket-create

And a config.json file:

{
    "QueueConfigurations": [
        {
            "QueueArn": "http://localhost:4576/123456789012/test",
            "Events": [
                "s3:ObjectCreated:*"
            ]
        }
    ]
}

And nothing fails, but when I create an object there are no messages on SQS.

Thanks!

Add randomized error injection (read/write throttles) for Kinesis and DynamoDB mocks

Based on #12 which added binary support (yes/no) for injecting Kinesis errors, we should add a configuration feature to inject indeterministic/random errors.
The same feature should also be applied for simulating DynamoDB read/write throttles.
In both cases, we can introduce ERROR_PROBABILITY config variables in the range [0,1] which define the probability for the framework to inject throughput exceeded exceptions.

Can't use java aws library to put object (etag header incorrect)

Trying to run the aws getting started sample (S3Sample.java)

putObject in the S3 client is failing with an NPE on the etag. Debugging the code the problem is your server returns the ETag header with lower case T, 'Etag', instead of what the aws client library is expecting, 'ETag'

Dockerfile for base image

I've been looking around and can't find a Dockerfile for the base image (fancylinq/alpine-oraclejdk8-mvn). It would be nice to know what this is image getting built off of.

Add ECS support?

Hello atlassian developers!

It'd be great to include AWS ECS (EC2 Container Service) support. It would be nice to have a reliable implementation of these AWS's services to use on our unit tests, especially considering how high an SDK cost ECS incurs to be functional.

Thanks for the great work!

┆Issue is synchronized with this Jira Bug by Unito

Could not retrieve uploaded file from S3

Steps to reproduce the issue: upload image to s3 -> try to access it from the browser

The error I get by the Docker container:

ERROR:localstack.mock.generic_proxy:Error forwarding request: Traceback (most recent call last): File "/opt/code/localstack/localstack/mock/generic_proxy.py", line 104, in forward self.wfile.write(response.text) File "/usr/lib/python2.7/socket.py", line 320, in write data = str(data) # XXX Should really reject non-string non-buffers UnicodeEncodeError: 'ascii' codec can't encode character u'\x89' in position 0: ordinal not in range(128)

Running Docker latest image getting a looping error

Any help would be appreciated.
using docker command
docker run -it --name AWSLite -p 4567-4578:4567-4578 -p 8080:8080 atlassianlabs/localstack

Running on MacOS 10.12.4

Docker Version:
Version 17.03.1-ce-mac5 (16048)

getting this error over and over
AssertionError
make: [Makefile:49: infra] Error 1
2017-04-11 15:18:00,939 INFO exited: infra (exit status 2; expected)
2017-04-11 15:18:01,946 INFO spawned: 'infra' with pid 1016
. .venv/bin/activate; exec localstack/mock/infra.py
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
2017-04-11 15:18:03,395 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
File "localstack/mock/infra.py", line 247, in check_infra_elasticsearch
out = es.cat.aliases()
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(args, params=params, kwargs)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
'aliases', name), params=params)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/opt/code/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 84, in perform_request
raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ConnectionError(('Connection aborted.', BadStatusLine("''",)))

Load dashboard on docker image

Add the dashboard to the docker image. Or add instructions on how to get it going.

'Called process error' on MAC OSX 10.11.15 and Ubuntu 16.04

Trying to install localstack on MAC and Ubuntu 16.04 and in both cases getting the below error.

I tried installing using "sudo pip install localstack" (python 2.7.x) and also tried a different version of localstack and still getting the same error. Tried to clone the repo and then run setup.py but still seeing the same issue. Any thoughts on how to resolve this ?

Traceback (most recent call last):
File "setup.py", line 105, in
"Topic :: Software Development :: Testing",
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 73, in run
self.do_egg_install()
File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 88, in do_egg_install
self.run_command('bdist_egg')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 185, in run
cmd = self.call_command('install_lib', warn_dir=0)
File "/usr/lib/python2.7/dist-packages/setuptools/command/bdist_egg.py", line 171, in call_command
self.run_command(cmdname)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 58, in run
do_make_install()
File "setup.py", line 41, in do_make_install
raise e
subprocess.CalledProcessError: Command 'make install' returned non-zero exit status 2

Docker image tags

Hi!

Do you have any plans regarding docker image tags? We are using localstack since there was no official docker image. As soon as it was added we switched to it. But the latest update broke our build. SQS stopped working. It would be nice to have image tags so it is possible to test new version first.

Thank you for this project. It is really helpful.

Bump KCL to amazon_kclpy version 1.4.0

https://github.com/awslabs/amazon-kinesis-client-python/releases

disable github issues

the readme looks like you guys don't want issues or anything opened up here, and just want to treat this repo as a mirror. You probably want to disable github issues to avoid confusion.

Add Dockerfile and push public docker image

We should build a Docker image, to allow people to spin up the infrastructure without having to mess with their system and install all the requirements.

Add callback to notify when KCL has started up

The amazon_kclpy library spawns a Java process that takes in the order of 10-40 seconds to fully initialize (find Kinesis stream shards, manage leases in DynamoDB table, etc).

Currently we rely on time.sleep to try and guess when this procedure is finished.

We need to add a callback method to reliably determine at which point the KCL process has started up successfully.

Improve documentation

Would be nice to have a wiki and some examples of how to use this

Nodejs Lambda functions?

Is there any plan to support nodejs files as lambda functions?
I can't get it to work with a .js handler :(

Thanks,
Andrei

Web Panel with Docker?

Hello,
I'm having some issues with figuring out how to get a web panel with the docker installation. I tried to also just clone the repo and run make web after make install-web, and am having issues with Anaconda.

$ make web
(. .venv/bin/activate; bin/localstack web --port=8081)
/bin/sh: .venv/bin/activate: No such file or directory
make: *** [web] Error 1

I get the same thing after trying to point anaconda to use the environment with source activate ENV.

Allow Lambda scripts to use file

Currently we are exec()ing the Lambda code to get the handler function. This prevents Lambdas from reflecting on their script location via __file__. This needs to be fixed.

Master tests fail as of commit ca986970abd65a4b752c8ae835173c4b5f336002

 ~/w/localstack   master  git pull origin master                                                                                                                                         Mon Apr 10 02:39:21 2017
From https://github.com/atlassian/localstack
 * branch            master     -> FETCH_HEAD
Already up-to-date.
 ~/w/localstack   master  git log -n 1                                                                                                                                           341ms  Mon Apr 10 02:39:26 2017
commit ca986970abd65a4b752c8ae835173c4b5f336002
Author: Waldemar Hummer <[email protected]>
Date:   Sun Apr 9 14:13:37 2017 +0000

    Fix issue with Elasticsearch 5.x network bind

Make install produced quite a bit of warnings

[WARNING] aws-java-sdk-dynamodb-1.11.0.jar, localstack-utils-1.0-SNAPSHOT.jar define 559 overlappping classes:
[WARNING] localstack-utils-1.0-SNAPSHOT.jar, aws-java-sdk-core-1.11.86.jar define 669 overlappping classes:
[WARNING] localstack-utils-1.0-SNAPSHOT.jar, joda-time-2.8.1.jar define 246 overlappping classes:

Make test fails on elasticsearch

~/w/localstack   master  make test                                                                                                                                              38.5s  Mon Apr 10 02:40:19 2017
make lint && \
		. .venv/bin/activate; DEBUG= PYTHONPATH=`pwd` nosetests --with-coverage --logging-level=WARNING --nocapture --no-skip --exe --cover-erase --cover-tests --cover-inclusive --cover-package=localstack --with-xunit --exclude='.venv.*' .
(. .venv/bin/activate; pep8 --max-line-length=120 --ignore=E128 --exclude=node_modules,legacy,.venv,dist .)
Starting local Elasticsearch (port 4571)...
Starting mock ES service (port 4578)...
Starting mock S3 server (port 4572)...
Starting mock SNS server (port 4575)...
Starting mock SQS server (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams (port 4570)...
Starting mock Firehose (port 4573)...
Starting mock Lambda (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift server (port 4577)...
Elasticsearch health check failed: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine("''",))) Traceback (most recent call last):
  File "/Users/mpandit/work/localstack/localstack/mock/infra.py", line 247, in check_infra_elasticsearch
    out = es.cat.aliases()
  File "/Users/mpandit/work/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/mpandit/work/localstack/.venv/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 23, in aliases
    'aliases', name), params=params)
  File "/Users/mpandit/work/localstack/.venv/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/Users/mpandit/work/localstack/.venv/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 123, in perform_request
    raise ConnectionError('N/A', str(e), e)
ConnectionError: ConnectionError(('Connection aborted.', BadStatusLine("''",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine("''",)))

ERROR checking state of local environment (after some retries): Traceback (most recent call last):
  File "/Users/mpandit/work/localstack/localstack/mock/infra.py", line 271, in check_infra
    check_infra_elasticsearch(expect_shutdown=expect_shutdown, print_error=print_error)
  File "/Users/mpandit/work/localstack/localstack/mock/infra.py", line 254, in check_infra_elasticsearch
    assert isinstance(out, basestring)
AssertionError

Shutdown
FStarting mock Kinesis (port 4568)...
Ready.
Shutdown

Failing build - Command "python setup.py egg_info" failed with error code 1

It was failing with pip install at the make install command.

    File "/private/tmp/pip-build-6jtlmY/localstack/setup.py", line 61, in run
      do_make_install()
    File "/private/tmp/pip-build-6jtlmY/localstack/setup.py", line 41, in do_make_install
      raise e
  subprocess.CalledProcessError: Command 'make install' returned non-zero exit status 2

$ make install

make setup-venv && \
		make install-libs && \
		make compile
(test `which virtualenv` || pip install virtualenv || sudo pip install virtualenv)
(test -e .venv || virtualenv .venv)
(. .venv/bin/activate && pip install --upgrade pip)
Requirement already up-to-date: pip in ./.venv/lib/python3.6/site-packages
(test ! -e requirements.txt || (. .venv/bin/activate && pip install -r requirements.txt))
Collecting airspeed==0.5.5.dev20160812 (from -r requirements.txt (line 1))
  Using cached airspeed-0.5.5.dev20160812.tar.gz
Collecting amazon_kclpy==1.4.1 (from -r requirements.txt (line 2))
  Using cached amazon_kclpy-1.4.1.tar.gz
Collecting awscli==1.11.75 (from -r requirements.txt (line 3))
  Using cached awscli-1.11.75-py2.py3-none-any.whl
Collecting boto3==1.4.0 (from -r requirements.txt (line 4))
  Using cached boto3-1.4.0-py2.py3-none-any.whl
Collecting coverage==4.0.3 (from -r requirements.txt (line 5))
  Using cached coverage-4.0.3.tar.gz
Collecting docopt==0.6.2 (from -r requirements.txt (line 6))
  Using cached docopt-0.6.2.tar.gz
Collecting elasticsearch==5.3.0 (from -r requirements.txt (line 7))
  Using cached elasticsearch-5.3.0-py2.py3-none-any.whl
Collecting flask==0.10.1 (from -r requirements.txt (line 8))
  Using cached Flask-0.10.1.tar.gz
Collecting flask_swagger==0.2.12 (from -r requirements.txt (line 9))
  Using cached flask-swagger-0.2.12.tar.gz
Collecting jsonpath-rw==1.4.0 (from -r requirements.txt (line 10))
  Using cached jsonpath-rw-1.4.0.tar.gz
Collecting moto-ext==0.4.32.1 (from -r requirements.txt (line 11))
  Using cached moto-ext-0.4.32.1.tar.gz
Collecting nose==1.3.7 (from -r requirements.txt (line 12))
  Using cached nose-1.3.7-py3-none-any.whl
Collecting pep8==1.7.0 (from -r requirements.txt (line 13))
  Using cached pep8-1.7.0-py2.py3-none-any.whl
Collecting python-coveralls==2.7.0 (from -r requirements.txt (line 14))
  Using cached python_coveralls-2.7.0-py2.py3-none-any.whl
Collecting requests==2.11.0 (from -r requirements.txt (line 15))
  Using cached requests-2.11.0-py2.py3-none-any.whl
Collecting requests-aws4auth==0.9 (from -r requirements.txt (line 16))
  Using cached requests_aws4auth-0.9-py2.py3-none-any.whl
Collecting restricted_pkg_ext==1.1.3 (from -r requirements.txt (line 17))
  Using cached restricted_pkg_ext-1.1.3.tar.gz
Collecting sh==1.11 (from -r requirements.txt (line 18))
  Using cached sh-1.11.tar.gz
Collecting subprocess32==3.2.7 (from -r requirements.txt (line 19))
  Using cached subprocess32-3.2.7.tar.gz
    Complete output from command python setup.py egg_info:
    This backport is for Python 2.x only.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/cm/y39fj6251mq4zjt0b_v70czm0000gn/T/pip-build-644m2tsd/subprocess32/
make[1]: *** [setup-venv] Error 1
make: *** [install] Error 2


$ python setup.py egg_info                                                                  
running egg_info
creating localstack.egg-info
writing requirements to localstack.egg-info/requires.txt
writing localstack.egg-info/PKG-INFO
writing top-level names to localstack.egg-info/top_level.txt
writing dependency_links to localstack.egg-info/dependency_links.txt
writing manifest file 'localstack.egg-info/SOURCES.txt'
reading manifest file 'localstack.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'localstack/mock/pom.xml'
warning: no files found matching '*.java' under directory 'localstack/mock/src'
no previously-included directories found matching 'localstack/dashboard/web/node_modules'
writing manifest file 'localstack.egg-info/SOURCES.txt'

Machine info:
macOS 10.12.4

$ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
$ python --version
Python 2.7.10

$ pip --version
pip 9.0.1 from /Library/Python/2.7/site-packages (python 2.7)

$ java -version
Picked up _JAVA_OPTIONS:
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b16, mixed mode)

$ javac -version
Picked up _JAVA_OPTIONS:
javac 1.8.0_112

$ mvn --version
Picked up _JAVA_OPTIONS:
Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-04T01:09:06+05:30)
Maven home: /usr/local/Cellar/maven/3.5.0/libexec
Java version: 1.8.0_112, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.4", arch: "x86_64", family: "mac"

$ node --version
v6.9.1

$ npm --version
3.10.8

Add ELB/ALB support

Hi atlassian developers

I'd like to suggest you guys to include AWS Elastic (and Application) Load Balancing support. It would be nice to have a reliable implementation of these AWS's services to use on our unit tests.

┆Issue is synchronized with this Jira Bug by Unito

make install-web command fails

make install-web command to install the Web Dashboard fails with the following error:

localstack > make install-web
(cd localstack/dashboard/web && (test ! -e package.json || npm install))
npm WARN deprecated [email protected]: IMPORTANT hotfix
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: several bugs fixed in v3.2.1
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Jade has been renamed to pug, please install the latest version of pug instead of jade
npm WARN deprecated [email protected]: graceful-fs v3.0.0 and before will fail on node releases >= v7.0. Please update to graceful-fs@^4.0.0 as soon as possible. Use 'npm ls graceful-fs' to find it in the tree.

> [email protected] preinstall /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/.staging/v8-debug-6f45980d
> node -e 'process.exit(0)'


> [email protected] preinstall /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/.staging/v8-profiler-c472ac90
> node -e 'process.exit(0)'

npm WARN prefer global [email protected] should be installed with -g
npm WARN prefer global [email protected] should be installed with -g
npm WARN prefer global [email protected] should be installed with -g
npm WARN prefer global [email protected] should be installed with -g

> [email protected] install /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/fsevents
> node install

[fsevents] Success: "/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/fsevents/lib/binding/Release/node-v51-darwin-x64/fse.node" is installed via remote

> [email protected] install /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug
> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(404): https://node-inspector.s3.amazonaws.com/debug/v0.7.7/node-v51-darwin-x64.tar.gz 
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v51 ABI) (falling back to source compile with node-gyp) 
  CXX(target) Release/obj.target/debug/src/debug.o
../src/debug.cc:38:27: error: too few arguments to function call, expected at least 2, have 1
        v8::Debug::Call(fn);
        ~~~~~~~~~~~~~~~   ^
/Users/arungupta/.node-gyp/7.3.0/include/node/v8-debug.h:198:3: note: 'Call' declared here
  static MaybeLocal<Value> Call(Local<Context> context,
  ^
../src/debug.cc:46:61: error: too few arguments to function call, single argument 'isolate' was not specified
        Isolate* debug_isolate = v8::Debug::GetDebugContext()->GetIsolate();
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/Users/arungupta/.node-gyp/7.3.0/include/node/v8-debug.h:248:3: note: 'GetDebugContext' declared here
  static Local<Context> GetDebugContext(Isolate* isolate);
  ^
../src/debug.cc:61:67: error: too few arguments to function call, single argument 'isolate' was not specified
        Local<Context> debug_context = v8::Debug::GetDebugContext();
                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/Users/arungupta/.node-gyp/7.3.0/include/node/v8-debug.h:248:3: note: 'GetDebugContext' declared here
  static Local<Context> GetDebugContext(Isolate* isolate);
  ^
../src/debug.cc:66:54: error: too few arguments to function call, single argument 'isolate' was not specified
          debug_context = v8::Debug::GetDebugContext();
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/Users/arungupta/.node-gyp/7.3.0/include/node/v8-debug.h:248:3: note: 'GetDebugContext' declared here
  static Local<Context> GetDebugContext(Isolate* isolate);
  ^
4 errors generated.
make[1]: *** [Release/obj.target/debug/src/debug.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Darwin 16.4.0
gyp ERR! command "/usr/local/Cellar/node/7.3.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64/debug.node" "--module_name=debug" "--module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64"
gyp ERR! cwd /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug
gyp ERR! node -v v7.3.0
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp build --fallback-to-build --module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64/debug.node --module_name=debug --module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:106:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:885:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
node-pre-gyp ERR! System Darwin 16.4.0
node-pre-gyp ERR! command "/usr/local/Cellar/node/7.3.0/bin/node" "/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug
node-pre-gyp ERR! node -v v7.3.0
node-pre-gyp ERR! node-pre-gyp -v v0.6.34
node-pre-gyp ERR! not ok 
Failed to execute 'node-gyp build --fallback-to-build --module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64/debug.node --module_name=debug --module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-debug/build/debug/v0.7.7/node-v51-darwin-x64' (1)

> [email protected] install /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler
> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(404): https://node-inspector.s3.amazonaws.com/profiler/v5.6.5/node-v51-darwin-x64.tar.gz 
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v51 ABI) (falling back to source compile with node-gyp) 
  CXX(target) Release/obj.target/profiler/src/profiler.o
  CXX(target) Release/obj.target/profiler/src/cpu_profiler.o
  CXX(target) Release/obj.target/profiler/src/cpu_profile.o
  CXX(target) Release/obj.target/profiler/src/cpu_profile_node.o
  CXX(target) Release/obj.target/profiler/src/heap_profiler.o
../src/heap_profiler.cc:35:18: warning: 'TryCatch' is deprecated [-Wdeprecated-declarations]
        TryCatch try_catch;
                 ^
/Users/arungupta/.node-gyp/7.3.0/include/node/v8.h:7015:40: note: 'TryCatch' has been explicitly marked deprecated here
  V8_DEPRECATED("Use isolate version", TryCatch());
                                       ^
1 warning generated.
  CXX(target) Release/obj.target/profiler/src/heap_snapshot.o
../src/heap_snapshot.cc:46:46: error: no member named 'GetHiddenValue' in 'v8::Object'
      info.GetReturnValue().Set(info.This()->GetHiddenValue(__root));
                                ~~~~~~~~~~~  ^
../src/heap_snapshot.cc:50:20: error: no member named 'SetHiddenValue' in 'v8::Object'
      info.This()->SetHiddenValue(__root, _root);
      ~~~~~~~~~~~  ^
2 errors generated.
make[1]: *** [Release/obj.target/profiler/src/heap_snapshot.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:106:13)
gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
gyp ERR! System Darwin 16.4.0
gyp ERR! command "/usr/local/Cellar/node/7.3.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64/profiler.node" "--module_name=profiler" "--module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64"
gyp ERR! cwd /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler
gyp ERR! node -v v7.3.0
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp build --fallback-to-build --module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64/profiler.node --module_name=profiler --module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:106:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:191:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:885:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
node-pre-gyp ERR! System Darwin 16.4.0
node-pre-gyp ERR! command "/usr/local/Cellar/node/7.3.0/bin/node" "/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler
node-pre-gyp ERR! node -v v7.3.0
node-pre-gyp ERR! node-pre-gyp -v v0.6.34
node-pre-gyp ERR! not ok 
Failed to execute 'node-gyp build --fallback-to-build --module=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64/profiler.node --module_name=profiler --module_path=/Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/v8-profiler/build/profiler/v5.6.5/node-v51-darwin-x64' (1)

> [email protected] postinstall /Users/arungupta/workspaces/localstack/localstack/dashboard/web/node_modules/sauce-connect-launcher
> node scripts/install.js

How to import external library

How can i import a external library into Lambda?

In AWS i just add the library in my package, but using the localstack i'm getting this error (eg. i'm trying to import avro)

ERROR:lambda_mock:Exception on /2015-03-31/functions [POST] Traceback (most recent call last): File "/opt/code/localstack/.venv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/opt/code/localstack/.venv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/opt/code/localstack/.venv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/opt/code/localstack/.venv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/opt/code/localstack/.venv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/opt/code/localstack/localstack/mock/apis/lambda_api.py", line 255, in create_function set_function_code(code, lambda_name) File "/opt/code/localstack/localstack/mock/apis/lambda_api.py", line 217, in set_function_code lambda_handler = exec_lambda_code(zip_file_content, lambda_cwd=lambda_cwd) File "/opt/code/localstack/localstack/mock/apis/lambda_api.py", line 157, in exec_lambda_code raise e ImportError: No module named avro

By build bash is
zip -r lambda.zip avro elasticsearch aws_kinesis_agg urllib3 google

Thanks,

localstack / localstack Goto Github PK

localstack's Introduction

Overview

Installation

Brew (macOS or Linux with Homebrew)

Binary download (MacOS, Linux, Windows)

PyPI (MacOS, Linux, Windows)

Quickstart

Running

Usage

Releases

Contributing

Get in touch

Contributors

Backers

Sponsors

License

localstack's People

Contributors

Stargazers

Watchers

Forkers

localstack's Issues

Error log:

Recommend Projects

Recommend Topics

Recommend Org