Giter VIP home page Giter VIP logo

localstack-java-utils's Introduction

CI Maven Central

⚠️ Note: This repo is not currently very actively maintained. Please consider using the Testcontainers LocalStack Java module as a potential alternative.

LocalStack Java Utils

Java utilities and JUnit integration for LocalStack.

Prerequisites

  • Java
  • Maven
  • Docker
  • LocalStack

Usage

In order to use LocalStack with Java, this project provides a simple JUnit runner and a JUnit 5 extension. Take a look at the example JUnit tests in src/test/java.

By default, the JUnit Test Runner starts LocalStack in a Docker container, for the duration of the test. The container can be configured by using the @LocalstackDockerProperties annotation.

...
import cloud.localstack.LocalstackTestRunner;
import cloud.localstack.ServiceName;
import cloud.localstack.TestUtils;
import cloud.localstack.docker.annotation.LocalstackDockerProperties;

@RunWith(LocalstackTestRunner.class)
@LocalstackDockerProperties(services = { ServiceName.S3, "sqs", "kinesis" })
public class MyCloudAppTest {

  @Test
  public void testLocalS3API() {
    AmazonS3 s3 = TestUtils.getClientS3()
    List<Bucket> buckets = s3.listBuckets();
    ...
  }

}

Or with JUnit 5:

@ExtendWith(LocalstackDockerExtension.class)
@LocalstackDockerProperties(...)
public class MyCloudAppTest {
   ...
}

Installation

The LocalStack JUnit test runner is published as an artifact in Maven Central. Simply add the following dependency to your pom.xml file:

<dependency>
    <groupId>cloud.localstack</groupId>
    <artifactId>localstack-utils</artifactId>
    <version>0.2.23</version>
</dependency>

Configuration

You can configure the Docker behaviour using the @LocalstackDockerProperties annotation with the following parameters:

property usage type default value
pullNewImage Determines if a new image is pulled from the docker repo before the tests are run. boolean false
services Determines which services should be run when the localstack starts. String[] All
imageName Use a specific image name (organisation/repo) for docker container String localstack/localstack
imageTag Use a specific image tag for docker container String latest
portEdge Port number for the edge service, the main entry point for all API invocations String 4566
portElasticSearch Port number for the elasticsearch service String 4571
hostNameResolver Used for determining the host name of the machine running the docker containers so that the containers can be addressed. IHostNameResolver localhost
environmentVariableProvider Used for injecting environment variables into the container. IEnvironmentVariableProvider Empty Map
bindMountProvider Used bind mounting files and directories into the container, useful to run init scripts before using the container. IBindMountProvider Empty Map
initializationToken Give a regex that will be searched in the logstream of the container, start is complete only when the token is found. Use with bindMountProvider to execute init scripts. String Empty String
useSingleDockerContainer Whether a singleton container should be used by all test classes. boolean false

For more details, please refer to the README of the main LocalStack repo: https://github.com/localstack/localstack

NOTE: These utilities assume docker is installed in one of the default locations (C:\program files\docker\docker\resources\bin\docker.exe, C:\program files\docker\docker\resources\docker.exe, usr/local/bin/docker or usr/bin/docker). If your docker executable is in a different location, then use the DOCKER_LOCATION environment variable to specify it.

Deprecated Configurations

Due to recent changes in LocalStack (exposing all services via a single edge port, 4566), the following configuration parameters are now deprecated in the latest version:

property usage type default value
randomizePorts Determines if the container should expose the default local stack ports (4567-4583) or if it should expose randomized ports. boolean false

Note: When specifying the port in the services property, you cannot use randomizePorts = true

Building

To build the latest version of the code via Maven:

make build

Releasing

To publish a release of the library, the "Maven Release" Github Action can be manually triggered in the repository, which will take the latest code on master branch and publish it to Maven Central.

Change Log

  • v0.2.23: Fix S3 endpoints to be compatible with LocalStack v2
  • v0.2.22: Fix sqs event mapping for new event format, some test fixes
  • v0.2.21: Bump version of AWS SDK v1; add AWS SDK v2 sync clients to TestUtils; add docker executable path under homebrew
  • v0.2.20: Fix extracting container logs for LocalStack startup check
  • v0.2.19: Bump version of log4j to 2.17.0 to fix further vulnerabilities related to recent CVE
  • v0.2.18: Fix for isRunning method after stopping the container; filter synthetic bridge methods during method detection for java Lambda full handler syntax; pass ADDITIONAL_MVN_ARGS to mvn javadoc command; bump version of log4j to 2.15.0 to fix CVE-2021-44228
  • v0.2.17: Fix issue with using :: to specify lambda handler which implements the RequestHandler interface, revert removal of EC2HostNameResolver annotation
  • v0.2.16: Add support for :: notation for Java Lambda handler specification, fix failing QLDB tests, fix failing tests with Jexter rules/extensions
  • v0.2.15: Fix Kinesis CBOR tests; fix project setup and classpath for SDK v1/v2 utils; fix awaiting results in tests using async clients; refactor classpath setup for v1/v2 SDKs; fall back to using edge port if port mapping cannot be determined from container
  • v0.2.14: Add ability to get handler class name through _HANDLER environment variable like on real AWS and Lambci environment
  • v0.2.11: Enable specification of "platform" when configuring container
  • v0.2.10: Add Lambda async utils for AWS SDK v2; add support for specifying bind mounts and init scripts via @LocalstackDockerProperties; add PowerMock integration for easy patching of AWS SDK to use local endpoints; add support for configuring the Docker image name via @LocalstackDockerProperties; add tests for templated emails
  • v0.2.8: Allow overwriting the port binding via environment variables
  • v0.2.7: Extend @LocalstackDockerProperties to include port binding
  • v0.2.6: Add new path to possible docker exe locations in Windows; add various additional tests for v1 and v2 SDKs (Kinesis, SQS, SSM & SecretsManager, ...)
  • v0.2.5: Refactor code to accommodate edge port config for all services; add CloudWatch Logs endpoint configuration
  • v0.2.2: Addition of CloudWatch Logs endpoint configuration; adjust tests to use central edge service endpoint
  • v0.2.1: Move Java sources into separate project; mark non-Docker Java LocalstackExtension as deprecated; update paths for Python code lookup in Docker container
  • v0.2.0: Last version still maintained in LocalStack main repo

PowerMock

You can use the PowerMock Library to call the builders default method and still get LocalStack version of the AWS clients.

...
@RunWith(PowerMockRunner.class)
@PowerMockRunnerDelegate(LocalstackTestRunner.class)
@LocalstackDockerProperties(services = { "ses" })
@PrepareForTest({ AmazonSimpleEmailServiceClientBuilder.class, AmazonSimpleEmailServiceAsyncClientBuilder.class })
@PowerMockIgnore({"javax.crypto.*", "org.hamcrest.*", "javax.net.ssl.*", "com.sun.org.apache.xerces.*", "javax.xml.*", "org.xml.*", "javax.management.*", "javax.security.*", "org.w3c.*"})
public class SESMessagingTest {
....
    @Before
    public void mockSES() {
        AmazonSimpleEmailService mockSes = TestUtils.getClientSES();
        PowerMockito.mockStatic(AmazonSimpleEmailServiceClientBuilder.class);
        when(AmazonSimpleEmailServiceClientBuilder.defaultClient()).thenReturn(mockSes);
    }
    @Test
    public void testSendEmail() throws Exception {
        AmazonSimpleEmailService client = amazonSimpleEmailServiceClientBuilder.defaultClient();
    ....

PowerMockLocalStack Utility

This utility makes easier to use PowerMock with Localstack.

...
public class PowerMockLocalStackExampleTest extends PowerMockLocalStack{
    private static final String TOPIC = "topic";
    @Before
    public void mock() {
        PowerMockLocalStack.mockSNS();
    }

    @Test
    public void testSendMessage() throws JMSException {
        final AmazonSNS clientSNS = AmazonSNSClientBuilder.defaultClient();
        ...
    }
}

Acknowledgements

I thank you Keith Humphreys, for showing us how to empower LocalStack with PowerMock to write tests even easier.

License

This code is released under the Apache License, Version 2.0 (see LICENSE.txt).

localstack-java-utils's People

Contributors

alexrashed avatar antonoellerer avatar bentsku avatar cbromberg avatar ceejaey avatar denisbr avatar dependabot[bot] avatar dfangl avatar duongpv7 avatar jasonab avatar jellybeat avatar jorgenota avatar madh93 avatar mans2singh avatar nofranks avatar pinzon avatar prakashdhanasekaran avatar raissaa avatar robfrank avatar scubasau avatar simonrw avatar upanshu21 avatar usmanovbf avatar vitorfec avatar whummer avatar wojciechszymski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

localstack-java-utils's Issues

UnknownHostException: localhost.localstack.cloud on mac os

I am using 0.2.15 version of the utils. The docker starts up without any issue, but the s3 client can not resolve the url.. here is the config I have:

@ExtendWith(LocalstackDockerExtension::class)
@LocalstackDockerProperties(services = [ServiceName.S3])
@SpringBootTest(classes = [Application::class, SomeControllerTest.S3TestConfiguration::class])
@TestPropertySource(
    properties = ["spring.main.allow-bean-definition-overriding=true"]
)
class SomeControllerTest : AbstractTest() {

    @Configuration
    class S3TestConfiguration {

        // override existing s3 client
        @Primary
        @Bean
        fun s3Client(): S3Client =
            S3Client.builder()
                .credentialsProvider(
                    StaticCredentialsProvider.create(
                        AwsBasicCredentials.create(
                            "key", "secret"
                        )))
                .endpointOverride(URI.create(Localstack.INSTANCE.endpointS3))
                .region(Region.AWS_GLOBAL)
                .serviceConfiguration { it.pathStyleAccessEnabled() }
                .build()
    }
   ...
}
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: localhost.localstack.cloud
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
...
Caused by: java.net.UnknownHostException: localhost.localstack.cloud
	at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:800)

Unknown port mapping for service: sqs

Version: 0.2.7
OS: MacOS Catalina

Cloned a repo with LocalStack related SQS tests, but seeing issues with the below test:

@ExtendWith(LocalstackDockerExtension.class)
@LocalstackDockerProperties(services = {"sqs"})
@EnabledOnOs({OS.MAC})
public class SqsStreamClientTest {

    @Test
    public void it_should_get_queue_url() throws ExecutionException, InterruptedException {
        //Arrange
        SqsAsyncClient sqsAsyncClient = TestUtils.getClientSQSAsyncV2();
        String queueName = "test-queue";
        sqsAsyncClient.createQueue(CreateQueueRequest.builder().queueName(queueName).build()).get();
        SqsStreamClient target = new SqsStreamClient(sqsAsyncClient);

        //Act
        String queueUrl = target.getQueueUrl(queueName, sqsAsyncClient);

        //Assert
        assertEquals("http://localhost:4566/000000000000/test-queue", queueUrl);
    }
}

Error:

Unknown port mapping for service: sqs
java.lang.IllegalArgumentException: Unknown port mapping for service: sqs
	at cloud.localstack.Localstack.getServicePort(Localstack.java:223)
	at cloud.localstack.Localstack.endpointForService(Localstack.java:214)
	at cloud.localstack.Localstack.getEndpointSQS(Localstack.java:166)
	at cloud.localstack.awssdkv2.TestUtils.getClientSQSAsyncV2(TestUtils.java:38)
	at com.trp.dit.lux.core.infrastructure.SqsStreamClientTest.it_should_get_queue_url(SqsStreamClientTest.java:24)
       ....

Debugging into

private void loadServiceToPortMap() {
        String localStackPortConfig = localStackContainer.executeCommand(Arrays.asList("cat", PORT_CONFIG_FILENAME));

        int edgePort = getEdgePort();
        Map<String, Integer> ports = new RegexStream(DEFAULT_PORT_PATTERN.matcher(localStackPortConfig)).stream()
                .collect(Collectors.toMap(match -> match.group(1), match -> edgePort));

        serviceToPortMap = Collections.unmodifiableMap(ports);
    }

localStackPortConfig = 'cat: can't open '/opt/code/localstack/.venv/lib/python3.8/site-packages/localstack_client/config.py': No such file or directory'

When I run localStackContainer.executeCommand(Arrays.asList("ls", "/opt/code/localstack/.venv/lib")), I see "python3.7" returned - how should I fix this mismatching version?

It seems related to localstack/localstack#2082.

Version 0.2.1 - Unknown port mapping for service: dynamodb

Really strange behaviour, what is working on a colleagues machine is not working on mine which i thought was not meant to be the case when running docker images.

Maven Dependency

 <dependency>
            <groupId>cloud.localstack</groupId>
            <artifactId>localstack-utils</artifactId>
            <version>0.2.1</version>
            <scope>test</scope>
        </dependency>

Getting the error

Apr 29, 2020 2:07:18 PM cloud.localstack.docker.annotation.LocalstackDockerAnnotationProcessor getExternalHostName
INFO: External host name is set to: localhost
Apr 29, 2020 2:07:21 PM cloud.localstack.docker.Container createLocalstackContainer
INFO: Started container: c356b2e82a2acbeefb4cb1f1032d814886b6474639476bc400dffb75124e02f8
Apr 29, 2020 2:07:21 PM cloud.localstack.Localstack startup
INFO: Waiting for LocalStack container to be ready...

java.lang.IllegalArgumentException: Unknown port mapping for service: dynamodb

	at cloud.localstack.Localstack.endpointForService(Localstack.java:204)
	at cloud.localstack.Localstack.getEndpointDynamoDB(Localstack.java:131)
	at cloud.localstack.TestUtils.getEndpointConfigurationDynamoDB(TestUtils.java:187)
	at cloud.localstack.TestUtils.getClientDynamoDB(TestUtils.java:143)
	at com.lmig.global.eventframeworktrackerinfra.DyamodbCRUDTest.setUp(DyamodbCRUDTest.java:36)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at cloud.localstack.LocalstackTestRunner.run(LocalstackTestRunner.java:42)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

Apr 29, 2020 2:07:50 PM cloud.localstack.docker.Container stop
INFO: Stopped container: c356b2e82a2acbeefb4cb1f1032d814886b6474639476bc400dffb75124e02f8

Process finished with exit code 255

Here is my class if it helps

import cloud.localstack.LocalstackTestRunner;
import cloud.localstack.TestUtils;
import cloud.localstack.docker.annotation.LocalstackDockerProperties;
import com.amazonaws.services.dynamodbv2.document.DynamoDB;
import com.amazonaws.services.dynamodbv2.document.Item;
import com.amazonaws.services.dynamodbv2.document.PutItemOutcome;
import com.amazonaws.services.dynamodbv2.document.Table;
import com.amazonaws.services.dynamodbv2.model.*;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.ActiveProfiles;
import software.amazon.ion.Timestamp;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeUnit;

/**
 * This class is not unit testing any Java classes, But it test out if INSERT
 * operation of the following fields possible while executing
 */
@RunWith(LocalstackTestRunner.class)
@LocalstackDockerProperties(services = {"dynamodb"})
@ActiveProfiles("test")
public class DyamodbCRUDTest {

	public static final String ERROR_TRACKER = "error-tracker";
	DynamoDB dynamoDB;

	@Before
	public void setUp() throws InterruptedException {
		dynamoDB = new DynamoDB(TestUtils.getClientDynamoDB());
		List<KeySchemaElement> keySchema = new ArrayList<KeySchemaElement>();
		keySchema.add(new KeySchemaElement().withAttributeName("messageId").withKeyType(KeyType.HASH));

		List<AttributeDefinition> attributeDefinitions = new ArrayList<AttributeDefinition>();
		attributeDefinitions.add(new AttributeDefinition().withAttributeName("messageId").withAttributeType("S"));

		CreateTableRequest request = new CreateTableRequest()
				.withTableName(ERROR_TRACKER)
				.withKeySchema(keySchema)
				.withAttributeDefinitions(attributeDefinitions)
				.withProvisionedThroughput(new ProvisionedThroughput()
						.withReadCapacityUnits(5L)
						.withWriteCapacityUnits(6L));

		Table table = dynamoDB.createTable(request);
		table.waitForActive();
		TimeUnit.SECONDS.sleep(5);
	}

	/**
	 * This test is to test the dynamo db INSERT operation with the db created through the setUp
	 */
	@Test
	public void testInsertIntoTable() {

		Table table = dynamoDB.getTable(ERROR_TRACKER);

		// Build the item
		Item item = new Item()
				.withPrimaryKey("messageId", "f46gsfkv789")
				.withString("Subscriber", "FivaTestSubscriber")
				.withString("Timestamp", Timestamp.now().toString())
				.withString("Status", "Failure")
				.withString("Description", String.valueOf(new IOException("Error accessing file").getStackTrace()));

		// Write the item to the table
		PutItemOutcome outcome = table.putItem(item);
	}
}

Always pulls new image regardless of setting on properties when running on Windows

Using the annotation @LocalstackDockerProperties we should be able to set "pullNewImage" to false - indeed it should be false by default.

However, I have found running this on Windows always attempts to pull a new image, regardless of the setting.

I believe this is due to the check for whether the image exists within cloud.localstack.docker.Container line 71 checks if the list of images contains the image name but does not account for Windows line endings within the ListImagesCommand.execute() method as the list is split on \n only, leaving a \r at the end of the name which is not on the expected fullImageName.

I only noticed this was the expected behaviour when I was trying to work around this issue and noticed that the option didn't work!

Steps to reproduce:

  • Create a JUnit class annotated with:
@LocalstackDockerProperties(
		imageTag = "0.12.11",
		pullNewImage = false,
		services = { "s3" }
)
  • Run the JUnit

Expected:
The log message below is not produced:

cloud.localstack.docker.Container createLocalstackContainer
INFO: Pulling latest image...

Actual:
It is.

Request for jackson cbor version bump to latest version

Localstack-java-utils is currently using older version of jackson-dataformat-cbor

Please update it to use the latest version which is:
<groupId>com.fasterxml.jackson.dataformat</groupId> <artifactId>jackson-dataformat-cbor</artifactId> <version>2.15.2</version> <scope>test</scope> </dependency>

This will remove the Vulnerability that jackson library brings

when running multiple junit test classes, getting error "A docker instance is starting or already started"

To test our opensearch cluster functionality we have many Junit 5 tests using localstack-utils annotations as below.

@ExtendWith(LocalstackDockerExtension.class)
@LocalstackDockerProperties(portEdge = "3000", portElasticSearch = "3100", services = { ServiceName.S3, ServiceName.ELASTICSEARCH_SERVICE })
@ExtendWith(SpringExtension.class)
public class MyOpenSearchTest {
...
}

Everything works fine when running the junit tests one by one.
However, since we have many test classes and it takes ~1 minute for the docker container to startup, we tried to speed up overall test duration by doing the following:

  1. use a different portEdge and portElasticSearch in @LocalstackDockerProperties for each junit test class
  2. we enabled parralel junit tets execution by adding the lines below to file /resources/junit-platform.properties
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.config.dynamic.factor = 1

With the above chganges, the junit tests are started in parallel but we get this error:

java.lang.IllegalStateException: A docker instance is starting or already started.

	at cloud.localstack.Localstack.startup(Localstack.java:67)
	at cloud.localstack.docker.LocalstackDockerExtension$StartedLocalStack.<init>(LocalstackDockerExtension.java:57)
	at cloud.localstack.docker.LocalstackDockerExtension.beforeAll(LocalstackDockerExtension.java:42)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$invokeBeforeAllCallbacks$7(ClassTestDescriptor.java:358)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeBeforeAllCallbacks(ClassTestDescriptor.java:358)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.before(ClassTestDescriptor.java:197)
	at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.before(ClassTestDescriptor.java:74)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:102)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:71)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:169)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1311)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1840)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1806)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

we are using this dependency:

<dependency>
  <groupId>cloud.localstack</groupId>
  <artifactId>localstack-utils</artifactId>
  <version>0.2.22</version>
  <scope>test</scope>
</dependency>

How can we run these Junit tests in parallel without getting this error?

Junit extension can't get port mapping for services with latest docker image

The latest docker image doesn't work with the JUnit extension. This is because it can't get the port mapping to build the AWS SDK clients.

Exception:

java.lang.IllegalArgumentException: Unknown port mapping for service: s3

	at cloud.localstack.Localstack.endpointForService(Localstack.java:199)
	at cloud.localstack.Localstack.getEndpointS3(Localstack.java:108)
	at cloud.localstack.TestUtils.getEndpointConfigurationS3(TestUtils.java:186)
	at cloud.localstack.TestUtils.getClientS3(TestUtils.java:122)
	at com.atlassian.migration.datacenter.core.fs.S3UploaderIT.uploadShouldUploadPathsFromQueueToS3(S3UploaderIT.java:61)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:628)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:117)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:184)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:180)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:127)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

I believe this is ocurring because the port mapping is obtained by running a script and the script has moved because the latest docker image uses Python 3.8 instead of 3.7.

String localStackPortConfig = this.localStackContainer.executeCommand(Arrays.asList("cat", "/opt/code/localstack/.venv/lib/python3.7/site-packages/localstack_client/config.py"));

Latest image:

bash-5.0# ls /opt/code/localstack/.venv/lib/python3.7/site-packages/localstack_client/config.py
ls: /opt/code/localstack/.venv/lib/python3.7/site-packages/localstack_client/config.py: No such file or directory

0.10.6

bash-5.0# ls /opt/code/localstack/.venv/lib/python3.7/site-packages/localstack_client/config.py
/opt/code/localstack/.venv/lib/python3.7/site-packages/localstack_client/config.py

Upgrade to com.amazonaws:aws-lambda-java-events:3.7.0 breaks compile

I recently tried to run a Java Lambda that consumes a DynamodbEvent on localstack. My lambda was never called because of

2021-03-25T17:46:47:INFO:localstack.services.awslambda.lambda_api: Error executing Lambda function 
arn:aws:lambda:us-east-1:000000000000:function:localstack-issue: Lambda process returned error status code: 1. Result: . Output:
Exception in thread "main" java.lang.NoSuchMethodError: 'void com.amazonaws.services.lambda.runtime.events.DynamodbEvent$DynamodbStreamRecord.setEventName(com.amazonaws.services.dynamodbv2.model.OperationType)'
     at cloud.localstack.lambda.DDBEventParser.parse(DDBEventParser.java:24)
     at cloud.localstack.LambdaExecutor.main(LambdaExecutor.java:84)

This was probably because my lambda was compiling against com.amazonaws:aws-lambda-java-events:3.7.0 and this project is running with version 2.2.7
This could also be the cause of localstack/localstack#1960
Is there way to make localstack work with lambdas that use aws-lambda-java-events:3.7.0 ?

Failure to detect localstack start due to multiple errors in the logs

Hey,
It seems as if in one of the last localstack nightly versions a change was introduced which causes the container to throw a large amount of error messages like

2021-11-10T12:16:46:DEBUG:stevedore.extension: found extension EntryPoint(name='sagemaker:pro', value='localstack_ext.services.providers:sagemaker', group='localstack.aws.provider')
2021-11-10T12:16:46:ERROR:plugin.manager: error importing entrypoint EntryPoint(name='sagemaker:pro', value='localstack_ext.services.providers:sagemaker', group='localstack.aws.provider'): No module named 'localstack_ext.services.providers'

This in turn causes localstack-java-utils to fail to detect the container startup, since the Ready. token is further up in the logs than the last 100 fetched lines.

When starting the image up by hand, those errors are not thrown.
The only properties I used were useSingleDockerContainer = true, I am using podman instead of docker though.
Do you know where the problem could be?
Related, is there some image tag such as stable? It seems strange that I always need to fetch the latest version if I don't want to specify a certain image.

localstack.log

Allow execution of JUnit tests without access to Docker (in Docker)

The current JUnit annotation requires a Docker executable to be available for pulling & running the localstack/localstack image.

This tends to work fine on a developer's local machine and possibly in some CI pipelines, however, due to historic insecurities with Docker-in-Docker (DinD), some CI tooling does not permit the running of DinD within pipelines. This prevents the use of LocalStack JUnits within CI pipelines, while other technologies, e.g. Python are fine to use within the same pipelines because they only rely on localstack being present (which can be run as a CI pipeline service) rather than docker itself.

Could a mechanism for skipping the docker pull/run/shutdown for Java tests be provided (e.g. a different annotation or setting in the existing annotation that skips the docker commands and simply expected localstack to be setup and running already)?

enable "Cloudwatch Logs" service=logs

What a cool -project and it is working well for me. Nice work!

I do not see service logs in public class ServiceName {} "Cloudwatch Logs" is supported in localstack on the default port 4586. Is this something that could be easily enabled?

A way to specify default LocalstackDockerProperties configuration

Sorry in case this is a trivial question but I'm looking for a way to specify default configuration for LocalstackDockerProperties annotation, which I'm reusing in multiple test classes. Is there any way to do that, instead of copy-pasting same parameters in every test class?

Kinesis sending Serialized object.

I'm trying to write object as bytes to Kinesis streams, but I'm getting errors. I wrote a class using Java Generic and sending objects as Array of Bytes.

java.io.StreamCorruptedException: invalid stream header: at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:1059)

Method to write to get bytes to ByteBuffer.wrap method.

public byte[] objectToBuffer(T record) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream out = null;

    out = new ObjectOutputStream(bos);
    out.writeObject(record);
    out.flush();
      return bos.toByteArray();
}

To convert ByteBuffer back to object. It's erroring here:

public T bufferToObject(ByteBuffer recordByteBuffer) throws IOException, ClassNotFoundException {

  byte[] bytes = new byte[recordByteBuffer.limit()];
  recordByteBuffer.get(bytes);

  ObjectInputStream objectInputStream = new ObjectInputStream(
          new ByteArrayInputStream(bytes));
  return (T) objectInputStream.readObject();
}

What's really strange is that code works in Junit using Localstack? But it doesn't work on the real AWS kinesis.

Localstack: localStackContainer.waitForLogToken(READY_TOKEN) does not work with LocalStack image version 0.13.0

localstack-utils: 0.2.17
localstack (docker): 0.13.0
os: MacOS Monterey
platform: M1 max (arm)

The statement localStackContainer.waitForLogToken(READY_TOKEN) never completes, because localstack 0.13.0 outputs a lot of log lines and the 'Ready.' token isn't found in the last 100 lines selected in the method

    private boolean logContainsPattern(Pattern pattern) {
        String logs = getContainerLogs();
        return pattern.matcher(logs).find();
    }

...in the cloud.localstack.docker.Container class

Docker location missing in the list

Currently, the library looking for a docker executable in few places, but the latest docker installation has it in docker\resources\docker.exe
C:\Program Files\Docker\Docker\resources\docker.exe
but it looking for a file in the 'bin' folder
C:\Program Files\Docker\Docker\resources*bin*\docker.exe

List of possible locations have to be updated

cloud.localstack.docker.exception.LocalstackDockerException: Could not start the localstack docker container. at cloud.localstack.Localstack.startup(Localstack.java:90) at cloud.localstack.LocalstackTestRunner.run(LocalstackTestRunner.java:41) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53) Caused by: java.lang.IllegalStateException: Cannot find docker executable. at cloud.localstack.docker.DockerExe.lambda$getDockerExeLocation$1(DockerExe.java:43) at java.base/java.util.Optional.orElseThrow(Optional.java:408) at cloud.localstack.docker.DockerExe.getDockerExeLocation(DockerExe.java:43) at cloud.localstack.docker.DockerExe.<init>(DockerExe.java:35) at cloud.localstack.docker.command.Command.<init>(Command.java:11)

Invalid SNS service ports

I was trying to create SNS topic for one of my tests, but the creation failed with error com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond.
The log is showing that it is trying to post to an endpoint with port 4575 (URL: http://localhost:4575). However the latest documentation says - starting with releases after v0.11.5, all services are now exposed via the edge service (port 4566) only! .

17:01:55.594 [main] DEBUG com.amazonaws.request - Sending Request: POST http://localhost:4575 / Parameters: ({"Action":["CreateTopic"],"Version":["2010-03-31"],"Name":["test-topic"]}Headers: (amz-sdk-invocation-id: b686b987-f3e1-edeb-efc7-68175b5805d3, User-Agent: aws-sdk-java/1.11.792 Mac_OS_X/10.15.7 OpenJDK_64-Bit_Server_VM/15.0.1+9-18 java/15.0.1 vendor/Oracle_Corporation, )

ERROR: java.util.LinkedHashMap cannot be cast to APIGatewayProxyRequestEvent

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Unfortunately, the issue exists in the current docker build

localstack.services.awslambda.lambda_executors.InvocationException: Lambda process returned error status code: 1. Result: . Output:
Exception in thread "main" java.lang.ClassCastException: class java.util.LinkedHashMap cannot be cast to class com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent (java.util.LinkedHashMap is in module java.base of loader 'bootstrap'; com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent is in unnamed module of loader 'app')
        at com.fitness.society.users.handlers.UserRegistrationAPIGatewayHandler.handleRequest(UserRegistrationAPIGatewayHandler.java:26)
        at cloud.localstack.LambdaExecutor.main(LambdaExecutor.java:119)

Expected Behavior

Should work with the below dependency as it was closed and verified in the original ticket localstack/localstack#1634

   <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-lambda-java-events</artifactId>
        <version>2.0.2</version>
    </dependency>

How are you starting LocalStack?

With a docker run command

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

docker run localstack/localstack

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

curl -X POST http://192.168.1.1:4566/restapis/mvy1yew0wa/prod/_user_request_/users

Environment

- OS: Amazon Linux2
- LocalStack: localstack/localstack   latest    087de49bf85a

Anything else?

No response

Allow for configuration of image name

Hi,

first of all kudos to this great project - I am currently in the process of migrating a project that used DynamoDbLocal and ElasticSearch spawned as child processes from Java for testing (basically because our on-prem CI does not allow docker containers) and docker containers for development, which caused a lot of classpath polution and also custom code that somebody (in this case me) to maintain. So I am excited to switch the CI to cloud (Bitbucket pipelines) and in the process switch to localstack spawned from Java as well.
As we are using ElasticSearch I ran into the problem of deferred download of ElasticSearch through the localstack container. Since we use throw-away containers we do not benefit from the in-container caching.
Long story short: If I create a PR to allow for the configuration of the image name to be able to use localstack-full, would you be willing to review that or is that not something you would like in this project.
Of course I would not break existing code
Cheers, Chris

awssdkv1 is missing in 0.2.15 release

From my understanding, it was introduced in 7e8c822. Not sure what was the idea, but I guess maven compiler plugin excludes couldn't exclude awssdkv1 and awssdkv2 simultaneously 🙂

Error when run make test

When I am attempt to upgrade the project to java 11 (localstack/localstack#2093), I encountered this issue when I run "make test".

[ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 35.166 s <<< FAILURE! - in cloud.localstack.S3FeaturesTest
[ERROR] testPresignedURLUpload(cloud.localstack.S3FeaturesTest) Time elapsed: 0.075 s <<< ERROR!
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <test.localhost.atlassian.io> doesn't match any of the subject alternative names: [localhost.localstack.cloud]
at cloud.localstack.S3FeaturesTest.testPresignedURLUpload(S3FeaturesTest.java:187)

The workaround for it is to either turn off the SSL by setting USE_SSEL=0

test: ## Run tests for Java/JUnit compatibility
USE_SSL=0 SERVICES=serverless,kinesis,sns,sqs,cloudwatch mvn $(MVN_ARGS) test

or change the replace the localhost name localhost.localstack.cloud in LocalStack.java.

UPDATE:
Change the localhost name does not help when it comes to travis ci job. Setting USE_SSL=0 does the trick.

Removal of the EC2HostNameResolver

7e8c822 removes the EC2HostNameResolver annotation which we are currently using in our tests, so attempting to upgrade to the latest version of cloud-localstack fails for us.
I understand that the docker in docker might not be the preferable solution for everyone, but besides that, was there a good reason to remove this functionality?

Is there a proposed new method of doing the same in the latest release?

Docker executable not found in Windows 10

When trying to execute a JUnit 5 test using the LocalstackDockerExtension, I'm getting an IllegalStateException with the message "Cannot find docker executable."

I've been debugging the issue and found that DockerExe class has a constant (POSSIBLE_EXE_LOCATIONS) with the possible locations to find the executable. It contain 3 possible paths:

  • "C:/program files/docker/docker/resources/bin/docker.exe"
  • "/usr/local/bin/docker"
  • "/usr/bin/docker"

However, neither of them matches the path of my docker executable. I've recently installed docker and the executable is in:

  • "C:\ProgramData\DockerDesktop\version-bin\docker.exe"

... this is not the real location, but a simbolinc link... the actual location is:

  • "C:\Program Files\Docker\Docker\resources\docker.exe"

Additional information:
The change of location is probably a recent change in Docker. I've even found a recent bug in docker 2.3.4.0 related to this change: docker/for-win#7898

Possible solution/improvement:
I guess the constant should be updated with the new path ("C:\Program Files\Docker\Docker\resources\docker.exe").

Why are the TestUtils sdk v2 sync clients deprecated?

The TestUtils class in the awssdkv2 package only has async clients; however, the TestUtils in the deprecated package has sync clients. Is there some historic context/reason why the sync ones were deprecated or weren't ported over to the awssdkv2 package as well? Maybe a compatibility issue or something? I couldn't infer the context from the commit messages.

Localstack docker beans do not work with PER_CLASS JUnit Lifecycle

Under a PER_CLASS JUnit lifecycle, attempting to wire up Spring test beans (such as a DynamoDB client) returns this error:
java.lang.RuntimeException: Container not started. I believe this is because the Spring application context is loaded before the extension's beforeAll method (https://github.com/localstack/localstack-java-utils/blob/master/src/main/java/cloud/localstack/docker/LocalstackDockerExtension.java#L33) initializes the Localstack containers within a PER_CLASS lifecycle.

Also noticed this issue which seems to mention the same problem: localstack/localstack#1067, but I'm not sure the PR mentioned in that issue was ever created and merged. I was able to fix this issue by creating a custom version of the LocalstackDockerExtension that implements TestInstancePostProcessor (which is loaded before the Spring application context under a PER_CLASS lifecycle) using the same code from the existing beforeAll method.

I can create a PR to add this fix, but I'm wondering if there's any better ways to get this working or if there's any issues with that approach?

When not using elasticsearch, the port is still binded during unit test

In my test, I only use S3 service, with this annotation:

@ExtendWith(LocalstackDockerExtension.class) @LocalstackDockerProperties( portEdge = "4577", services = {"s3"}, hostNameResolver = HostResolver.class)

I configured the edge port because I have multiple tests running on the same runner at the same time, but they fail because the elasticsearch port is already binded by another test suite.

[error] docker: Error response from daemon: driver failed programming external connectivity on endpoint hopeful_fermi (a98226257cb706ef22f296a7cb04d3b0ce148d1f013f251f57dd38a9c2cfe668): Bind for 0.0.0.0:4571 failed: port is already allocated.

If I am not using elastic search, I think the port should not be in use.

Allow specification of platform for the docker run command via LocalstackDockerProperties annotation

I ran into a problem when writing some unit tests that utilised test managed localstack instances on an M1 loaded mac. When the docker run command is issued at Container#createLocalstackContainer, the expected return is a container id. However, when running on apple silicon, the following message is received:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
ee89c26b70622fca6afe1227af6e51ff1ed9f4b5877ee4181ceabf4968d42303

Consequently, the "ready" token is never found in the container log and the test run eventually times out. I didn't see an existing way to do this so I forked the repo. It seems like this can be overcome by allowing the specification of the platform flag in the @LocalstackDockerProperties annotation. I'm happy to contribute to add this flexibility, though it'd be my first contribution so I'd have to understand what the process is.

Thanks.

Using localstack-java-utils together with Testcontainers throws UnsatisfiedLinkError

Problem occurred while developing on Mac with M1 chip.

Maven pom.xml dependency sample to reproduce the problem:

        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>cloud.localstack</groupId>
            <artifactId>localstack-utils</artifactId>
            <version>0.2.21</version>
            <scope>test</scope>
        </dependency>

During tests, following error pops up:

java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/8h/c61frhdx4h755wk7dbgny_w80000gn/T/jna--1389878725/jna11007918705638209843.tmp

After investigating dependencies tree, I've discovered that JNA dependency of this library overrides JNA version of Testcontainers:

[INFO] +- org.testcontainers:testcontainers:jar:1.17.6:test
[INFO] |  +- junit:junit:jar:4.13.2:test
[INFO] |  |  \- org.hamcrest:hamcrest-core:jar:2.2:test
[INFO] |  +- org.apache.commons:commons-compress:jar:1.22:test
[INFO] |  +- org.rnorth.duct-tape:duct-tape:jar:1.0.8:test
[INFO] |  |  \- org.jetbrains:annotations:jar:17.0.0:compile
[INFO] |  +- com.github.docker-java:docker-java-api:jar:3.2.13:test
[INFO] |  \- com.github.docker-java:docker-java-transport-zerodep:jar:3.2.13:test
[INFO] |     \- com.github.docker-java:docker-java-transport:jar:3.2.13:test
[INFO] +- org.testcontainers:junit-jupiter:jar:1.17.6:test
[INFO] |  \- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test
[INFO] |     +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] |     +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
[INFO] |     \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] \- cloud.localstack:localstack-utils:jar:0.2.21:test
[INFO]    +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO]    +- net.java.dev.jna:jna:jar:4.1.0:test

Solution that helped:

        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>cloud.localstack</groupId>
            <artifactId>localstack-utils</artifactId>
            <version>0.2.21</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>net.java.dev.jna</groupId>
                    <artifactId>jna</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

with resulting dependency tree:

[INFO] +- org.testcontainers:testcontainers:jar:1.17.6:test
[INFO] |  +- junit:junit:jar:4.13.2:test
[INFO] |  |  \- org.hamcrest:hamcrest-core:jar:2.2:test
[INFO] |  +- org.apache.commons:commons-compress:jar:1.22:test
[INFO] |  +- org.rnorth.duct-tape:duct-tape:jar:1.0.8:test
[INFO] |  |  \- org.jetbrains:annotations:jar:17.0.0:compile
[INFO] |  +- com.github.docker-java:docker-java-api:jar:3.2.13:test
[INFO] |  \- com.github.docker-java:docker-java-transport-zerodep:jar:3.2.13:test
[INFO] |     +- com.github.docker-java:docker-java-transport:jar:3.2.13:test
[INFO] |     \- net.java.dev.jna:jna:jar:5.8.0:test
[INFO] +- org.testcontainers:junit-jupiter:jar:1.17.6:test
[INFO] |  \- org.junit.jupiter:junit-jupiter-api:jar:5.8.2:test
[INFO] |     +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO] |     +- org.junit.platform:junit-platform-commons:jar:1.8.2:test
[INFO] |     \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] \- cloud.localstack:localstack-utils:jar:0.2.21:test

Question: is it possible to update JNA to version to 5.8.0?

Connection refused on 4597 when using ec2 service

Hi,

I'm hitting the following exception when using localstack-java-utils (0.2.1) and Junit5:

software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Connect to localhost:4597 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)

at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.handleThrownException(RetryableStage.java:137)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:95)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:63)
at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:43)

I see this error for a test I have when following the pattern described in the readme i.e.

@ExtendWith({LocalstackDockerExtension.class, MockitoExtension.class})
@LocalstackDockerProperties(services = {"ec2"}, imageTag = "0.11.2")
public class MyEc2Test {
   ...
}

If I apply a breakpoint during test execution and check docker ps I can see the relevant containers are up, although the tests still fail

CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS              PORTS                                                                 NAMES
35192b6c57c6        quay.io/testcontainers/ryuk:0.2.3   "/app"                   16 seconds ago      Up 16 seconds       0.0.0.0:32900->8080/tcp                                               testcontainers-ryuk-83b761b3-b318-4cae-bbea-d823438e0a22
d2903c7cc267        localstack/localstack:0.11.2        "docker-entrypoint.sh"   32 seconds ago      Up 32 seconds       4566/tcp, 4585-4597/tcp, 8080/tcp, 0.0.0.0:4567-4584->4567-4584/tcp   thirsty_swartz

Interestingly, If I run the localstack manually as below, the tests pass (having removed the annotations @ExtendWith and @LocalstackDockerProperties) and are able to query localhost on port 4597

 docker run -p 4597:4597 -e SERVICES=ec2 -e DEFAULT_REGION=us-east-1 localstack/localstack:0.11.2

It feels like port 4597 isnt being properly mapped/exposed when ec2 is run via @LocalstackDockerProperties?

When waiting for a container to get ready, 10 lines might be too few

Hey,
When testing w/ localstack-java-utils, I noticed that the 10 lines NUM_LOG_LINES might be too few to spot the Ready. token if the container needs some time to start up, as the logging of the unsuccessful POST / requests might already use up more than this.
Are there any reservations against increasing this number to 100?
Best regards,
Anton Oellerer

Is isRunning method works fine?

I try to make a simple e2e test (Java + Spring Framework) which check our API by stopping Localstack instance, sending message to broker instance and finally asserting HTTP error response code. This test is part of bigger test suite with DirtiesContext annotation (with after each method mode).

Out Localstack bean is customized. In Spring Configuration we defined a bean with custom init and destroy methods. Init method will be posted below, destroy method just send purge requests into all queues. We don't want to stop Localstack instance - time optimization.

Init method:

        if (!localstack.isRunning()) {
            localstack.startup(LOCALSTACK_CONFIGURATION);
            Runtime.getRuntime().addShutdownHook(new Thread(localstack::stop));
        }

After localstack.stop(); - our init method will never work because isRunning method returns always true even when docker doesn't have running containers (docker ps return empty list).

If Localstack object (unfortunately a static object) has non-null instance of localStackContainer - isRunning method return true response (with empty list of available ports underneath). Seems like stop method do not unset localStackContainer field?

Container.isRunning method:

try {
            new PortCommand(containerId).execute();
            return true;
        } catch(Exception e) {
            return false;
        }

Could you allow to unset localStackContainer field or just unset this instance inner stop method? We just want to find out (using isRunning method) that docker image is running or not to avoid unnecessary Localstack restart between single test (using DirtiesContext annotation).

Allow to mount files to the container started through @LocalstackDockerProperties

Hi,

I have been able to migrate a project of decent size from running ElasticSearch locally and DynamoDbLocal to localstack and it works great 🥇 - we were able to drop implementation classes that we used to replace S3 with files locally 💯

I have the setup code for the resources created in a dev-profile bean in the app and I would like to remove that because I have to add extra dependencies to the classpath to make that code work. localstack has the right mechanism for that - the init scripts. So I would like to create a PR for allowing the Java code set the mount parameter for the init scripts as well, that way we can remove the initialization Java code. I imagine that that is not a huge amount of code that needs to be written in localstack-java-utils as it only exposes existing functionality of the localstack container.
Would you welcome that feature / PR as well?
Cheers, Chris

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.