Giter VIP home page Giter VIP logo

spring-cloud-aws's Introduction

spring-cloud-aws has moved! This repository is no longer actively maintained by VMware, Inc.

Spring Cloud for Amazon Web Services

Spring Cloud for Amazon Web Services, part of the Spring Cloud umbrella project, eases the integration with hosted Amazon Web Services. It offers a convenient way to interact with AWS provided services using well-known Spring idioms and APIs, such as the messaging or caching API. Developers can build their application around the hosted services without having to care about infrastructure or maintenance.

Note: Further Spring Cloud AWS versions development moved to awspring/spring-cloud-aws. The only active branch in this repository is 2.2.x.

Checking out and building

To check out the project and build it from source, do the following:

git clone https://github.com/spring-cloud/spring-cloud-aws.git
cd spring-cloud-aws
mvn package

If you encounter out of memory errors during the build, increase available heap and permgen for Maven:

MAVEN_OPTS='-XX:MaxPermSize=258m -Xmx1024m'

To build and install jars into your local Maven cache:

mvn install

Building documentation

Documentation can be built by activating the `docs`profile in the maven build. If there is an ruby error like

LoadError: no such file to load -- asciidoctor

then the user must install the asciidoctor gem and set the environment variable `GEM_HOME`to the ruby gem folder. For example:

  # Get gem info
  gem environment

  export GEM_HOME=<PATH FOR GEM ENVIRONMENT>

Using IntelliJ IDEA

Spring Cloud AWS development is done with IntelliJ IDEA. In order to create all IntelliJ IDEA project files, you have to import the file within idea as a maven project.

Note: Please make sure to revert all changes in the .idea config file directory, as the maven plugin overwrites the configuration files kept in the scm.

Running integration tests

Spring Cloud AWS contains a test-suite which runs integration tests to ensure compatibility with the Amazon Web Services. In order to run the integration tests, the build process has to create different resources on the Amazon Webservice platform (Amazon EC2 instances, Amazon RDS instances, Amazon S3 Buckets, Amazon SQS Queues). Creating these resources takes time and costs money, because every instance creation is charged with a one hour usage. Therefore Spring Cloud AWS does not execute the integration tests by default.

In order to run integration tests you must prepare three properties named accessKey,secretKey and rdsPassword. These two properties accessKey and secretKey are account/user specific and should never be shared to anyone. To retrieve these settings you have to open your account inside the AWS console and retrieve them through the Security Credentials Page. Note: In general we recommend that you use an Amazon IAM user instead of the account itself. The last password rdsPassword is used to access the database inside the integration tests. This password has a minimum length of 8 characters.

Also you must prepare the sender and recipient mail addresses to test the Amazon Simple E-Mail Service. These two addresses must be verified for the Amazon SES Service.

To build with the integration tests you must execute

mvn verify -Daws-integration-tests.access-key=<access key>  -Daws-integration-tests.secret-key=<secret key> -DrdsPassword=<rds password> -DsenderAddress=<sender address> -DrecipientAddress=<recipient address>

The integration test will create an Amazon Web Services CloudFormation stack and execute the tests. The stack is destroyed after executing the tests (either successful or failed) to ensure that there are no unnecessary costs.

Costs of integration tests

The costs for one integration test run should not be more than 0.40 $ per hour (excl. VAT).

Developing using Amazon Web Services

During development it might be time-consuming to run the integration tests regularly. In order to create a stack only once, and reuse them for the tests run, you have to create the stack manually using the template found in /spring-cloud-aws-integration-test/src/test/resources. You will need to create the stack with the name "IntegrationTestStack" to ensure that the integration tests will re-use the stack.

Getting in touch

Spring Cloud Team on Twitter

Individual team members can be found on different social media channels

Contributing

Spring Cloud is released under the non-restrictive Apache 2.0 license, and follows a very standard Github development process, using Github tracker for issues and merging pull requests into master. If you want to contribute even something trivial please do not hesitate, but follow the guidelines below.

Sign the Contributor License Agreement

Before we accept a non-trivial patch or pull request we will need you to sign the Contributor License Agreement. Signing the contributor’s agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests.

Code of Conduct

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

Code Conventions and Housekeeping

None of these is essential for a pull request, but they will all help. They can also be added after the original pull request but before a merge.

  • Use the Spring Framework code format conventions. If you use Eclipse you can import formatter settings using the eclipse-code-formatter.xml file from the Spring Cloud Build project. If using IntelliJ, you can use the Eclipse Code Formatter Plugin to import the same file.

  • Make sure all new .java files to have a simple Javadoc class comment with at least an @author tag identifying you, and preferably at least a paragraph on what the class is for.

  • Add the ASF license header comment to all new .java files (copy from existing files in the project)

  • Add yourself as an @author to the .java files that you modify substantially (more than cosmetic changes).

  • Add some Javadocs and, if you change the namespace, some XSD doc elements.

  • A few unit tests would help a lot as well — someone has to do it.

  • If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project).

  • When writing a commit message please follow these conventions, if you are fixing an existing issue please add Fixes gh-XXXX at the end of the commit message (where XXXX is the issue number).

Checkstyle

Spring Cloud Build comes with a set of checkstyle rules. You can find them in the spring-cloud-build-tools module. The most notable files under the module are:

spring-cloud-build-tools/
└── src
    ├── checkstyle
    │   └── checkstyle-suppressions.xml (3)
    └── main
        └── resources
            ├── checkstyle-header.txt (2)
            └── checkstyle.xml (1)
  1. Default Checkstyle rules

  2. File header setup

  3. Default suppression rules

Checkstyle configuration

Checkstyle rules are disabled by default. To add checkstyle to your project just define the following properties and plugins.

pom.xml
<properties>
<maven-checkstyle-plugin.failsOnError>true</maven-checkstyle-plugin.failsOnError> (1)
        <maven-checkstyle-plugin.failsOnViolation>true
        </maven-checkstyle-plugin.failsOnViolation> (2)
        <maven-checkstyle-plugin.includeTestSourceDirectory>true
        </maven-checkstyle-plugin.includeTestSourceDirectory> (3)
</properties>

<build>
        <plugins>
            <plugin> (4)
                <groupId>io.spring.javaformat</groupId>
                <artifactId>spring-javaformat-maven-plugin</artifactId>
            </plugin>
            <plugin> (5)
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-checkstyle-plugin</artifactId>
            </plugin>
        </plugins>

    <reporting>
        <plugins>
            <plugin> (5)
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-checkstyle-plugin</artifactId>
            </plugin>
        </plugins>
    </reporting>
</build>
  1. Fails the build upon Checkstyle errors

  2. Fails the build upon Checkstyle violations

  3. Checkstyle analyzes also the test sources

  4. Add the Spring Java Format plugin that will reformat your code to pass most of the Checkstyle formatting rules

  5. Add checkstyle plugin to your build and reporting phases

If you need to suppress some rules (e.g. line length needs to be longer), then it’s enough for you to define a file under ${project.root}/src/checkstyle/checkstyle-suppressions.xml with your suppressions. Example:

projectRoot/src/checkstyle/checkstyle-suppresions.xml
<?xml version="1.0"?>
<!DOCTYPE suppressions PUBLIC
		"-//Puppy Crawl//DTD Suppressions 1.1//EN"
		"https://www.puppycrawl.com/dtds/suppressions_1_1.dtd">
<suppressions>
	<suppress files=".*ConfigServerApplication\.java" checks="HideUtilityClassConstructor"/>
	<suppress files=".*ConfigClientWatch\.java" checks="LineLengthCheck"/>
</suppressions>

It’s advisable to copy the ${spring-cloud-build.rootFolder}/.editorconfig and ${spring-cloud-build.rootFolder}/.springformat to your project. That way, some default formatting rules will be applied. You can do so by running this script:

$ curl https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/.editorconfig -o .editorconfig
$ touch .springformat

IDE setup

Intellij IDEA

In order to setup Intellij you should import our coding conventions, inspection profiles and set up the checkstyle plugin. The following files can be found in the Spring Cloud Build project.

spring-cloud-build-tools/
└── src
    ├── checkstyle
    │   └── checkstyle-suppressions.xml (3)
    └── main
        └── resources
            ├── checkstyle-header.txt (2)
            ├── checkstyle.xml (1)
            └── intellij
                ├── Intellij_Project_Defaults.xml (4)
                └── Intellij_Spring_Boot_Java_Conventions.xml (5)
  1. Default Checkstyle rules

  2. File header setup

  3. Default suppression rules

  4. Project defaults for Intellij that apply most of Checkstyle rules

  5. Project style conventions for Intellij that apply most of Checkstyle rules

Code style
Figure 1. Code style

Go to FileSettingsEditorCode style. There click on the icon next to the Scheme section. There, click on the Import Scheme value and pick the Intellij IDEA code style XML option. Import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Spring_Boot_Java_Conventions.xml file.

Code style
Figure 2. Inspection profiles

Go to FileSettingsEditorInspections. There click on the icon next to the Profile section. There, click on the Import Profile and import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Project_Defaults.xml file.

Checkstyle

To have Intellij work with Checkstyle, you have to install the Checkstyle plugin. It’s advisable to also install the Assertions2Assertj to automatically convert the JUnit assertions

Checkstyle

Go to FileSettingsOther settingsCheckstyle. There click on the + icon in the Configuration file section. There, you’ll have to define where the checkstyle rules should be picked from. In the image above, we’ve picked the rules from the cloned Spring Cloud Build repository. However, you can point to the Spring Cloud Build’s GitHub repository (e.g. for the checkstyle.xml : https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/main/resources/checkstyle.xml). We need to provide the following variables:

Important
Remember to set the Scan Scope to All sources since we apply checkstyle rules for production and test sources.

spring-cloud-aws's People

Contributors

aemruli avatar alainsahli avatar andersonkyle avatar aygiz avatar bsamartins avatar cstettler avatar darrenforsythe avatar devinsba avatar eddumelendez avatar kingbbode avatar louisalexander avatar lukasznyrc avatar luramarchanjo avatar maciejwalkowiak avatar marcingrzejszczak avatar matejnedic avatar mkatircioglu avatar olgamaciaszek avatar ryanjbaxter avatar sayembd avatar senthilpanneerselvam avatar sfussenegger avatar spencergibb avatar spring-builds avatar spring-operator avatar tac-yacht avatar tgianos avatar tmnuwan12 avatar trevormarshall avatar voytech avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spring-cloud-aws's Issues

BeanCreationException when using context-resource-loader

I’m encountering the exception below when trying to use the <context-resource-loader /> in my application. I’ve narrowed it down to the following line of code in ResourceLoaderBeanPostProcessor:

beanFactory.registerResolvableDependency(BeanFactory.class, this.applicationContext);

When I comment this line out, the application deploys successfully and I’m able to leverage the SimpleStorageResourceLoader injected via:

@Inject
private ResourceLoader resourceLoader;

I’m not going to pretend to fully understand what is going on but it seems like any BeanPostProcessor that gets applied after the ResourceLoaderBeanPostProcessor fails. I’m working on putting a smaller test case together to demonstrate the issue but wanted to log the ticket in case anyone has encountered it.

Spring Cloud AWS config:

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:aws-context="http://www.springframework.org/schema/cloud/aws/context"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/cloud/aws/context
        http://www.springframework.org/schema/cloud/aws/context/spring-cloud-aws-context-1.0.xsd">

  <aws-context:context-region region="us-west-2" />
  <aws-context:context-credentials />
  <aws-context:context-resource-loader />
</beans>

Relevant Spring dependencies:

    <spring.version>4.1.4.RELEASE</spring.version>
    <spring-cloud-aws-context.version>1.0.0.RC2</spring-cloud-aws-context.version>
    <spring-data-commons.version>1.9.2.RELEASE</spring-data-commons.version>
    <spring-data-jpa.version>1.7.2.RELEASE</spring-data-jpa.version>
    <spring-integration.version>4.1.2.RELEASE</spring-integration.version>
    <spring-mobile-device.version>1.1.3.RELEASE</spring-mobile-device.version>
    <spring-security.version>3.2.5.RELEASE</spring-security.version>

Exception (happens on start up):

Wrapped by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.integration.expression.IntegrationEvaluationContextAwareBeanPostProcessor#0': BeanPostProcessor before instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration': BeanPostProcessor before instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.security.config.annotation.method.configuration.GlobalMethodSecurityConfiguration': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire method: public void org.springframework.security.config.annotation.method.configuration.GlobalMethodSecurityConfiguration.setObjectPostProcessor(org.springframework.security.config.annotation.ObjectPostProcessor); nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'objectPostProcessor' defined in class org.springframework.security.config.annotation.configuration.ObjectPostProcessorConfiguration: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.springframework.beans.factory.config.AutowireCapableBeanFactory]: : No qualifying bean of type [org.springframework.beans.factory.config.AutowireCapableBeanFactory] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {}; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [org.springframework.beans.factory.config.AutowireCapableBeanFactory] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {}
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:472) ~[AbstractAutowireCapableBeanFactory.class:4.1.4.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303) ~[AbstractBeanFactory$1.class:4.1.4.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[DefaultSingletonBeanRegistry.class:4.1.4.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299) ~[AbstractBeanFactory.class:4.1.4.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[AbstractBeanFactory.class:4.1.4.RELEASE]
    at org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:232) ~[PostProcessorRegistrationDelegate.class:4.1.4.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:615) ~[AbstractApplicationContext.class:4.1.4.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:465) ~[AbstractApplicationContext.class:4.1.4.RELEASE]
    at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:403) ~[ContextLoader.class:4.1.4.RELEASE]
    at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:306) ~[ContextLoader.class:4.1.4.RELEASE]
    at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:106) [ContextLoaderListener.class:4.1.4.RELEASE]
    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4795) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5221) [catalina.jar:8.0.14]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:724) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:700) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:714) [catalina.jar:8.0.14]
    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:919) [catalina.jar:8.0.14]
    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:461) [catalina.jar:8.0.14]
    at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1495) [catalina.jar:8.0.14]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_20]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_20]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_20]
    at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_20]
    at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:300) [tomcat-coyote.jar:8.0.14]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) [na:1.8.0_20]
    at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) [na:1.8.0_20]
    at org.apache.catalina.manager.ManagerServlet.check(ManagerServlet.java:1432) [catalina.jar:8.0.14]
    at org.apache.catalina.manager.ManagerServlet.deploy(ManagerServlet.java:711) [catalina.jar:8.0.14]
    at org.apache.catalina.manager.ManagerServlet.doPut(ManagerServlet.java:423) [catalina.jar:8.0.14]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:647) [servlet-api.jar:na]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) [servlet-api.jar:na]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.14]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-websocket.jar:8.0.14]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.14]
    at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) [catalina.jar:8.0.14]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [catalina.jar:8.0.14]
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:618) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [catalina.jar:8.0.14]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:8.0.14]
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) [catalina.jar:8.0.14]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:8.0.14]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537) [catalina.jar:8.0.14]
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081) [tomcat-coyote.jar:8.0.14]
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658) [tomcat-coyote.jar:8.0.14]
    at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) [tomcat-coyote.jar:8.0.14]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566) [tomcat-coyote.jar:8.0.14]
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523) [tomcat-coyote.jar:8.0.14]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_20]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_20]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.14]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_20]

Add Spring Boot auto configuration support for Cloudformation

As a application developer I should be able to enable the CloudFormation support in a Spring Boot Application. After that I should be able to inject all resource in the stack into my application classes

Example
@EnableCloudFormationConfiguration

will allow users to inject automatically all data sources, message / notification templates etc. into their application without explicitly configuring the resources

Revise spring-cloud-aws-context dependencies

I'm not interested in RedisCacheFactory, but I see that spring-data-redis is pulled anyway.
Must be optional. From other side it pull transitively org.springframework:spring-context-support:4.0.9.RELEASE, when we are on the SF 4.1.5.

I'd say the same about any aws-java-sdk-* module, but let it be...

Elastic Beanstalk Environment Params

Is it possible within the spring-cloud-aws project to pull the environment information from an Elastic Beanstalk setup?

At the simplest level I would like to pull a custom tag from the Elastic Beanstalk configuration (not the instance) and use it dynamically to load a spring profile.

To further clarify the use case, our CI (Atlassian Bamboo) deploys the app into 3 environments. Development, staging and production. The goal would be for Spring to determine where it is loaded to take on profile properties from the environment.

User Guide

I want to use this in my project, but without a user guide it is hard to get started.

Add support for custom regions

AWS SDK allows to override region configuration with com/amazonaws/regions/override/regions.xml file, see RegionUtils.class.

But StaticRegionProvider relies on Regions emun, so it is impossible to pass custom region id with spring-boot-aws.

TaskExecutor is not set for ResourceLoader with autoconfiguration

When I tried to upload a file to S3 with resource loader I received an exception

Caused by: java.lang.IllegalArgumentException: TaskExecutor must not be null
    at org.springframework.util.Assert.notNull(Assert.java:112)
    at org.springframework.core.task.support.ExecutorServiceAdapter.<init>(ExecutorServiceAdapter.java:55)
    at org.springframework.cloud.aws.core.io.s3.SimpleStorageResource$SimpleStorageOutputStream.<init>(SimpleStorageResource.java:167)
    at org.springframework.cloud.aws.core.io.s3.SimpleStorageResource.getOutputStream(SimpleStorageResource.java:132)
    at org.springframework.core.io.WritableResource$getOutputStream.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at ru.ptsecurity.cloudmanagement.carunner.Application.run(Application.groovy:89)
    at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:677)
    ... 9 more

Sample code

@Slf4j
@Configuration
@EnableAutoConfiguration(exclude = [
        AmazonRdsDatabaseRegistrar.class,
        MessagingAutoConfiguration.class,
        ElastiCacheAutoConfiguration.class,
        MailSenderAutoConfiguration.class])
@ComponentScan
@EnableAsync
public class Application implements CommandLineRunner {
    public static void main(String[] args) {
        def context = SpringApplication.run(Application.class, args)
    }

    @Autowired
    private ResourceLoader resourceLoader

    @Override
    void run(String... args) throws Exception {
        Resource resource = resourceLoader.getResource('s3://testbucket/test.txt')
        WritableResource writableResource = (WritableResource) resource
        OutputStream outputStream = writableResource.getOutputStream()
        outputStream.write('test'.getBytes())
        outputStream.close()
    }
}

@CompileStatic
@Configuration
@EnableAsync(proxyTargetClass = true)
class SchedulingAndAsyncConfiguration implements AsyncConfigurer {

    @Bean
    @Override
    Executor getAsyncExecutor() {
        def executor = new ThreadPoolTaskExecutor()
        executor.corePoolSize = 4
        executor.maxPoolSize = 32
        executor.queueCapacity = 2048
        executor.waitForTasksToCompleteOnShutdown = true
        executor.awaitTerminationSeconds = 60
        executor.threadNamePrefix = 'asyncThreadPoolTaskExecutor-'

        executor
    }

    @Bean
    @Override
    AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        new SimpleAsyncUncaughtExceptionHandler()
    }
}

It will be nice if ResourceLoaderConfigureAdapter will be added in order to fine tune ResourceLoader, eg specify custom TaskExecutor.

Support type serialization during SNS convertAndSend

When sending messages from SNS, only String payloads are permitted in any of the convertAndSend instances.

NotificationMessagingTemplate.java#L91-L101

/**
 * <b>IMPORTANT</b>: the underlying message channel {@link org.springframework.cloud.aws.messaging.core.TopicMessageChannel} only
 * supports {@code String} as payload. Therefore only {@code String} payloads are accepted.
 *
 * @see org.springframework.messaging.core.MessageSendingOperations#convertAndSend(Object, Object, java.util.Map)
 */
@Override
public <T> void convertAndSend(String destinationName, T payload, Map<String, Object> headers, MessagePostProcessor postProcessor) throws MessagingException {
    Assert.isInstanceOf(String.class, payload, "Payload must be of type string");
    super.convertAndSend(destinationName, payload, headers, postProcessor);
}

After glancing through the code it seems like this constraint is not necessary as when AbstractMessagingSendingTemplate in spring-messaging is invoked it creates a string backed payload:

AbstractMessageSendingTemplate.java#L128-L134

@Override
public void convertAndSend(D destination, Object payload, Map<String, Object> headers,
        MessagePostProcessor postProcessor) throws MessagingException {

    Message<?> message = doConvert(payload, headers, postProcessor);
    send(destination, message);
}

AbstractMessageSendingTemplate.java#L157

Message<?> message = getMessageConverter().toMessage(payload, messageHeaders);

MessageConverter
applies toMessage AbstractMessageSendingTemplate.java#L157
which calls AbstractMessageConverter.java#L201
which calls MappingJackson2MessageConverter.java#L191.

payload = writer.toString();

The other codepath in MappingJackson2MessageConverter allows a byte array to be returned. In order to prevent this, NotificationMessagingTemplate can have the Jackson converter added similar to
QueueMessagingTemplate.java#L86-L94.

if (messageConverter != null) {     
    messageConverters.add(messageConverter);
} else if (JACKSON_2_PRESENT) {
    MappingJackson2MessageConverter mappingJackson2MessageConverter = new MappingJackson2MessageConverter();
    mappingJackson2MessageConverter.setSerializedPayloadClass(String.class);
    messageConverters.add(mappingJackson2MessageConverter);
}
setMessageConverter(new CompositeMessageConverter(messageConverters));

As message conversion ordering is guaranteed within CompositeMessageConverter, as long as jackson is added first, the output is guaranteed to be a String. This would make consuming types from SNS->SQS (or other delivery channels) easier.

Add scheduler support for SQS queue listener

@MessageMapping is aggressively polling messages at a rate of between 16 and 26 times a second on a single EC2 node.

It would be great to be able to specify the scheduled rate (ideally using cron expression) at which messages are pull from the queue.

Add support to SimpleStorageResourceLoader for versioned S3 objects

Versioning in S3 is a means of keeping multiple variants of an object in the same bucket.

http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html

Spring Cloud AWS Context supports loading resources using the convention "s3://bucket/objectName"

Context's SimpleStorageResourceLoader creates SimpleStorageResources by parsing the resource location string and splitting it into a bucket String and an objectName String. The Amazon S3 Client's getObject and getObjectMetadata methods are then passed the bucket and objectName Strings to return the S3 content and metadata respectively.

Amazon S3 is quite permissive in what characters are allowed in object names, but there is some guidance on using certain characters:

http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html

I propose that a convention is introduced that allows retrieving the resource of a particular version of an S3 object in a bucket that has versioning enabled, for example:

"s3://bucket/objectName^versionId"

If a versionId is not supplied for an object within a bucket with versioning enabled, the HEAD/latest version will be returned. If a versionId is not supplied for an object within a bucket without versioning enabled, the object is returned as normal.

Make ResourceLoader lazy

It will be great if S3 ResourceLoader will fetch meta-data, may be optional, only if told to so, eg calling exists(), contentLenth() and etc methods.

AmazonEc2 instance is not found for DI

I have a simple Spring boot app and tring to inject default EC2Client but it seems that it is not created. When I try to inject S3Client evirything is fine.

@Configuration
@EnableAutoConfiguration(exclude = [
        AmazonRdsDatabaseRegistrar.class,
        MessagingAutoConfiguration.class,
        ElastiCacheAutoConfiguration.class,
        MailSenderAutoConfiguration.class])
@ComponentScan
public class Application implements CommandLineRunner {

    @Autowired
    private AmazonS3 amazonS3

    @Autowired
    private AmazonEC2 amazonEc2;

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args)
    }

    @Override
    void run(String... args) throws Exception {
        def list = amazonS3.listBuckets()
        log.info(list)
    }
}

Propagate content type for SNS requests

NotificationMessagingTemplate convertAndSend accepts a set of headers but later does not propagate the full set in TopicMessageChannel.send.

@Override
public <T> void convertAndSend(String destinationName, T payload, Map<String, Object> headers) throws MessagingException {
    Assert.isInstanceOf(String.class, payload, "Payload must be of type string");
    super.convertAndSend(destinationName, payload, headers);
}

TopicMessageChannel.java#L45-L50

@Override
public boolean send(Message<?> message, long timeout) {
    this.amazonSns.publish(new PublishRequest(this.topicArn,
            message.getPayload().toString(), findNotificationSubject(message)));
    return true;
}

For QueueMessagingTemplate to receive json formatted messages sent by the NotificationMessageTemplate they need to have an application/json contentType which will never be propagated as the TopicMessageChannel ignores the contentType header.

Propagating the content-type header from the message into the publish request as a message attribute will allow QueueMessageTemplate to correctly deserialize json encoded types.

QueueMessageTemplate performs a similar process when sending messages:
QueueMessageChannel.java#L55-L80

Map<String, MessageAttributeValue> messageAttributes = getContentTypeMessageAttributes(message);
if (!messageAttributes.isEmpty()) {
    sendMessageRequest.withMessageAttributes(messageAttributes);
}

Also, it might be useful to note in the docs that in order to enable this use case, raw message delivery must be set to True.

Important
To use message attributes with Amazon SQS endpoints, you must set the subscription attribute, Raw Message Delivery, to True. For more information about Raw Message Delivery, see Appendix: Large Payload and Raw Message Delivery


Until content-type is propagated, we are using the following workaround:
In the consumer, add a custom argument resolver:

<aws-messaging:annotation-driven-queue-listener wait-time-out="20">
    <aws-messaging:argument-resolvers>
        <bean class="com.bar.FooResolver"/>
    </aws-messaging:argument-resolvers>
</aws-messaging:annotation-driven-queue-listener>

Argument Resolver:

public class FooResolver implements HandlerMethodArgumentResolver {
    private final MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter() {
        {
            setStrictContentTypeMatch(false);
        }
    };

    @Override
    public boolean supportsParameter(@NonNull MethodParameter parameter) {
        return parameter.getParameterType().isAssignableFrom(MyRequest.class);
    }

    @Override
    public Object resolveArgument(MethodParameter parameter, @NonNull Message<?> message) throws Exception {
        Object result = converter.fromMessage(message, parameter.getParameterType());
        if (result == null) {
            throw new MessageConversionException(message, "Failed to read JSON: " + message.getPayload());
        }

        return result;
    }

ResourceLoaderBeanPostProcessor trying to set applicationContext on ApplicationObjectSupport which has applicationContext already set

Beans extending ApplicationObjectSupport (that implements ApplicationContextAware) are being post processed by ApplicationContextAwareProcessor and ResourceLoaderBeanPostProcessor i.e. both try to set application context. Bean initialisation fails when application context is tried to set second time as ApplicationObjectSupport.setApplicationContext method throws an ApplicationContextException. One example of such a bean type is RequestMappingHandlerMapping.

Maybe ResourceLoaderBeanPostProcessor should only set application context if it has not been already set.

Add support for SQS redrives

Currently, we can't find a way to prevent the message from being deleted after it was received. Generally, when a message fails to process (an exception is thrown from the message handler) the message should be left in the queue to be redriven. SQS Redrive Policy with DLQ

In executeMessage catching throwable prevents the interruption of deleteMessage:(SimpleMessageListenerContainer.java#L125-L131)

protected void executeMessage(org.springframework.messaging.Message<String> stringMessage) {
    try {
        getMessageHandler().handleMessage(stringMessage);
    } catch (Throwable throwable) {
        handleError(throwable);
    }
}

(SimpleMessageListenerContainer.java#L191-L203)

        @Override
        public void run() {
            String receiptHandle = this.message.getReceiptHandle();
            String payload = this.message.getBody();
            MessageBuilder<String> messageBuilder = MessageBuilder.
                    withPayload(payload).
                    setHeader(QueueMessageHandler.Headers.LOGICAL_RESOURCE_ID_MESSAGE_HEADER_KEY, this.logicalQueueName);
            copyAttributesToHeaders(messageBuilder);
            executeMessage(messageBuilder.build());
            getAmazonSqs().deleteMessage(new DeleteMessageRequest(this.queueUrl, receiptHandle));
            getLogger().debug("Deleted message with id {} and receipt handle {}", this.message.getMessageId(), this.message.getReceiptHandle());
        }
    }

Ideally letting an uncaught exception be thrown from the task should prevent the message from being deleted (it will become available again after the visibility timeout has elapsed). Alternatively, throwing an exception inherited from a newly introduced exception (e.g. RedrivableException) that was handled before the throwable would work as well.

AutoDetectingStackNameProvider Fails For Auto-Scaling Groups/ELB

Trying to determine the stack name for an instance fails when the instance is connected to the stack via an auto-scaling group as opposed to directly, resulting in an exception that fails the application loading. This exception should likely be swallowed up, allowing the name detector to move on to its fallback logic.

Caused by: com.amazonaws.AmazonServiceException: Stack for i-9ae92175 does not exist (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: 4ccbd832-ad6f-11e4-8f95-617fd8834716)
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1071)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:719)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:454)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:294)
    at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.invoke(AmazonCloudFormationClient.java:1078)
    at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.describeStackResources(AmazonCloudFormationClient.java:875)
    at org.springframework.cloud.aws.core.env.stack.config.AutoDetectingStackNameProvider.autoDetectStackName(AutoDetectingStackNameProvider.java:63)
    at org.springframework.cloud.aws.core.env.stack.config.AutoDetectingStackNameProvider.afterPropertiesSet(AutoDetectingStackNameProvider.java:53)
    at org.springframework.cloud.aws.core.env.stack.config.AutoDetectingStackNameProvider.<init>(AutoDetectingStackNameProvider.java:44)
    at org.springframework.cloud.aws.core.env.stack.config.AutoDetectingStackNameProvider.<init>(AutoDetectingStackNameProvider.java:48)
    at org.springframework.cloud.aws.core.env.stack.config.StackResourceRegistryFactoryBean.<init>(StackResourceRegistryFactoryBean.java:57)
    at org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration$StackAutoDetectConfiguration.stackResourceRegistryFactoryBean(ContextStackAutoConfiguration.java:63)
    at org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration$StackAutoDetectConfiguration$$EnhancerBySpringCGLIB$$98368e1.CGLIB$stackResourceRegistryFactoryBean$0(<generated>)
    at org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration$StackAutoDetectConfiguration$$EnhancerBySpringCGLIB$$98368e1$$FastClassBySpringCGLIB$$1351ccc2.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:309)
    at org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration$StackAutoDetectConfiguration$$EnhancerBySpringCGLIB$$98368e1.stackResourceRegistryFactoryBean(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
    ... 89 more

Don't automatically delete SQS messages upon receipt

When using @MessageMapping, the SimpleMessageListenerContainer seems to simply hand off the message to the handler and delete the message regardless of what happens during the actual invocation of the handler. This provides no method for error handling, and the message is completely lost. Message should not be auto-deleted unless the invocation is successful (and probably @sendto as well).

This might bubble up to spring-messaging, but perhaps an optional Ackable argument to @MessageMapping methods would be useful. This would give users some direct control over deletion of messages from SQS.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.