Collections in model POJOs should be wrapped in unmodifiable collections to prevent mutation after they are configured.

This affects streaming sync clients like S3Client.

We are relying on the context class loader (the class loader for the current thread) when loading the HTTP implementations. https://github.com/aws/aws-sdk-java-v2/blob/master/core/src/main/java/software/amazon/awssdk/http/loader/SdkServiceLoader.java#L29

We should instead use the class loader that loaded the SDK to preserve the semantics of ordinary classloading.

ServiceLoader.load(clzz, SdkServiceLoader.class.getClassLoader()) 

Some pieces of configuration are supported per-request (eg. read timeouts), but currently can only be specified at the client level. It would be useful to be able to override these configurations on a per-request basis when certain requests require different timeouts.

Review the inherited state of V1 progress listeners and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Clients generated for API gateway customers use the "core" module for transmission and other pieces of core functionality. This means that such clients also pull in AWS-specific functionality, like credential providers, regions, etc.

These AWS-specific pieces of functionality should be moved to a separate "aws-core" module that depends on "core" so that API gateway clients do not need to include them.

Since all access to EC2MetadataUtils is static, the class is very difficult to use in a properly tested application. It can't be mocked; you can't give canned answers when running in a non-EC2 environment.

If there were an EC2Metadata interface or abstract class of which there were a default implementation, then it would be easier to use.

A contrived example:

public class Application {
    EC2Metadata metadata;
    public static void main(String args[]) {
        new Application(EC2MetadataUtils.getDefaultInstance());
    }
    public Application(EC2Metadata metadata) {
        this.metadata = metadata;
        LOG.info("We're running on " + metadata.getInstanceId());
    }
}

public class ApplicationTest {
    @Mock EC2Metadata metadata;
    @Test public testApp() {
        when(metadata.getDefaultInstance()).then(return "i-fakeinstance");
        new Application(metadata);
        verifyLoggedLine("We're running on i-fakeinstance");
    }
}

As it stands today, I need to run a whole HTTP process that mimics the metadata server and somehow get the system property to point to it; and even that is a global setting for the whole JVM: parallel tests can interfere with each other. Or I can write my own EC2Metadata interface and use it throughout my app, but it's not something each developer should have to do.

In some cases, S3 will return an HTTP 200, but will request that the request be retried in the message payload. These cases should be automatically retried by the SDK.

Make sure we play nice with the module systems in OSGI and Java9 (especially our detection of the default HTTP implementation). Ensure that we can provide modularized service clients to OSGI environments instead of a single uber jar containing the entire SDK.

The 2.0.x SDK has inherited the 1.11.x dependency on joda-time. The SDK should be updated to use Java 8 time features so that we can remove the dependency on joda-time.

We currently need Python and the jmespath package to generate waiters. We should remove the dependency by using a Jmespath library built for Java (either our own or a 3rd party).

Review the inherited state of V1 profile file support and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Customers are currently required to iterate over paginated result sets themselves in almost all locations by making extra service calls to retrieve more results if they're loading large sets of results.

We currently provide depaginators for S3 (S3Objects) and DynamoDBMapper results (PaginatedList), but these patterns are applied inconsistently across the code base.

Provide general-purpose List-view methods for traversing and accessing full result sets without having to make multiple service calls manually or loading the full result set in memory at one time.

As a user of the SDK, having a formal name for the EventLoop threads, will make it easier to debug and troubleshoot production issues.

For example, a named thread pool is easier to find in a thread dump or while doing some CPU profiling.

Threads could be named something like:
"AWS-SDK-EventLoopThread-<GroupNumber>-<WorkerNumber>".

GroupNumber is the EventLoopGroup number. Typically most clients will have one instance of the AWS SDK instantiated per JVM and this will always be 1. For some reason, they do spin up multiple, having different GroupNumber will avoid thread name collisions.

WorkerNumber is the thread number within an EventLoopGroup.

The new v2 API looks to be following most of the recommendations for good Java APIs, which is great. One key thing I've not seen mentioned however is the use of null.

Ideally, no method in the SDK should return null. The Optional class should be used instead. This includes POJOs as well as key classes like AmazonS3.

If this proves to be too big a change, at least the key classes like AmazonS3 should never return null.

This particularly affects cases where a remote call is made and the target is not found, eg row not found in Dynamo. At present, some services return null while others throw an exception, and this is often not well documented. A consistent use of Optional for missing data (or a consistent use of exceptions) would be desirable.

It isn't obvious without looking for an SdkInternal annotation (that may not even be there) which classes are internal to the SDK. If internal classes were moved to an package with "internal" in the name, this would make it more obvious what should be used by customers. It will also simplify things during Java 9 modularization.

In 1.11.x, customers can execute certain EC2 requests as a "dry-run" request. This adds a special header to tell EC2 to not actually execute the request.

Support for this feature should be added in 2.0.

Review the inherited state of V1 exceptions and determine whether any changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Execution and request timeouts are currently exposed on the client configuration override object, but don't actually work. This functionality should be fixed.

Review the inherited state of V1 retry policies and determine whether any changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Review the inherited state of V1 waiters and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Support for API gateway client generation isn't yet supported for 2.0.x-style SDKs. The 2.0.x should be able to support generating API gateway clients, especially because it is planned to support such clients in a more straight-forward way than 1.11.x.

The S3 models treat tags inconsistently and unintuitively between APIs, which makes it difficult to copy values between APIs or specify tags on requests (how do you format the String tagging field on a PutObjectRequest?)

Add a customization to the generated S3 models to make this more consistent and intuitive.

The credential file parsing is currently public, as well as the writer. The location of these files is marked as internal though. We should look into making the parser able to write updated credentials back to the file.

For customers that are willing to (or have to) trade functionality for reduced dependencies, we should provide an HTTP implementation that uses native Java libraries for transmission instead of Apache.

For some AWS APIs, customers may wish to execute multiple concurrent requests over a single connection without relying on the service having implemented a "batch" version of the API. An API could be provided to execute multiple "pipelinable" requests over a single connection if the HTTP client supports it.

DynamoDbMapper requires all mapped classes to be mutable. Immutable domain objects are increasingly the norm in Java, and using DynamoDbMapper means creating mutable duplicates of all domain objects or giving up on the benefits of immutability.

The step functions DSL is relatively new, so it shouldn't need major changes, but we should still review its current state and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

SLF4J is more commonly suggested as an alternative to Apache commons logging because of its simplicity and reduction in magic. Look at cold-start performance, features, usability and consider migrating to SLF4J.

Generated clients currently inherit their close() method from AutoCloseable, which currently declares that it throws Exception. In actuality, checked exceptions will never be throw. The client interfaces should override the close() method to remove the throws statement.

The default user-agent format will reader aws-sdk-java/2.0.0-preview-1 Linux/4.9.37-1-lts Java_HotSpot(TM)_64-Bit_Server_VM/25.131-b11/1.8.0_131 scala/2.12.2 on my machine. According to rfc 7231 this is illegal. The part Java_HotSpot(TM)_64-Bit_Server_VM/25.131-b11/1.8.0_131 contrains two versions and the product name contains ().

Background: I'm implementing a async http client using akka-http, which implement parsers according to the RFCs. This might not be an issue in the past but since using a different http client is a feature now, everything being passed to/from the http client should be rfc compliant.

The 1.11.x S3 client automatically routed requests to the correct bucket region in a complicated manner. The 2.0.x client does not perform this automatic re-routing.

We should create a wrapper to pool S3 clients for different regions and automatically route requests to the region in which the bucket was created.

Review the inherited state of V1 Dynamo DB mapper support and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Ensure the way we load the default HTTP implementation is compatible with common shade use cases like re-namespacing the SDK.

$ mvn dependency:tree
+- software.amazon.awssdk:aws-sdk-java:jar:2.0.0-preview-1:compile
|  +- software.amazon.awssdk:cloudsearch:jar:2.0.0-preview-1:compile
|  |  \- com.github.tomakehurst:wiremock:jar:1.55:compile
|  |     +- org.mortbay.jetty:jetty:jar:6.1.26:compile
|  |     |  +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
|  |     |  \- org.mortbay.jetty:servlet-api:jar:2.5-20081211:compile

The org.mortbay classes (especially the servlet-api.jar) are very old versions of the servlet api that (in my case at least) ended up shadowing the modern servlet api, meaning my server wouldn't work after I added the aws-sdk-java dependency.

I can probably exclude things to make it work, but it seems like cloudsearch shouldn't bring in wiremock at all, let alone letting it bring in an ancient version of jetty.

Browsing the code I don't see anywhere wiremock is used by the cloudsearch jar.

It failed with this stack trace in a dropwizard server:

WARN  [2017-07-11 20:49:36,099] org.eclipse.jetty.server.HttpChannel: /swagger.json
! java.lang.NoSuchMethodError: javax.servlet.http.HttpServletResponse.getStatus()I
! at com.codahale.metrics.jetty9.InstrumentedHandler.updateResponses(InstrumentedHandler.java:289)
! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:252)
! at io.dropwizard.jetty.ContextRoutingHandler.handle(ContextRoutingHandler.java:38)
! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
! at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:68)
! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
! at org.eclipse.jetty.server.Server.handle(Server.java:564)
! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
! at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
! at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
! at java.lang.Thread.run(Thread.java:745)

With the rise of serverless frameworks, library's cold-start latency has become increasingly important to minimize. Libraries that rely heavily on reflection, class-path scanning, needlessly-large thread pools, etc. greatly increase the time it takes to get a lambda JVM to a state that it can process requests.

Cold-start latency of the SDK should be benchmarked and expensive libraries (I'm looking at you, Jackson databind) should be moved out of the critical startup path.

The IAM policy builder includes some hacks and changes, like removing dashes from account IDs, that could be considered unintuitive.

Review the inherited state of the V1 IAM policy builder and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

As a user of the AWS SDK, I want to be able to configure the number of threads used assigned to the EventLoopGroup.

Currently, this is not configurable and the library relies on Netty to pick the number of threads. Netty defaults to 2 * number of cores, as the thread pool size: https://github.com/netty/netty/blob/4.1/transport/src/main/java/io/netty/channel/MultithreadEventLoopGroup.java#L40-L41

While this is good for most users, some users would want to configure this based on application requirements and available resources on client's host.

Client-side encryption and signing is supported by the S3 and dynamo DB clients in 1.11.x, but not in 2.0.x. This feature is used by a large number of customers and should be supported in 2.0.x.

4/6/23 Update: The AWS crypto tools team has launched the S3 encryption client with support for the AWS SDK for Java 2.x! Check it out here:

• https://docs.aws.amazon.com/amazon-s3-encryption-client/latest/developerguide/what-is-s3-encryption-client.html
• https://github.com/aws/amazon-s3-encryption-client-java

11/17/23 Update: A new Database Encryption client was launched with support for the AWS SDK for Java 2.x, and it's also maintained by the AWS Crypto Tools team. For more info:

• https://docs.aws.amazon.com/database-encryption-sdk/latest/devguide/what-is-database-encryption-sdk.html
• https://github.com/aws/aws-database-encryption-sdk-dynamodb-java/

Review the inherited state of V1 signers and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Implement payload checksum validation on V2 to work for the new generated S3 client.

Review the inherited state of V1 marshallers and unmarshallers and determine which changes are necessary for V2. In particular, they'll need to be migrated to be generated by poet instead of freemarker.

(Feel free to comment on this issue with desired changes).

Review the inherited state of V1 metrics and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

Review the inherited state of V1 Dynamo DB document API and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).

One of the most controversial things in blog post is a serialization that should be done by using builders

String serialized = mapper.writeValueAsString(request.toBuilder());

ListTablesRequest deserialized = mapper.readValue(serialized, ListTablesRequest.serializableBuilderClass())
                                       .build()

As you already set java 8 as a baseline and use jackson as a serialization library, why not use https://github.com/FasterXML/jackson-modules-java8 that will give you correct immutable object creation with constructor without extra manual step?

Request handlers have become particularly messy in 1.11.x. There are multiple versions of request handlers and none are feature complete enough to support features that make sense to be implemented as request handlers, like metrics, signing and encryption. This makes add-on features like the API gateway clients require more significant changes in the core than could usually be required.

Update the request handler abstraction to support the expected current and future use-cases.

(Feel free to comment on this issue with desired changes).

All model objects currently extend AmazonWebServiceRequest, which includes a large number of deprecated 1.11.x concepts. This base class should be removed or replaced to clean up the contents of our model objects.

Customers are currently required to manually poll an SQS queue for data. An event-driven SQS consumer should be provided that handles this polling so that customers can just focus on processing messages.

Review the inherited state of the V1 transfer manager and determine which changes are necessary for V2.

(Feel free to comment on this issue with desired changes).