CodeArtifact .publishPackageVersion fails for large generic artifact with 'InvalidSignatureException' about aws-sdk-kotlin HOT 7 CLOSED

Hi, thanks for the detailed report, I've reproduced the issue and am working on a fix. In the meantime, here is a temporary workaround using an interceptor:

class PrecomputedHashInterceptor(val hash: String): HttpInterceptor {
    override suspend fun modifyBeforeRetryLoop(context: ProtocolRequestInterceptorContext<Any, HttpRequest>): HttpRequest {
        context.executionContext[AwsSigningAttributes.HashSpecification] = HashSpecification.Precalculated(hash)
        return context.protocolRequest
    }
}

val testFile = File("build/sql-database-fingerprint-baseline.bak.zip")
val testFileSha256 = testFile.sha256()

val client = CodeartifactClient {
    credentialsProvider = creds
    region = "us-east-1"
    logMode = LogMode.LogRequest + LogMode.LogResponse
}

client.withConfig { 
    interceptors += PrecomputedHashInterceptor(testFileSha256)
}.use { precomputedHashClient ->
    precomputedHashClient.publishPackageVersion {
        ...
        assetSha256 = testFileSha256
    }
}

from aws-sdk-kotlin.

Hi, I merged a fix for this. It should be available in tomorrow's release, v1.0.61. Thanks!

Besides upgrading, there is no action needed from you to receive the fix. The change disables aws-chunked content encoding for all services except S3, since that's the only service that explicitly supports it.

from aws-sdk-kotlin.

fail-kotlin-sdk.txt

Full debug log

from aws-sdk-kotlin.

fail-awscli.txt

AWS CLI debug logs

from aws-sdk-kotlin.

In case this is a service issue, the AWS case # is 170802144901259

from aws-sdk-kotlin.

The signature algorithm appears to be incorrect for Kotlin SDK, when compared to the same request for Java SDK v2.

Java SDK sends the payload as a single chunk, whereas Kotlin SDK uses was-chunk encoding.

From the documented signature algorithms (single chunks, multiple chunks) the value of x-amz-content-sha256 is different:

• for single chunk its the calculated sha256 of the entire payload, matching what is passed to this CodeArtifact call;
• for multiple chunks its the constant STREAMING-AWS4-HMAC-SHA256-PAYLOAD

For the Java SDK call it correctly provides the x-amz-content-sha256 header and signature; for the Kotlin SDK call it provides x-amz-content-sha256:f2e321d69a55ce94bb472bc2c8e83b8271d7775a7ba61b690e1d2a654b6af4f1 instead of x-amz-content-sha256:STREAMING-AWS4-HMAC-SHA256-PAYLOAD, in both the canonical request and request headers.

Java canonical request:
    POST
    /v1/package/version/publish
    asset=the.asset.name&domain=repro-test-domain&domain-owner=331062209604&format=generic&namespace=whatever&package=this.is.the.package2&repository=generic&version=1.0.0
    amz-sdk-invocation-id:56ffa762-0f24-34c1-f3c5-37b5da6be7b2
    amz-sdk-request:attempt=1; max=4
    content-length:937099048
    content-type:application/octet-stream
    host:codeartifact.us-east-1.amazonaws.com
    x-amz-content-sha256:f2e321d69a55ce94bb472bc2c8e83b8271d7775a7ba61b690e1d2a654b6af4f1
    x-amz-date:20240216T035949Z
    x-amz-security-token:<redacted>

    amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token
    f2e321d69a55ce94bb472bc2c8e83b8271d7775a7ba61b690e1d2a654b6af4f1

Kotlin canonical request
    POST
    /v1/package/version/publish
    asset=the.asset.name&domain=repro-test-domain&domain-owner=331062209604&format=generic&namespace=whatever&package=this.is.the.package&repository=generic&unfinished=false&version=1.0.0
    amz-sdk-invocation-id:b3fa77eb-4953-4eae-abf6-bd36f57a3745
    amz-sdk-request:attempt=1; max=3
    content-encoding:aws-chunked
    content-type:application/octet-stream
    host:codeartifact.us-east-1.amazonaws.com
    transfer-encoding:chunked
    x-amz-content-sha256:f2e321d69a55ce94bb472bc2c8e83b8271d7775a7ba61b690e1d2a654b6af4f1
    x-amz-date:20240215T172033Z
    x-amz-decoded-content-length:937099048
    x-amz-security-token:<redacted>
    x-amz-user-agent:aws-sdk-kotlin/1.0.56

    amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-type;host;transfer-encoding;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-security-token;x-amz-user-agent
    STREAMING-AWS4-HMAC-SHA256-PAYLOAD

...this results in the CodeArtifact service calculating a different signature than what as incorrectly calculated by the Kotlin SDK client.

Using a chunked transfer for this CodeArtifact call also raises the question of where does the asset-sha256 value go (it's part of the SDK parameters for this call)? Is it then unused, or ... ? Likely a better question for the service-side of this, what is expected/supported (i.e. is was-chunked even supported for this API call)?

Answering my own question... from the API docs this call looks to require single-chunk transfer:

POST /v1/package/version/publish?asset=assetName&domain=domain&domain-owner=domainOwner&format=format&namespace=namespace&package=package&repository=repository&unfinished=unfinished&version=packageVersion HTTP/1.1
x-amz-content-sha256: assetSHA256

assetContent

The Kotlin SDK code is partially consistent with that API spec:

        builder.headers {
            if (input.assetSha256?.isNotEmpty() == true) append("x-amz-content-sha256", input.assetSha256)
        }

...however, manually setting the x-amz-content-sha256 header is incorrect wrt multi-chunk encoding.

Omitting the assetSha256 parameter removes the x-amz-content-sha256 header entirely, which is also incorrect wrt multi-chunk encoding (this also fails with the same signature error).

So it appears the issue here is the Kotlin SDK, for this API call, is selecting multi-chunk encoding which isn't compatible with the spec for this call.

Likely fixable in aws.smithy.kotlin.runtime.http.auth.AwsHttpSigner#sign - before checking for body.isEligibleForAwsChunkedStreaming check to see if x-amz-content-sha256 is already set, if so then return a HashSpecification.HashLiteral for it. Perhaps with a pre-send check to ensure that x-amz-content-sha256 is set appropriately vis-a-vis content-encoding.

from aws-sdk-kotlin.

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

from aws-sdk-kotlin.