I think it breaks if I EnableHttpCompression() on the ConnectionSettings about elasticsearch-net-aws HOT 11 CLOSED

Hi, we are experiencing the same issue using .NET 5 + Elasticsearch.Net.Aws 7.1. When HTTP compression is enabled in NEST configuration and in an Elasticsearch cluster, the following error occurs:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

Any updates on it?

from elasticsearch-net-aws.

Merged the PR and published 7.2.0. I also addressed the potential stack overflow that was pointed out in the Dispose in 9186e73

I'll close this issue for now. Feel free to reopen if there are still problems. Thanks for all the feedback!

from elasticsearch-net-aws.

Closing this (housekeeping)

from elasticsearch-net-aws.

This seems like a legitimate issue. When I enable compression, AWS tells me that my signature doesn't match. Everything works fine until I turn compression on.

Am I just missing something obvious? Is this a known problem?

from elasticsearch-net-aws.

@jp-cai can you provide:

• code snippets
• runtime version
• elasticsearch.net version

This will help me reproduce the problem. Thanks!

from elasticsearch-net-aws.

Sure!
This is in a .NET Core 3.1 application, using NEST + Elasticsearch.Net v7.8.2 and Elasticsearch.Net.Aws v7.0.6.

The issue occurs both running directly on my Windows machine (dotnet --version 3.1.402), and in a Linux container (mcr.microsoft.com/dotnet/core/aspnet:3.1).

As mentioned, this all works fine with EnableHttpCompression(false);

        private static IElasticClient BuildElasticClient(Uri uri)
        {
            uri.MustNotBeNull(() => throw new InvalidConfigurationException("Elasticsearch:Uri cannot be null."));

            var awsConnection = new AwsHttpConnection();

            var connectionSettings = new ConnectionSettings(
                new SingleNodeConnectionPool(uri),
                sourceSerializer: (builtin, settings) => new JsonNetSerializer(
                    builtin, settings,
                    () => new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore }
                ),
                connection: awsConnection
            );

            //todo: this seems to result in an invalid signature
            connectionSettings.EnableHttpCompression();
            connectionSettings.ThrowExceptions();
            connectionSettings.DefaultFieldNameInferrer(x => x);
#if DEBUG
            connectionSettings.EnableDebugMode().PrettyJson();
#endif

            var client = new ElasticClient(connectionSettings);

            return client;
        }

from elasticsearch-net-aws.

Thanks. I'll take a look and get back to you.

from elasticsearch-net-aws.

@jp-cai sorry it took so long to get back to you. I attempted to solve the problem with this PR. However I was unable to get this to work with an AWS elasticsearch instance I was targeting. I did get past the signing errors but then ran into a 500 instead. I opened a support case with AWS and they weren't much help. Can you try compiling this yourself and seeing if it works against your target Elasticsearch service domain?

from elasticsearch-net-aws.

@skovalyova this PR was supposed to address the issue. I gets past the signing error but I end up getting a 500 on the version of AWS ESS that I was using. If you have the time/inclination you can try it out on your domain. If it works for you let me know.

I didn't have much luck with AWS support but I haven't tried it on the latest 7.x+ versions so it's worth a shot.

from elasticsearch-net-aws.

Actually this code (with PR) works for my team right now
AWS Lambda + dotnet 3.1 + AWS Elasticsearch with compression on

But
We're experiencing stack overflow exception in aws and I believe that this library is a reason for that
We're trying to find that part of the code that is a reason for SO

update: I think dispose pattern is not implemented correctly in SigningHttpMessageHandler. After we implemented as it should be (https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/implementing-dispose#implement-the-dispose-pattern-for-a-derived-class) SO exception disappeared

from elasticsearch-net-aws.

Update. 7.2.0 had a bug that still caused the stack overflow. Please use 7.2.1

from elasticsearch-net-aws.