Use with Serilog Sink about elasticsearch-net-aws HOT 10 CLOSED

Thanks a lot, your post and little snippet helped me figure out how I could integrate this. Turns out they have a ModifyConnectionSettings hook, I just didn't really know how to use it.

If anyone else was curious (this would probably be better off posted in their repo, but oh well).
Here's a pretty crude example of how you could get it to work.

  const string esUrl = "https://aws-es-thinger.us-west-1.es.amazonaws.com";
  Log.Logger = new LoggerConfiguration()
                .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(esUrl))
                {
                    ModifyConnectionSettings = conn =>
                    {
                        var httpConnection = new AwsHttpConnection("us-west-1", new StaticCredentialsProvider(
                            new AwsCredentials
                            {
                                // Service User Keys
                                AccessKey = "",
                                SecretKey = ""
                            }));
                        var pool = new SingleNodeConnectionPool(new Uri(esUrl));
                        return new ConnectionConfiguration(pool, httpConnection);
                    }
                })
                .CreateLogger();

Thanks a bunch for making this library! Saved me probably like 80 hours of fiddling with and trying to sign them requests!

from elasticsearch-net-aws.

It looks like you would probably need to fork the Serilog Elasticsearch Sink so that you can set up the client in a custom way. Maybe you can simply provide a setup overload that lets you inject a custom ElasticLowLevelClient instance or something and submit a PR back to the main line. https://github.com/serilog/serilog-sinks-elasticsearch/blob/dev/src/Serilog.Sinks.Elasticsearch/Sinks/ElasticSearch/ElasticsearchSinkState.cs#L84

You'll then need to setup an ElasticLowLevelClient instance like so

            var httpConnection = new AwsHttpConnection("us-east-1");
            var pool = new SingleNodeConnectionPool(new Uri("https://..."));
            var config = new ConnectionConfiguration(pool, httpConnection);
            var client = new ElasticLowLevelClient(config);

from elasticsearch-net-aws.

@KyleGobel I'm glad you figured that out.

That code example would be a good addition to the README. Feel free to PR it. I might do it myself if you don't :).

from elasticsearch-net-aws.

How would one provide basic auth credentials for the connection? In my use case, I have basic auth credentials (not AWS security credentials) that work in the browser, but not in the sink configuration.

from elasticsearch-net-aws.

@mariotacke I don't think AWS Elasticsearch Service supports basic http authentication. Either the request must be signed with valid AWS credentials or the inbound traffic needs to come from a whitelisted IP address. You can whitelist IP addresses in your elasticsearch domain's access policy.

from elasticsearch-net-aws.

Thanks @bcuff. I've asked my ops guys to give me proper credentials, that's probably the way to go.

from elasticsearch-net-aws.

Thanks a lot, your post and little snippet helped me figure out how I could integrate this. Turns out they have a ModifyConnectionSettings hook, I just didn't really know how to use it.

If anyone else was curious (this would probably be better off posted in their repo, but oh well).
Here's a pretty crude example of how you could get it to work.

  const string esUrl = "https://aws-es-thinger.us-west-1.es.amazonaws.com";
  Log.Logger = new LoggerConfiguration()
                .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(esUrl))
                {
                    ModifyConnectionSettings = conn =>
                    {
                        var httpConnection = new AwsHttpConnection("us-west-1", new StaticCredentialsProvider(
                            new AwsCredentials
                            {
                                // Service User Keys
                                AccessKey = "",
                                SecretKey = ""
                            }));
                        var pool = new SingleNodeConnectionPool(new Uri(esUrl));
                        return new ConnectionConfiguration(pool, httpConnection);
                    }
                })
                .CreateLogger();

Thanks a bunch for making this library! Saved me probably like 80 hours of fiddling with and trying to sign them requests!

This works only in .net core with was elastic search. Can you help me on how to handle the same using with standard asp.net

from elasticsearch-net-aws.

You're welcome. Thanks for sharing the snippet!

from elasticsearch-net-aws.

@mariotacke Hi, any updates on this? I'm in a similar situation and searching web for a solution..

from elasticsearch-net-aws.

@orhunerdem, it's been a long time but IIRC, I ended up using AWS credentials instead of basic auth as suggested by @bcuff and @rtrscience .

from elasticsearch-net-aws.