ealsur / httpdatacollectorapi Goto Github PK

Azure Log Analytics HTTP Data Collector API wrapper

License: Apache License 2.0

C# 100.00%

azure-log-analytics oms azure log-collector

httpdatacollectorapi's Introduction

Azure Log Analytics HTTP Collector API wrapper

This package is a .Net adaptation of the Powershell code for implementing the HTTP Collector API for Azure Log Analytics as seen on the announcement.

Full API Documentation

Get it

You can obtain this project as a Nuget Package.

Install-Package HTTPDataCollectorAPI

Or reference it and use it according to the License.

Use it

Using it is simple:

var collector = new  HTTPDataCollectorAPI.Collector("{Your_Workspace_Id}", "{Your_Workspace_Key}");
await collector.Collect("TestLogType", "{\"TestAttribute\":\"TestValue\"}");

You can also pass serializable objects (they will be serialized with Newtonsoft.Json):

var anObject = new MySerializableClass(){
  MyIntAttribute = 4,
  MyStringAttribute = "hello",
  MyListAttribute = new List<string>(){"one","two"}
};
var collector = new  HTTPDataCollectorAPI.Collector("{Your_Workspace_Id}", "{Your_Workspace_Key}");
await collector.Collect("TestLogType", anObject);

If you are using a Depedency Injection mechanism (or ASP.NET Core) you can use the available interface. In ASP.NET Core for example:

public void ConfigureServices(IServiceCollection services)
{
  //... some other code
  services.AddSingleton<HTTPDataCollectorAPI.ICollector, HTTPDataCollectorAPI.Collector>();
}

Does it work on Azure Gov clouds?

Yes, just use the constructor overload to define the cloud service endpoint:

var collector = new  HTTPDataCollectorAPI.Collector("{Your_Workspace_Id}", "{Your_Workspace_Key}", "ods.opinsights.azure.us");

Issues

Please feel free to report any issues you might encounter. Keep in mind that this library won't assure that your JSON payloads are being indexed, it will make sure that the HTTP Data Collection API responds an Accept but there is no way (right now) to know when has the payload been indexed completely.

Supported Frameworks

.Net 4.5 Full Framework
.Net 4.6 Full Framework
.Net 4.6.1 Full Framework
.Net Standard Library 1.3

httpdatacollectorapi's People

Contributors

Stargazers

Watchers

Forkers

votok vascoinnofactor amd989

httpdatacollectorapi's Issues

HttpWebRequest retry strategy

Hi, is it possible to implement some HttpWebRequest retry strategy in your Collector?

Failed to get this to work

Hi,
Thank you for this library, I tried running the library but no data was posted to OMS.
I can post data with the same key and id through powershell, but was not able to do it via the library.

var collector = new HTTPDataCollectorAPI.Collector("{MyId}", "{MyKey}");
collector.Collect("TestLogType", "{"TestAttribute":"TestValue"}");
I haven't tried debugging it, but if you see anything wrong with the way I am calling it, please let me know.
I have very little experience with C#.

No support for time-generated-field header

According to this article https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-data-collector-api#create-a-request time-generated-field can be used to specify actual data timestamp.

Would be very useful for my scenario.

Will this library work for logging into private cloud like, US Gov cloud ?

Hi,

I observed that the code in Collector class, located at
...../HTTPDataCollectorAPI/HTTPDataCollectorAPI/Collector.cs, contains the domain name hard coded for azure public cloud as "ods.opinsights.azure.com". But this is different for private clouds.
Ex. for USGovt : "ods.opinsights.azure.us". Hence this may not work for other private clouds where this domain name differs.

the code in Collector.cs looks like below... where the domain name is highlighted in bold.

public async Task Collect(string LogType, string JsonPayload, string ApiVersion="2016-04-01", string timeGeneratedPropertyName = null)
{
var utf8Encoding = new UTF8Encoding();
Byte[] content = utf8Encoding.GetBytes(JsonPayload);

        string url = "https://" + _WorkspaceId + ".ods.opinsights.azure.com/api/logs?api-version=" + ApiVersion;

.............................................
}

This may have to be changed as configuration to make it work for all private/public clouds in azure.

Any quick solution is appreciable.

Thanks in advance.

Regards,
Kiran Kumar Gollapelly

JSON data is not showing up

JSON data is not showing up in OMS search portal. I see that I am able to successfully post, because it creates a type that I specified, but the body of the data record in OMS shows only two properties:
{
TimeGenerated: "....",
SourceSystem: RestAPI
}

the object I posted though looks like:

public class OMSEvent
{
public int Id { get; set; }
public double Value { get; set; }
public DateTime Timestamp { get; set; }
}

Code is very simple:

var message = new OMSEvent
{
Id = count,
Value = v,
Timestamp = DateTime.UtcNow
};
await mCollector.Collect("OMSEvent", message);

I thought I should be able to search in my properties....

Authorization header content length

When creating request header Authorization signature there is a jsonPayload.length used now.
However the request uses UTF8 encoded content (sometimes different) length and it results in azure returning 403 Forbidden response.

For example if jsonPayload contains some special characters such as 'ě', then it fails.

Please use the same (encoded) content length also when building signature.