danielfireman / esperf Goto Github PK

command_win.go:9:2: cannot find package "github.com/inconshreveable/mousetrap"

How are you creating the indexes for testing?

It would be important to understand what we have done, as well as for reporting. We could use JSON for that.

Some (hopefully) useful links:

• https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html
• https://www.elastic.co/guide/en/elasticsearch/guide/current/percentiles.html
• https://www.elastic.co/blog/averages-can-dangerous-use-percentile

As of ES 5.3 the Content-Type header should really be set: https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests. It will be required from 6.0 and onwards.

My ES deprecation log spams the following for every test at the moment:

[2017-09-28T14:02:17,197][WARN ][o.e.d.r.RestController   ] Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header.

Low priority, but would be super nice!

That will allow us to play with shed responses.

The numbers printed at the end (to double check sanity), could go to its own CSV (it is going to be useful in reports)

To make it easier to run experiments using production data, we need to anonymize fields from slowlogs and indexes. In our first evaluation, we decided for the following approach:

• Instead of using logstash or some other tool we are going to be based on match_all query to extract the data, process it and produce raw anonymized index sources. Those sources could be bulk added later.
• The index processing will generate two results:
  • anonymized sources items
  • a file containing which fields have been replaced. The old and new values
• The process of slowlogs will get the raw slowlogs and the anonymization file as input and will produce anonymized logs as output.

Need to wait for the workers to finish and then send the finish signal to the stats collector.

I'm running the command:
cat logs/elasticsearch_index_search_slowlog.log | ./esperf loadspec parseslowlog > slowlogs.loadspec.json
./esperf is a simlink for esperf-v0.1.2-windows-4.0-386.exe

This is the response I'm getting

panic: runtime

error: index out of range

goroutine 1 [running]:
github.com/danielfireman/esperf/cmd/loadspec.glob..func2(0x846320, 0x85919c, 0x0, 0x0, 0x0, 0x0)
/ext-go/1/src/github.com/danielfireman/esperf/cmd/loadspec/parse_slowlog.go:79 +0xa4b
github.com/danielfireman/esperf/vendor/github.com/spf13/cobra.(*Command).execute(0x846320, 0x85919c, 0x0, 0x0, 0x846320, 0x85919c)
/ext-go/1/src/github.com/danielfireman/esperf/vendor/github.com/spf13/cobra/command.go:644 +0x2e8
github.com/danielfireman/esperf/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x845fc0, 0x6caa20, 0x0, 0x0)
/ext-go/1/src/github.com/danielfireman/esperf/vendor/github.com/spf13/cobra/command.go:734 +0x28b
github.com/danielfireman/esperf/vendor/github.com/spf13/cobra.(*Command).Execute(0x845fc0, 0x0, 0x0)
/ext-go/1/src/github.com/danielfireman/esperf/vendor/github.com/spf13/cobra/command.go:693 +0x21
main.main()
/ext-go/1/src/github.com/danielfireman/esperf/main.go:11 +0x23

This is the content of my slowlog:

[2018-11-15 10:57:43,659][WARN ][index.search.slowlog.query] [Colonel] [test][0] took[23.3ms], took_millis[23], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],
[2018-11-15 10:57:43,665][WARN ][index.search.slowlog.query] [Colonel] [test][2] took[26.8ms], took_millis[26], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],
[2018-11-15 10:57:43,661][WARN ][index.search.slowlog.query] [Colonel] [test][3] took[107.9micros], took_millis[0], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],
[2018-11-15 10:57:43,674][WARN ][index.search.slowlog.query] [Colonel] [test][1] took[13.3ms], took_millis[13], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],
[2018-11-15 10:57:43,671][WARN ][index.search.slowlog.query] [Colonel] [test][4] took[95.1micros], took_millis[0], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],
[2018-11-15 10:57:43,694][WARN ][index.search.slowlog.fetch] [Colonel] [test][1] took[10.5ms], took_millis[10], types[], stats[], search_type[QUERY_THEN_FETCH], total_shards[5], source[{"query":{"match":{"test":"test"}}}], extra_source[],

I'm running this on Windows 10 Enterprise version 1803 using GitBash v2.19.1.

response.time file currently records the service time recorded by ES, as depicted in the fragment bellow:

r.responseTimes.Record(searchResp.TookInMillis)

As we move to a cluster-wide analysis - which might have an LB or reverse-proxy in the middle of the request service lifecycle - we are going to need to measure the actual end-to-end latency, from esperf back.

Measuring the time from https://github.com/danielfireman/esperf/blob/master/cmd/replay/replay.go#L236 to https://github.com/danielfireman/esperf/blob/master/cmd/replay/replay.go#L281

Unit tests are super welcomed!

Load should follow random variables (instead of only constant). Lets start with Poisson.

Useful links:

• https://github.com/leesper/go_rng

esperf currently only works correctly when the cluster has one node. This issue is about adding support for multiple nodes.

Our first approach will be to add the node-name to all metrics. Thus, instead of having: cpu_baseline_poisson_1.csv we would have cpu_baseline_node-1_poisson_1.csv. Leaving the actual metric name in the beginning and the experiment id at the end of the file name.

I am curious about how pauses behave with different types of load

Also ResponseTime struct

Just to have a human readable time

We are currently only using one term to perform the query. Lets use a query file, similar to the one used by solrjmeter. Our query file could be the format:

name value

For instance, "text America" would search for the America term at the Wikipedia index's text field.

We could start with a simple approach, load the whole file in memory in a slice of pairs. Each worker could then randomly select an index to start with and range the slice in a circular fashion.

Error message : {"error":"Content-Type header is missing","status":406}
ES version : 6.5

command : ./esperf counthits

Possible reason : https://www.elastic.co/blog/strict-content-type-checking-for-elasticsearch-rest-requests

Currently only using for stats.

I think It will decrease the size of the release binary a lot.

Suggest to support load test of Index data,and could help create test data automatic.