The Pravega benchmark tool used for the performance benchmarking of pravega streaming storage cluster. This tool performs the throughput and latency analysis for the multi producers/writers and consumers/readers of pravega. it also validates the end to end latency. The write and/or read latencies can be stored in a CSV file for later analysis. At the end of the performance benchmarking, this tool outputs the 50th, 75th, 95th , 99th, 99.9th and 99.99th latency percentiles.
- Java 8+
Checkout the source code:
git clone https://github.com/pravega/pravega-benchmark
cd pravega-benchmark
Build the Pravega benchmark Tool:
./gradlew build
untar the Pravega benchmark tool to local folder
tar -xvf ./build/distributions/pravega-benchmark.tar -C ./run
Running Pravega benchmark tool locally:
<dir>/pravega-benchmark$ ./run/pravega-benchmark/bin/pravega-benchmark -help
usage: pravega-benchmark
-consumers <arg> Number of consumers
-controller <arg> Controller URI
-enableConnectionPooling <arg> Set to false to disable connection
pooling
-events <arg> Number of events/records if 'time'
not specified;
otherwise, Maximum events per second
by producer(s) and/or Number of
events per consumer
-flush <arg> Each producer calls flush after
writing <arg> number of of
events/records; Not applicable, if
both producers and consumers are
specified
-help Help message
-producers <arg> Number of producers
-readcsv <arg> CSV file to record read latencies
-readWatermarkPeriodMillis <arg> If -1 (default), watermarks will not
be read.
If >0, watermarks will be read with a
period of this many milliseconds.
-recreate <arg> If the stream is already existing,
delete and recreate the same
-scope <arg> Scope name
-segments <arg> Number of segments
-size <arg> Size of each message (event or
record)
-stream <arg> Stream name
-throughput <arg> if > 0 , throughput in MB/s
if 0 , writes 'events'
if -1, get the maximum throughput
-time <arg> Number of seconds the code runs
-transactionspercommit <arg> Number of events before a transaction
is committed
-writecsv <arg> CSV file to record write latencies
-writeWatermarkPeriodMillis <arg> If -1 (default), watermarks will not
be written.
If 0 and not using transactions,
watermarks will be written after
every event.
If >0 and not using transactions,
watermarks will be written with a
period of this many milliseconds.
If >= 0 and using transactions,
watermarks will be written on each
commit.
-validateCertHostName Whether to turn on host name verification
for TLS certificates
The Pravega benchmark tool can be executed to
- write/read specific amount of events/records to/from the Pravega cluster
- write/read the events/records for the specified amount of time
The Pravega benchmark tool can be executed in the following modes:
- Burst Mode
- Throughput Mode
- OPS Mode or Events Rate / Rate limiter Mode
- End to End Latency Mode
In this mode, the Pravega benchmark tool pushes/pulls the messages to/from the Pravega client as much as possible. This mode is used to find the maximum and throughput that can be obtained from the Pravega cluster. This mode can be used for both producers and consumers.
For example:
<pravega benchmark directory>/run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname1 \
-segments 1 \
-producers 1 \
-size 100 \
-throughput -1 \
-time 60
The -throughput -1
indicates the burst mode.
This test will executed for 60 seconds because option -time 60
is used.
This test tries to write and read events of size 100 bytes to/from the stream 'streamname1'.
The option -controller tcp://127.0.0.1:9090
specifies the pravega controller IP address and port number.
Note that -producers 1 indicates 1 producer/writers.
in the case you want to write/read the certain number of events use the -events
option without -time
option as follows
<pravega benchmark directory>/run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname1 \
-segments 1 \
-producers 1 \
-size 100 \
-throughput -1 \
-events 1000000
-events <number>
indicates that total of events to write/read
In this mode, the Pravega benchmark tool pushes the messages to the Pravega client with specified approximate maximum throughput in terms of Mega Bytes/second (MB/s). This mode is used to find the least latency that can be obtained from the Pravega cluster for given throughput. This mode is used only for write operation.
For example:
<pravega benchmark directory>/run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname5 \
-segments 5 \
-producers 5 \
-size 100 \
-throughput 10 \
-time 300
The -throughput <positive number>
indicates the Throughput mode.
This test will be executed with approximate max throughput of 10MB/sec.
This test will executed for 300 seconds (5 minutes) because option -time 60
is used.
This test tries to write and read events of size 100 bytes to/from the stream 'streamname5' of 5 segments.
If the stream 'streamname5' is not existing , then it will be created with the 5 segments.
if the steam is already existing then it will be scaled up/down to 5 segments.
Note that -producers 5
indicates 5 producers/writers .
in the case you want to write/read the certain number of events use the -events
option without -time
option as follows:
<pravega benchmark directory>/run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname5 \
-segments 5 \
-producers 1 \
-size 100 \
-throughput 10 \
-events 1000000
-events 1000000
indicates that total 1000000 (1 million) of events will be written at the throughput speed of 10MB/sec
This mode is another form of controlling writers throughput by limiting the number of events per second. In this mode, the Pravega benchmark tool pushes the messages to the Pravega client with specified approximate maximum events per sec. This mode is used to find the least latency that can be obtained from the Pravega cluster for events rate. This mode is used only for write operation.
For example:
<pravega benchmark directory>/run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname1 \
-segments 1 \
-producers 5 \
-size 100 \
-events 1000 \
-time 60
The -events <event numbers>
(1000 ) specifies the events per second to write.
Note that the option -throughput
SHOULD NOT supplied for this OPS Mode or Events Rate / Rate limiter Mode.
This test will be executed with approximate 1000 events per second by 6 producers.
This test will executed for 300 seconds (5 minutes) because option -time 60
is used.
Note that in this mode, there is 'NO total number of events' to specify hence user must supply the time to run using -time option.
In this mode, the Pravega benchmark tool writes and read the messages to the Pravega cluster and records the end to end latency. End to end latency means the time duration between the beginning of the writing event/record to stream and the time after reading the event/record. in this mode user must specify both the number of producers and consumers. The -throughput option (Throughput mode) or -events (late limiter) can used to limit the writers throughput or events rate.
For example:
<pravega benchmark directory>./run/pravega-benchmark/bin/pravega-benchmark \
-controller tcp://127.0.0.1:9090 \
-stream streamname3 \
-segments 1 \
-producers 1 -consumers 1 \
-size 100 \
-throughput -1 \
-time 60
The user should specify both producers and consumers count for write to read or End to End latency mode. it should be set to true.
The -throughput -1
specifies the writes tries to write the events at the maximum possible speed.
User can use the options -writecsv <file name>
to record the latencies of writers and -readcsv <file name>
for readers.
in case of End to End latency mode, if the user can supply only -readcsv
to get the end to end latency in to the csv file.
To run against Security Enabled Pravega, you will need to specify TLS and authentication/authorization parameters
for the client via the JAVA_TOOL_OPTIONS
environment variable, before executing this tool.
Below is an example:
JAVA_TOOL_OPTIONS="-Djavax.net.ssl.trustStore=/path/to/client.truststore.jks"
JAVA_TOOL_OPTIONS="${JAVA_TOOL_OPTIONS} -Dpravega.client.auth.method=Basic"
JAVA_TOOL_OPTIONS="${JAVA_TOOL_OPTIONS} -Dpravega.client.auth.token=YWRtaW46MTExMV9hYWFh"
export JAVA_TOOL_OPTIONS
Note:
- The specified token
YWRtaW46MTExMV9hYWFh
shown in the example above equals Base64 encoded value of credentials in{username}:{password}
format supported by the specified authentication methodBasic
. The specified token is Base64 encoded value of stringadmin:1111_aaaa
, represent the admin account available in the Password Auth Handler database. If you are using another authentication method supported by a custom Pravega Auth Handler, generate a corresponding token, and specify the token and method here instead. - TLS host name verification is turned off by default to make it easier to run this tool. To enable hostname
verification, specify
-validateCertHostName true
option when executing it.
Build and push the Docker image to your repo.
export DOCKER_REPOSITORY=<your Docker user name>
export IMAGE_TAG=latest
scripts/build-docker.sh
Sample command to run in Docker:
docker run --rm -it \
--network host \
${DOCKER_REPOSITORY}/pravega-benchmark:${IMAGE_TAG} \
-controller tcp://localhost:9090 \
-stream benchmark1 -producers 1 -size 100 -throughput 0.01 -time 15
You must first build and push the Docker image as described above.
Edit the file scripts/pravega-benchmark.yaml. You must change the image value to match the DOCKER_REPOSITORY and IMAGE_TAG you used when built the Docker image. You must change the Pravega controller URI. You may also set other parameters as desired.
export NAMESPACE=examples
scripts/pravega-benchmark-k8s.sh.