This project empirically evaluates Memcached, Redis and Aerospike key-value store leveraging Yahoo! Cloud System Benchmark (YCSB) with various workloads and benchmarking settings aiming to provide useful insight for each kv-store.
- The code in this repository are based on YCSB repository.
- Modified code is in redis and redis-binding directory. (Diff)
- Automated scripts are added for benchmarking the three databases.
- Configuration files for running YCSB in our benchmarking is available in benchmark directory.
- Sample configuration files for running YCSB in our benchmarking is available in benchmark/db_configs directory.
Installing via apt-get
apt-get -y update
apt-get -y install memcached
Changing the configuration
vim /etc/memcached.conf
Running Memcached server
service memcached start
Installing via tarball
wget http://download.redis.io/redis-3.2.5.tar.gz
tar xvzf redis-3.2.5.tar.gz
cd redis-3.2.5
make
make install
Changing configuration file
vim redis-3.2.5/redis.conf
Running Redis server
redis-server redis-3.2.5/redis.conf
Instsalling Aerospike server
wget -O aerospike.tgz 'http://aerospike.com/download/server/latest/artifact/ubuntu12'
tar -xvf aerospike.tgz
cd aerospike-server-community-*-ubuntu12
sudo ./asinstall
Changing configuration file
vim /etc/aerospike/aerospike.conf
Runing Aerospike server
`sudo service aerospike start &&
sudo tail -f /var/log/aerospike/aerospike.log | grep cake
Start the databases server(s) as instructed above.
Install the modified YCSB project
git clone https://github.com/anthonyaje/ycsb-kvstore-comparison.git
Notice before running cluster mode benchmarking
cd ~/ycsb-kvstore-comparison/redis-binding
cp -rf ./cluster-lib/* ./lib/
Notice before running single node benchmarking
cd ~/ycsb-kvstore-comparison/redis-binding
cp -rf ./single-lib/* ./lib/
Execute the YCSB load phase
./load_benchmark.sh [redis|riak|memcached] [single|cluster] [balanced|write_heavy|read_heavy]
Execute the YCSB run phase
./run_benchmark.sh [redis|riak|memcached] [single|cluster] [balanced|write_heavy|read_heavy]
This script will execute 4 iterations of YCSB run phase for each number of threads 1 4 8 12 16 20 24 28 32.
The results will be generated in results/$1/$2/$3/out_run_$t-$k
where
$1 $2 $3
are the parameters passed when executing the scripts
$t
and $k
are #thread and #iteration.
Remove the result for the 1st run (the result for the first cold start)
cd
to the results directory
rm out_run_*-1
Calculate the average throughput and latencies
cd
to the results directory
cp ~/ycsb-kvstore-comparison/calc_* ./
./calc_tp_avg.sh
./calc_lat_read_avg.sh
./calc_lat_read_avg.sh
This scripts generate tp_avg
, lat_read_avg
, and lat_write_avg
that contain the average throughput, read latency, and write latency for each #thread respectively.