Vodka is a new benchmark suite for benchmarking HTAP database systems, which is extended from the popular TPC-C and TPC-H benchmark. Details about our designs and experiments on three open-source HTAP Systems (OceanBase, TiDB, PostgreSQL Streaming Replication(PostgreSQL-SR)) can be found in the Technique Report:Vodka: Fairly Benchmarking HTAP Systems on Demand (2024 VLDB In submission). We show how to use Vodka as follows.
Vodka project is forked from benchmarksql v5.0 and distributed under the Apache-2.0 license.
- A client machine to run Vodka benchmarking program and workload proxy (ObProxy for OceanBase cluster, Haproxy for TiDB cluster). Proxy is unnecessary in the case of PostgreSQL-SR.
- A cluster of 3 machines to deploy database service.
- Modify hostname, remove redundant clients and servers, set correct user and password (make sure that client0 and server1 exist).
- Apache ivy is used to manage dependencies, ant is used to build the project and jdk8 is required.
yum install -y ant java-1.8* apache-ivy
- Clone the project to the client machine and build the project.
yum install -y git
git clone https://github.com/Wind-Gone/Vodka --recursive
cd Vodka
ant resolve
ant
- Install necessary tools, copy client ssh key to servers, adjust server parameters and synchronize clocks.
wget https://github.com/chaosblade-io/chaosblade/releases/download/v1.6.1/chaosblade-linux-amd64.tar.gz
tar -zxvf chaosblade-linux-amd64.tar.gz
sudo yum install -y sshpass
ssh-keygen -t rsa
# make sure that you can connect to the client and servers in root user
# copy ssh key from client to servers
./cluster_initialize.sh PROPERTIES_FILE copySshKey
# install chaosblade on servers for fault injection
./cluster_initialize.sh PROPERTIES_FILE installChaosblade
# configure system parameters for client and servers
./cluster_initialize.sh PROPERTIES_FILE configureSystem
# synchronize ntp clock in servers (change network segment first)
./cluster_initialize.sh PROPERTIES_FILE ntpSynchronize
# copy os_collector script to servers
./cluster_initialize.sh PROPERTIES_FILE copyOsCollector
- All deployment scripts and config files are placed in the deploy directory, refer to readme for details about deployment procedure.
- Connect to database server with mysql client or psql and create database Vodka.
CREATE DATABASE IF NOT EXISTS Vodka;
- Modify the parameters in the property file, details about the meaning of each parameter can be found in readme.
- Create tables and load in data with the modified property file.
./runDatabaseBuild.sh ../config/oceanbase.properties
Otherwise, use the given template property files instead (also need to modify connection parameters first). Check template for more details about each template properties file.
./runDatabaseBuild.sh ../config/template/oceanbase/Vodka.properties
Though a brief report is printed to standard output stream at the end of benchmark, more detailed statistical indicators such as throughput per second and system resource usage are also provided.
- Install R language tool.
sudo yum install -y R
R
# install R packages
install.packages('ggplot2')
install.packages('gridExtra')
install.packages('dplyr')
- Suppose the result directory is 'results/oceanbase/Vodka/my_result_%tY-%tm-%td_%tH%tM%tS', generate statistics reports by running generateResults.sh. Check misc for details about outputs.
./generateResults.sh results/oceanbase/Vodka/my_result_%tY-%tm-%td_%tH%tM%tS