den-crane / clickhouse-regression Goto Github PK

This repository contains Altinity ClickHouse regression tests written using TestFlows.

• 1 Supported Environment
• 2 How to Run Tests
• 3 What is TestFlows
• 4 Running Tests Locally
  • 4.1 Output Verbosity
  • 4.2 Running Only Selected Tests
  • 4.3 Running S3 Suites
    • 4.3.1 Minio
    • 4.3.2 AWS S3 Storage
    • 4.3.3 GCS (Google Cloud Storage)
  • 4.4 Running Tiered Storage Suites
    • 4.4.1 Normal
    • 4.4.2 Running on Minio
    • 4.4.3 Running on AWS S3 Storage
    • 4.4.4 Running on GCS Storage

• Ubuntu 20.04
• Python 3 >= 3.8
• TestFlows >= 1.9.12

Before running tests you need to install the following:

• Docker install
• Docker Compose install

To install all necessary Python modules (including TestFlows), execute the following command:

$ pip3 install -r pip_requirements.txt

If you are interested to learn about TestFlows visit https://testflows.com where you can read the handbook https://testflows.com/handbook/.

You can run tests locally by passing --clickhouse-binary-path to the top level regression.py or cd into any sub-folders to run suite specific regression.py.

• --clickhouse-binary-path specifies the path to the ClickHouse binary on the host machine that will be used during the regression run. You can also use docker image that should have docker:// prefix.

For example,

$ ./regression.py --clickhouse-binary-path /usr/bin/clickhouse

$ ./regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine

Note: you can pass -h or --help argument to the regression.py to see a help message.

./regression.py -h

Note: make sure that the ClickHouse binary has correct permissions. If you are using /usr/bin/clickhouse its owner and group is set to root:root by default and it needs to be changed to clickhouse:clickhouse. You can change the owner and the group using the following command:

sudo chown clickhouse:clickhouse /usr/bin/clickhouse

Using the default ClickHouse installation and its server binary at /usr/bin/clickhouse, you can run regressions locally using the following command.

python3 regression.py --clickhouse-binary-path "/usr/bin/clickhouse"

You can control verbosity of the output by specifying the output format with -o or --output option. See --help for more details.

You can run only the selected tests by passing --only option to the regression.py.

For example,

./regression.py --clickhouse-binary-path /usr/bin/clickhouse --only "/clickhouse/rbac/syntax/grant privilege/*"

will execute all rbac/syntax/grant privilege tests.

If you want to run only a single test such as the /clickhouse/rbac/syntax/grant privilege/grant privileges/privilege='KILL QUERY', on=('*.*',), allow_introspection=False, you can do it as follows:

./regression.py --clickhouse-binary-path /usr/bin/clickhouse --only "/clickhouse/rbac/syntax/grant privilege/grant privileges/privilege='KILL QUERY', on=('[*].[*]',), allow_introspection=False/*"

Note that you need to surround special characters such as * with square brackets, for example [*].

Note that you need to end the filtering pattern with /* to run all the steps inside the test.

For more information, please see Filtering section in the TestFlows Handbook.

Minio is the default test suite, but can be specificed using --storage minio.

Examples:

Explicit storage declaration:

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --storage minio

Utilizing default values:

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine

You can also specify the minio uri (--minio-uri), root user (--minio-root-user), and root password (--minio-root-password). However, this is not necessary.

AWS storage suite requires a region (--aws-s3-region) and a bucket (--aws-s3-bucket) (the bucket must end with /), in addition to the key id and secret access key. AWS storage can be specified using --storage aws_s3.

Environment variables:

$ export AWS_ACCESS_KEY_ID=
$ export AWS_SECRET_ACCESS_KEY=
$ export AWS_DEFAULT_REGION=
$ export S3_AMAZON_BUCKET=

Examples:

Passing AWS parameters on the command line

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --aws_s3_key_id [masked] --aws_s3_access_key [masked] --aws-s3-bucket [masked] --aws-s3-region [masked] --storage aws_s3

Using AWS parameters defined using environment variables

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --storage aws_s3

GCS requires a gcs uri (--gcs-uri) (the uri must end with /), gcs key id (--gcs-key-id), and gcs key secret(--gcs-key-secret), in addition to the s3 key id and secret access key. GCS can be specified using --storage gcs.

Environment variables:

$ export GCS_URI=
$ export GCS_KEY_ID=
$ export GCS_KEY_SECRET=

Examples:

Passing GCS parameters on the command line:

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --gcs-uri [masked] --gcs-key-id [masked] --gcs-key-secret [masked] --storage gcs

Using GCS parameters defined using environment variables:

$ ./s3/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --storage gcs

Normal tiered storage suite does not require any variables to be provided.

From the regression directory, it can be run with the following command:

$ ./tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine

Minio tiered storage suite only requires that --with-minio is specified.

It can be run with the following command:

$ ./tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --with-minio

AWS S3 tiered storage requires an access key (--aws-s3-access-key), a key id (--aws-s3-key-id), and a uri (--aws-s3-uri). The uri must end with /. These can be passed as environment variables. AWS S3 must be specified using --with-s3amazon.

Environment variables:

$ export AWS_ACCESS_KEY_ID=
$ export AWS_SECRET_ACCESS_KEY=
$ export S3_AMAZON_URI=

Examples:

Passing AWS parameters on the command line:

$ ./tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --aws-s3-key-id [masked] --aws-s3-access-key [masked] --aws-s3-uri [masked] --with-s3amazon

Using AWS parameters defined using environment variables:

$ ./tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --with-s3amazon

GCS tiered storage requires a gcs uri (--gcs-uri) (the uri must end with /), gcs key id (--gcs-key-id), and gcs key secret(--gcs-key-secret). GCS can be specified using --with-s3gcs.

Environment variables:

$ export GCS_URI=
$ export GCS_KEY_ID=
$ export GCS_KEY_SECRET=

Examples:

Passing GCS parameters on the command line:

$ tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --gcs-uri [masked] --gcs-key-id [masked] --gcs-key-secret [masked] --with-s3gcs

Using GCS parameters defined using environment variables:

$ ./tiered_storage/regression.py --clickhouse-binary-path docker://clickhouse/clickhouse-server:22.3.6.5-alpine --with-s3gcs