mediarithmics / liquibase-clickhouse Goto Github PK

License: Apache License 2.0

Java 100.00%

liquibase-clickhouse's Introduction

liquibase-clickhouse

Supported operations: update, rollback (with provided SQL script), tag

Maven dependency:

<dependency>
    <groupId>com.mediarithmics</groupId>
    <artifactId>liquibase-clickhouse</artifactId>
    <version>Latest version from Maven Central</version>
</dependency>

The cluster mode can be activated by adding the liquibaseClickhouse.conf file to the classpath (liquibase/lib/).

cluster {
    clusterName="{cluster}"
    tableZooKeeperPathPrefix="/clickhouse/tables/{shard}/{database}/"
    tableReplicaName="{replica}"
}

In this mode, liquibase will create its own tables as replicated.
All changes in these files will be replicated on the entire cluster.
Your updates should also affect the entire cluster either by using ON CLUSTER clause, or by using replicated tables.

Important changes

From the version 0.7.0 the liquibase-clickhouse supports replication on a cluster. Liquibase v4.6.1.

From the version 0.6.0 the extension adapted for the liquibase v4.3.5.

liquibase-clickhouse's People

Contributors

Stargazers

Watchers

liquibase-clickhouse's Issues

liquibase:tag does not work on cluster

Hello,

I have tried to add a tag to my changelog but got the following error.

[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:4.6.1:tag (default-cli) on project 36mOpenseeLiquidityDataModel: [ERROR] Error setting up or running Liquibase: [ERROR] ClickHouse exception, code: 62, host: iqoprdweb057, port: 8123; Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 1 ('UPDATE') : UPDATE DATABASECHANGELOG SET TAG = '1.0.0' WHERE DATEEXECUTED = (SELECT MAX(DATEEXECUTED) FROM DATABASECHANGELOG). Expected one of: ALTER query, Query with output, ALTER PROFI LE, RENAME DATABASE, SHOW PRIVILEGES query, TRUNCATE, KILL, KILL QUERY query, SELECT query, possibly with UNION, list of union elements, ALTER ROLE, SELECT subquery, DESCRIBE qu ery, SELECT query, subquery, possibly with UNION, SHOW GRANTS, SHOW CREATE, WATCH, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, SHOW PROCESSLIST query, ALTER POLICY, ALTER USER, CREATE VIEW query, CHECK TABLE, SET ROLE, SELECT query, SELECT, REVOKE, CREATE USER, CREATE DICTIONARY, CREATE PROFILE, SET ROLE DEFAULT, EXPLAIN, ALTER SETTINGS PR OFILE, SYSTEM, ALTER LIVE VIEW, RENAME TABLE, DROP query, SHOW ACCESS, OPTIMIZE query, USE, DROP access entity query, RENAME DICTIONARY, DETACH, SET, SHOW, DESC, OPTIMIZE TABLE, CREATE ROW POLICY, SET DEFAULT ROLE, EXCHANGE DICTIONARIES, CREATE POLICY, ALTER ROW POLICY, INSERT INTO, INSERT query, SHOW [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name ' [[NOT] [I]LIKE 'str'] [LIMIT expr], GRANT, RENAME query, SHOW GRANTS query, SHOW PRIVILEGES, EXISTS, DROP, SYSTEM query, CREATE LIVE VIEW query, CREATE ROW POLICY or ALTER ROW POLICY query, CREATE QUOTA or ALTER QUOTA query, SHOW PROCESSLIST, ALTER QUOTA, CREATE QUOTA, CREATE DATABASE query, SET query, Query, CREATE, WITH, CREATE ROLE or ALTER ROLE q uery, EXTERNAL DDL FROM, EXCHANGE TABLES, EXISTS or SHOW CREATE query, WATCH query, REPLACE, CREATE ROLE, CREATE SETTINGS PROFILE, SET ROLE or SET DEFAULT ROLE query, CREATE USE R or ALTER USER query, EXTERNAL DDL query, SHOW ACCESS query, SHOW CREATE QUOTA query, USE query, ATTACH, DESCRIBE, ALTER TABLE, ShowAccessEntitiesQuery, GRANT or REVOKE query, CREATE TABLE or ATTACH TABLE query (version 21.6.9.7 (official build)) [ERROR] [Failed SQL: (62) UPDATE DATABASECHANGELOG SET TAG = '1.0.0' WH ERE DATEEXECUTED = (SELECT MAX(DATEEXECUTED) FROM DATABASECHANGELOG)] [ERROR] -> [Help 1]

some SQL command dont work to generateChangeLog

Hello,

I tried to build a maven project to manage my clikchouse database model.
Using apache-maven 3.6.3
and Clickhouse 21.6.9

Here is my property file:

outputChangeLogFile=src/main/resources/liquibase-outputChangeLog.xml
url=jdbc:clickhouse://test:8123/default
username=xxx
password=yyy
driver=ru.yandex.clickhouse.ClickHouseDriver
databaseClass=liquibase.ext.clickhouse.database.ClickHouseDatabase
strict=true

And pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>DataModel</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
    </properties>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-release-plugin</artifactId>
                    <version>2.5.3</version>
                </plugin>
                <plugin>
                    <groupId>org.liquibase</groupId>
                    <artifactId>liquibase-maven-plugin</artifactId>
                    <version>4.3.5</version>
                    <configuration>
                        <propertyFile>src/main/resources/liquibase.properties</propertyFile>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>com.mediarithmics</groupId>
                            <artifactId>liquibase-clickhouse</artifactId>
                            <version>0.6.1</version>
                        </dependency>
                    </dependencies>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

when executing the following command
mvn liquibase:generateChangeLog

I have the following error

[INFO] Parsing Liquibase Properties File src/main/resources/liquibase.properties for changeLog parameters
[INFO] Executing on Database: jdbc:clickhouse://iqoprdweb057:8123/default
[INFO] there are no resolved artifacts for the Maven project.
[INFO] Generating Change Log from database null @ jdbc:clickhouse://iqoprdweb057:8123/default (Default Schema: default)
BEST PRACTICE: The changelog generated by diffChangeLog/generateChangeLog should be inspected for correctness and completeness before being deployed.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.351 s
[INFO] Finished at: 2021-11-23T16:27:42+01:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "unittest" could not be activated because it does not exist.
[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:4.3.5:generateChangeLog (default-cli) on project DataModel:
[ERROR] Error setting up or running Liquibase:
[ERROR] liquibase.command.CommandExecutionException: liquibase.exception.DatabaseException: Error getting jdbc:clickhouse://iqoprdweb057:8123/default view with liquibase.statement.core.GetViewDefinitionStatement@740dcae3: Error executing SQL select view_definition from information_schema.views where table_name='LiquidityMetricsFacet' and table_catalog='default': ClickHouse exception, code: 81, host: iqoprdweb057, port: 8123; Code: 81, e.displayText() = DB::Exception: Database information_schema doesn't exist (version
 21.6.9.7 (official build))
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Ma          ven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

make new release

Hello everyone, a month ago I have made the PR which was merged #34
For as it's a critical fix, and we would like to get released version with this fix if it possible, otherwise we should to solve a lot of non-technical troubles to avoid the bug, so we would be really appriciate if you could assist and make the release.
@igorepim is it possible?

clickhouse-jdbc and liquibase-clickhouse --> Transactions are not supported

Hi guys,

We have a very basic setup of clickhouse and liquibase-clickhouse but it doenst work with the official Clickhouse Driver.
The official driver always gives error:

liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.sql.SQLFeatureNotSupportedException: Transactions are not supported
	at liquibase.database.AbstractJdbcDatabase.setAutoCommit(AbstractJdbcDatabase.java:1234)
	at liquibase.changelog.ChangeSet.execute(ChangeSet.java:704)
	at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:49)
	at liquibase.changelog.ChangeLogIterator$2.lambda$null$0(ChangeLogIterator.java:111)
	at liquibase.Scope.lambda$child$0(Scope.java:177)

SpringLiquibase config:

    @Bean(name = CLICKHOUSE_DATASOURCE)
    public DataSource createClickhouseDataSource() {
        ClickHouseProperties properties = new ClickHouseProperties();
        properties.setPassword(yyy);
        properties.setUser(xxx);
       ...
        return new ClickHouseDataSource(zzz, properties);
    }

  @Primary
    @Lazy(false)
    @Bean(name = CLICKHOUSE_LIQUIBASE_BEAN_NAME)
    @DependsOn(CLICKHOUSE_DATASOURCE)
    public SpringLiquibase liquibaseClickhouse(@Qualifier(CLICKHOUSE_DATASOURCE) DataSource dataSource) {
        // Configure custom Clickhouse Liquibase settings
        ...
        DatabaseFactory.getInstance().register(new ClickHouseDatabase());

        SpringLiquibase liquibase = new SpringLiquibase();
        liquibase.setDataSource(dataSource);
        ....
        return liquibase;
    }

The official connection (ClickHouseConnectionImpl) does the following:

    @Override
    public void setAutoCommit(boolean autoCommit) throws SQLException {
        if (autoCommit) {
            return;
        }
        throw new SQLFeatureNotSupportedException("Transactions are not supported");
    }

I already tried to set runInTransaction="false" inside liquibase xmls but this doesnt change anything.

Version info:
clickhouse-jdbc: 0.3.1-patch --> https://github.com/ClickHouse/clickhouse-jdbc
clickhouse-liquibase: 0.6.1

LockException when using cluster mode

When running using cluster mode and clickhouse-jdbc 0.3.2-patch11 I consistently see the following exception in the log:

2022-12-05 16:51:31.847 ERROR 68398 --- [           main] liquibase                                : Could not release lock

liquibase.exception.LockException: liquibase.exception.LockException: Did not update change log lock correctly.

-1 rows were updated instead of the expected 1 row using executor liquibase.executor.jvm.JdbcExecutor there are 1 rows in the table
	at liquibase.lockservice.StandardLockService.releaseLock(StandardLockService.java:394) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Liquibase.lambda$update$1(Liquibase.java:306) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Scope.lambda$child$0(Scope.java:180) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Scope.child(Scope.java:189) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Scope.child(Scope.java:179) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Scope.child(Scope.java:158) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Liquibase.runInScope(Liquibase.java:2447) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Liquibase.update(Liquibase.java:236) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.Liquibase.update(Liquibase.java:221) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:328) ~[liquibase-core-4.17.1.jar:na]
	at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:283) ~[liquibase-core-4.17.1.jar:na]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.23.jar:5.3.23]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.23.jar:5.3.23]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.23.jar:5.3.23]
	at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66) ~[spring-boot-2.7.5.jar:2.7.5]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734) ~[spring-boot-2.7.5.jar:2.7.5]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.5.jar:2.7.5]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-2.7.5.jar:2.7.5]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) ~[spring-boot-2.7.5.jar:2.7.5]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) ~[spring-boot-2.7.5.jar:2.7.5]
	...
Caused by: liquibase.exception.LockException: Did not update change log lock correctly.

I suspect that this is due to the ALTER TABLE UPDATE mutation is running asynchronously. In order for the query to return the number of affected rows the mutations_sync setting needs to be 1 or 2: https://clickhouse.com/docs/en/operations/settings/settings/#mutations_sync

I think that the best solution would be to add this setting in the SQL query. Will soon prepare a PR.

clickhouse-jdbc-driver upgrade and resolve Transactions are not supported error

changes

replace clickhouse-jdbc-driver from ru.yandex.clickhouse(0.3.1) to com.clickhouse(version 0.3.2)
liquibase version from 4.6.1 to 4.15.0
testcontainer from 1.15.3 to 1.17.3
resolve Transactions are not supported error
Add mutations_sync setting to ALTER statements

Transactions are not supported

Hello everyone.
I'm trying to use your library on a sprig boot(2.5.7) application. This error happens

error:

[app] Caused by: liquibase.exception.MigrationFailedException: Migration failed for change set db/changesets/1/createTables-clickhouse.yml::create-database::1:
[app]      Reason: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.sql.SQLFeatureNotSupportedException: **Transactions are not supported**

It looks like your extension is not being picked up.

application.yml

spring:
  liquibase:
    url: "jdbc:clickhouse://${ip}:${port}"
    driverClassName: "ru.yandex.clickhouse.ClickHouseDriver"
    change-log: classpath:db/liquibase-click-changelog.yml

liquibase-click-changelog.yml

databaseChangeLog:
  - include:
      file: changesets/1/createTables-clickhouse.yml
      relativeToChangelogFile: true

createTables-clickhouse.yml

databaseChangeLog:
  - changeSet:
      id: 1
      author: 1
      changes:
        - sql: create database if not exist dbname;

gradle:

    implementation 'org.liquibase:liquibase-core:4.3.5'
    implementation group: 'com.mediarithmics', name: 'liquibase-clickhouse', version: '0.7.2'

All queries executes to 'default' database

I ran into a problem, there is a database called "dbname" and there is a sql changelog

CREATE TABLE ping (
    ID UInt8,
    `data.message` String,
    created_at DateTime('Europe/Moscow')
) ENGINE MergeTree() ORDER BY ID

properties.file

changeLogFile=changelog.xml
classpath=changelog/clickhouse
url=jdbc:clickhouse://clickhouse:8123/dbname
username=default
password=1212
driver=shaded.liquibase.ru.yandex.clickhouse.ClickHouseDriver
databaseClass=liquibase.ext.clickhouse.database.ClickHouseDatabase
strict=false

Migration result:

dbname table list: DATABASECHANGELOG, DATABASECHANGELOGLOCK
default db table list: ping

Whatever I write the name of the database, requests still go to "default". I tried to make another user, with 1 of my database, who does not know about the existence of the "default" database, I still get the system tables in its database, and an error when creating a query to a non-existent database

Altinity Clickhouse migration, Unexpected error running Liquibase

Hello,

afte clearing up md5 checksum of migration, we get:
Unexpected error running Liquibase: 'liquibase.change.CheckSum liquibase.changelog.ChangeSet.generateCheckSum()'

Bug: syntax error: failed at position 35 ('-')

if clickhouse db/scheme contains special characte, in our case it is '-', then it fails.

Caused by: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: 127.0.0.1, port: 8123; Code: 62. DB::Exception: Syntax error: failed at position 35 ('-'): -db.DATABASECHANGELOGLOCK ON CLUSTER '{cluster}' (ID Int64,LOCKED UInt8,LOCKGRANTED Nullable(DateTime64),LOCKEDBY Nullable(String)) ENGINE ReplicatedMergeTree(. Expected one of: token, Dot, UUID, ON, OpeningRoundBracket, storage definition, ENGINE, PARTITION BY, PRIMARY KEY, ORDER BY, SAMPLE BY, TTL, SETTINGS, AS, COMMENT, INTO OUTFILE, FORMAT, end of query. (SYNTAX_ERROR) (version 22.2.2.1)

No suitable ClickHouse client(out of 0) found in classpath.

Hello guys,

I'm really trying hard to make it work, but I will need a bit of help if you can.

I'm using the master version of liquibase-clickhouse and the shaded jar
Here is my liquibase.properties :

changeLogFile: /liquibase/changelog/root-changelog.xml
url: jdbc:clickhouse://my_hosted_clickhouse:8123
username: admin_rw
password: *************
classpath: /liquibase/changelog
driver: shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver
databaseClass: liquibase.ext.clickhouse.database.ClickHouseDatabase
strict: true

And I run liquibase on docker versio 4.15 like in the pom file:

 docker run -it --rm -v $(pwd)/liquibase/changelog:/liquibase/changelog -v$(pwd)/liquibase-clickhouse/target/liquibase-clickhouse-0.7.3-SNAPSHOT-shaded.jar:/liquibase/lib/clickhouse.jar liquibase/liquibase:4.15 --defaults-file=/liquibase/changelog/liquibase.properties --log-level=debug update

But I get the following error :

[2023-05-18 20:02:29] FINE [liquibase.database] Properties:
[2023-05-18 20:02:29] FINE [liquibase.database] Key:'user' Value:'admin_rw'
[2023-05-18 20:02:29] FINE [liquibase.database] Connecting to the URL:'jdbc:clickhouse://*************:8123' using driver:'shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver'
[2023-05-18 20:02:29] FINE [liquibase.servicelocator] Loaded liquibase.database.DatabaseConnection instance liquibase.database.jvm.JdbcConnection
[2023-05-18 20:02:29] FINE [liquibase.servicelocator] Loaded liquibase.database.DatabaseConnection instance com.datical.liquibase.ext.database.jvm.ProJdbcConnection
[2023-05-18 20:02:29] SEVERE [liquibase.integration] No suitable ClickHouse client(out of 0) found in classpath.
liquibase.exception.CommandExecutionException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.command.CommandScope.execute(CommandScope.java:179)
	at liquibase.integration.commandline.CommandRunner.call(CommandRunner.java:51)
	at liquibase.integration.commandline.CommandRunner.call(CommandRunner.java:21)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2314)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at liquibase.integration.commandline.LiquibaseCommandLine.lambda$execute$1(LiquibaseCommandLine.java:353)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:165)
	at liquibase.integration.commandline.LiquibaseCommandLine.execute(LiquibaseCommandLine.java:318)
	at liquibase.integration.commandline.LiquibaseCommandLine.main(LiquibaseCommandLine.java:84)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at liquibase.integration.commandline.LiquibaseLauncher.main(LiquibaseLauncher.java:107)
Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:139)
	at liquibase.integration.commandline.Main.doMigration(Main.java:1485)
	at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:397)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.integration.commandline.Main$1.run(Main.java:396)
	at liquibase.integration.commandline.Main$1.run(Main.java:218)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:165)
	at liquibase.integration.commandline.Main.run(Main.java:218)
	at liquibase.command.AbstractCliWrapperCommandStep.run(AbstractCliWrapperCommandStep.java:33)
	at liquibase.command.CommandScope.execute(CommandScope.java:173)
	... 20 more
Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:218)
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:176)
	at liquibase.database.DatabaseFactory.openDatabase(DatabaseFactory.java:141)
	at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:93)
	... 33 more
Caused by: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.database.ConnectionServiceFactory.create(ConnectionServiceFactory.java:35)
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:215)
	... 36 more
Caused by: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at shaded.liquibase.com.clickhouse.client.ClickHouseClientBuilder.build(ClickHouseClientBuilder.java:103)
	at shaded.liquibase.com.clickhouse.jdbc.internal.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:185)
	at shaded.liquibase.com.clickhouse.jdbc.internal.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:168)
	at shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver.connect(ClickHouseDriver.java:145)
	at shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver.connect(ClickHouseDriver.java:41)
	at liquibase.database.jvm.JdbcConnection.open(JdbcConnection.java:83)
	at com.datical.liquibase.ext.database.jvm.ProJdbcConnection.open(Unknown Source)
	at liquibase.database.ConnectionServiceFactory.create(ConnectionServiceFactory.java:32)
	... 37 more


Unexpected error running Liquibase: No suitable ClickHouse client(out of 0) found in classpath.

liquibase.exception.CommandExecutionException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.command.CommandScope.execute(CommandScope.java:179)
	at liquibase.integration.commandline.CommandRunner.call(CommandRunner.java:51)
	at liquibase.integration.commandline.CommandRunner.call(CommandRunner.java:21)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2314)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at liquibase.integration.commandline.LiquibaseCommandLine.lambda$execute$1(LiquibaseCommandLine.java:353)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:165)
	at liquibase.integration.commandline.LiquibaseCommandLine.execute(LiquibaseCommandLine.java:318)
	at liquibase.integration.commandline.LiquibaseCommandLine.main(LiquibaseCommandLine.java:84)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at liquibase.integration.commandline.LiquibaseLauncher.main(LiquibaseLauncher.java:107)
Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:139)
	at liquibase.integration.commandline.Main.doMigration(Main.java:1485)
	at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:397)
	at liquibase.Scope.lambda$child$0(Scope.java:180)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:179)
	at liquibase.Scope.child(Scope.java:158)
	at liquibase.integration.commandline.Main$1.run(Main.java:396)
	at liquibase.integration.commandline.Main$1.run(Main.java:218)
	at liquibase.Scope.child(Scope.java:189)
	at liquibase.Scope.child(Scope.java:165)
	at liquibase.integration.commandline.Main.run(Main.java:218)
	at liquibase.command.AbstractCliWrapperCommandStep.run(AbstractCliWrapperCommandStep.java:33)
	at liquibase.command.CommandScope.execute(CommandScope.java:173)
	... 20 more
Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:218)
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:176)
	at liquibase.database.DatabaseFactory.openDatabase(DatabaseFactory.java:141)
	at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:93)
	... 33 more
Caused by: liquibase.exception.DatabaseException: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at liquibase.database.ConnectionServiceFactory.create(ConnectionServiceFactory.java:35)
	at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:215)
	... 36 more
Caused by: java.lang.IllegalStateException: No suitable ClickHouse client(out of 0) found in classpath.
	at shaded.liquibase.com.clickhouse.client.ClickHouseClientBuilder.build(ClickHouseClientBuilder.java:103)
	at shaded.liquibase.com.clickhouse.jdbc.internal.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:185)
	at shaded.liquibase.com.clickhouse.jdbc.internal.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:168)
	at shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver.connect(ClickHouseDriver.java:145)
	at shaded.liquibase.com.clickhouse.jdbc.ClickHouseDriver.connect(ClickHouseDriver.java:41)
	at liquibase.database.jvm.JdbcConnection.open(JdbcConnection.java:83)
	at com.datical.liquibase.ext.database.jvm.ProJdbcConnection.open(Unknown Source)
	at liquibase.database.ConnectionServiceFactory.create(ConnectionServiceFactory.java:32)
	... 37 more

How to use project with Liquibase?

I'm currently learning about Clickhouse and I am trying to get Liquibase to play nicely with Clickhouse with the simplest possible set-up, but without success. I'm assuming that this project is a Liquibase extension for Clickhouse, meaning I can run Liquibase commands against Clickhouse - is this the case?

I've started Clickhouse using Docker:

docker run --ulimit nofile=262144:262144 --volume=$(pwd)/clickhouse_data:/var/lib/clickhouse -p 8123:8123 yandex/clickhouse-server
I'm now wanting to run Liquibase against Clickhouse, so I also run that in Docker:

docker run -it --rm -v $(pwd)/external-resources/clickhouse/liquibase/:/liquibase/changelog liquibase/liquibase:4.2.2 --defaultsFile=/liquibase/changelog/liquibase.docker.properties --logLevel=debug update
My liquibase properties file looks like this:

changeLogFile: /liquibase/changelog/root-changelog.xml
url: jdbc:clickhouse://192.168.0.16:8123
username: default
password:
classpath: /liquibase/changelog/drivers/liquibase-clickhouse-0.5.3-SNAPSHOT-shaded.jar
driver: shaded.liquibase.ru.yandex.clickhouse.ClickHouseDriver

I've built a shaded version of your jar so that all dependencies are in it and added it to the classpath for Liquibase. However, I am currently seeing an exception which is causing the process to fail:

SEVERE [liquibase.integration] Unexpected error running Liquibase: 'liquibase.executor.ExecutorService'
liquibase.executor.ExecutorService.getInstance()'
java.lang.NoSuchMethodError: 'liquibase.executor.ExecutorService liquibase.executor.ExecutorService.getInstance()'
at liquibase.ext.clickhouse.lockservice.ClickHouseLockService.getExecutor(ClickHouseLockService.java:89)
at liquibase.ext.clickhouse.lockservice.ClickHouseLockService.hasDatabaseChangeLogLockTable(ClickHouseLockService.java:78)
at liquibase.lockservice.StandardLockService.init(StandardLockService.java:98)
at liquibase.lockservice.StandardLockService.acquireLock(StandardLockService.java:251)
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:212)
at liquibase.Liquibase.lambda$update$1(Liquibase.java:218)
at liquibase.Scope.lambda$child$0(Scope.java:160)
at liquibase.Scope.child(Scope.java:169)
at liquibase.Scope.child(Scope.java:159)
at liquibase.Scope.child(Scope.java:138)
at liquibase.Liquibase.runInScope(Liquibase.java:2277)
at liquibase.Liquibase.update(Liquibase.java:215)
at liquibase.Liquibase.update(Liquibase.java:201)
at liquibase.integration.commandline.Main.doMigration(Main.java:1761)
at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:361)
at liquibase.Scope.lambda$child$0(Scope.java:160)
at liquibase.Scope.child(Scope.java:169)
at liquibase.Scope.child(Scope.java:159)
at liquibase.Scope.child(Scope.java:138)
at liquibase.Scope.child(Scope.java:222)
at liquibase.Scope.child(Scope.java:226)
at liquibase.integration.commandline.Main$1.run(Main.java:360)
at liquibase.integration.commandline.Main$1.run(Main.java:193)
at liquibase.Scope.child(Scope.java:169)
at liquibase.Scope.child(Scope.java:145)
at liquibase.integration.commandline.Main.run(Main.java:193)
at liquibase.integration.commandline.Main.main(Main.java:156)

Is this an error in the project? Or am I supposed to use this project in a different way please?

Strange error on second update command

Hello everyone! Such a question set up migrations and completed them successfully. Next, I added a new sql file with a new table, I try to start the update and I get this error.

Caused by: java.sql.SQLException: no column shaded.liquibase.comment in columns list database table name type default_type default_expression comment
        at shaded.liquibase.ru.yandex.clickhouse.response.ClickHouseResultSet.findColumn(ClickHouseResultSet.java:726)
        at shaded.liquibase.ru.yandex.clickhouse.response.ClickHouseResultSet.getString(ClickHouseResultSet.java:291)
        at shaded.liquibase.ru.yandex.clickhouse.ClickHouseDatabaseMetadata.getColumns(ClickHouseDatabaseMetadata.java:860)
        at liquibase.snapshot.JdbcDatabaseSnapshot$CachingDatabaseMetaData$GetColumnResultSetCache.fastFetchQuery(JdbcDatabaseSnapshot.java:370)
        at liquibase.snapshot.ResultSetCache$SingleResultSetExtractor.fastFetch(ResultSetCache.java:367)
        at liquibase.snapshot.ResultSetCache.get(ResultSetCache.java:72)

To be honest, I don't even know where to look.

I`m using liquibase as docker image

docker run -it --rm \
  --network=clickhouse-network\
  -v $(pwd)/liquibase/changelog:/liquibase/changelog\
  -v $(pwd)/liquibase/liquibase-clickhouse/target/liquibase-clickhouse-0.7.0-SNAPSHOT-shaded.jar:/liquibase/lib/clickhouse.jar\
  liquibase/liquibase:4.6.1\
  --defaultsFile=/liquibase/changelog/liquibase.properties\
  update

With that config:

changeLogFile=root-changelog.xml
classpath=/liquibase/changelog
url=jdbc:clickhouse://clickhouse01:8123
username=default
password=
driver=shaded.liquibase.ru.yandex.clickhouse.ClickHouseDriver
databaseClass=liquibase.ext.clickhouse.database.ClickHouseDatabase
liquibase.hub.mode=off
liquibase.logLevel=warning

and my root-changelog is

<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
                      http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.6.xsd">
    <includeAll path="sql/v0.1.0"/>
    <includeAll path="sql/v0.2.0"/>
</databaseChangeLog>

mediarithmics / liquibase-clickhouse Goto Github PK

liquibase-clickhouse's Introduction

liquibase-clickhouse

Important changes

liquibase-clickhouse's People

Contributors

Stargazers

Watchers

Forkers

liquibase-clickhouse's Issues

changes

Recommend Projects

Recommend Topics

Recommend Org