patka / cassandra-migration Goto Github PK

Currently the code that does isLineComment() uses the following regex to parse line comments:

This regex will support line comments that start at the beginning of the line but will fail for comments like below.

CREATE TABLE users (
  // Some comment that does not start a the beginning of the line
  userid text PRIMARY KEY,
// This comment start at the beginning of the line and is supported
  first_name text,
 );

Can we change the regex to this: "(^\s*--.*)|(^\s*//.*)" to support CQL files like above?

is it possible to support the script version in format like 1.1,1.2?
so we can create separate cql file for each table.

I set script location to = "db/migration/cassandra" and inside this folder i have another folder named "init" (I want to separate the scripts to sub folders for each version).

When i run the migration i get exception:
Exception in thread "main" org.cognitor.cassandra.migration.MigrationException: Error for script init/100000000_dbCreation.cql. Unable to extract version. at org.cognitor.cassandra.migration.MigrationRepository.extractScriptVersion(MigrationRepository.java:203) at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:185) at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:132) at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:114) at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:102) at com.solaredge.dbc.cassndra.migration.CassandraMigrator.migrate(CassandraMigrator.java:74) at com.solaredge.dbc.cassndra.migration.CassandraMigrator.main(CassandraMigrator.java:64) Caused by: java.lang.NumberFormatException: For input string: "init/100000000" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.cognitor.cassandra.migration.MigrationRepository.extractScriptVersion(MigrationRepository.java:201) ... 6 more

After a quick debugging, the function 'extractScriptName' return the scriptName only without the script location that i set in the beginning (db/migration/cassandra) and return the sub folder with the script name (it returned 'init/100000000_dbCreation.cql' instead of '100000000_dbCreation.cql').

I changed the function extractScriptName to :
private String extractScriptName(String resourceName) { return resourceName.substring(resourceName.lastIndexOf("/")); }
and its work fine.

Hi,

Are there any plans for the library to support spring boot 2+? the current spring boot version being used is 1.5.21.RELEASE

Hi. I want to use the new feature, where I can set ReplicationStrategy in the spring starter. Any chance a new release is coming soon?

Maybe you have some sort of Baseline that helps to use this tool on an existing database. Like Flyway baseline.

Elassandra contains both Cassandra and ElasticSearch databases, so they both require migrations. The current project is the closest one capable to provide both types of migrations.

For that purpose I suggest to use files with extensions esrest.json with content:

{
  "url": "http://${cassandra-host}:${elastic-port}",
  "method": "PUT",
  "path": "/${cassandra-keyspace}",
  "payload": {
    "settings": {
      "index": {
        "drop_on_delete_index": false
      }
    },
    "mappings": {
      "enforcement": {
        "properties": {
          "name": {
            "type": "text",
            "cql_collection": "singleton"
          },
        }
      }
    }
  }
}

Does it support Java based migrations just like flyway?

Hi!

I'm interested in the following scenario. We deploy our app in k8s which spawns multiple versions of it. Every time the app starts, it runs its migrations. So I wonder if the library provides some way to prevent concurrent migrations?

Because in JDBC world, even though each migration is run inside a transaction, they either lock the migration table for reading or insert a temporary flag in the database meaning that somebody is running the migration process at the moment. As a result, only one node or Pod succeeds in running their migrations.

Does the library provide any kind of locking or a temporary flag maybe? Or did I miss something?

Ivan

Hey @patka! I saw you released 2.5.0_v4 with the latest execution profile updates. I'm not seeing this version available from maven. Could you publish the new version? Thanks!

https://mvnrepository.com/artifact/org.cognitor.cassandra/cassandra-migration

How Avoid Cassandra db exception during migration?
for eg:
@table(value = "Student ")
@DaTa
public class Student {
private String id;
private String name;
}

in my application using spring-data-cassandra, so when i will start application, spring will create table during server start-up.
now i want to add age column in Student table
@table(value = "Student ")
@DaTa
public class Student {
private String id;
private String name;
private Integer age;
}
when i will start application, spring will not create table during server start-up. because student table is already exist.
so here i using cassandra migration and have written cql script file with ALTER TABLE student add age int;
so it is working fine for existing db.
if i'm using same cql script file for new fresh db then giving error like
Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Invalid column name age because it conflicts with an existing column
Caused by: org.cognitor.cassandra.migration.MigrationException: Error during migration of script 1_test.cql while executing 'ALTER TABLE student add age int;'
at org.cognitor.cassandra.migration.Database.execute(Database.java:187) ~[cassandra-migration-2.2.0.jar:?]
at java.util.ArrayList.forEach(ArrayList.java:1257) ~[?:1.8.0_181]
at org.cognitor.cassandra.migration.MigrationTask.migrate(MigrationTask.java:52) ~[cassandra-migration-2.2.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_181]

this error is coming because spring is already added the age column in student table because it is a fresh new db and student table doesn't exist.

so is there any way to skip this type of errors?

HI,
I have used Cassandra- migration in one of my project.
i have facing issue with the migration script when i deployed it in aws-server.
in my local its able to find but facing issue in dev-server.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in class path resource [com/rishabh/saas/audit/config/AuditLogConsumerConfig.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.datastax.oss.driver.api.core.CqlSession]: Factory method 'cassandraSession' threw exception; nested exception is org.cognitor.cassandra.migration.MigrationException: Error while scanning script folder for new scripts.
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:658)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:638)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:64)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:745)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:420)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
at com.rishabh.saas.audit.Application.main(Application.java:14)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.datastax.oss.driver.api.core.CqlSession]: Factory method 'cassandraSession' threw exception; nested exception is org.cognitor.cassandra.migration.MigrationException: Error while scanning script folder for new scripts.
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 27 common frames omitted
Caused by: org.cognitor.cassandra.migration.MigrationException: Error while scanning script folder for new scripts.
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:137)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:118)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:106)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:93)
at com.rishabh.saas.audit.config.AuditLogConsumerConfig.cassandraMigrationCqlSession(AuditLogConsumerConfig.java:55)
at com.rishabh.saas.audit.config.AuditLogConsumerConfig.cassandraSession(AuditLogConsumerConfig.java:64)
at com.rishabh.saas.audit.config.AuditLogConsumerConfig$$EnhancerBySpringCGLIB$$4303e232.CGLIB$cassandraSession$2()
at com.rishabh.saas.audit.config.AuditLogConsumerConfig$$EnhancerBySpringCGLIB$$4303e232$$FastClassBySpringCGLIB$$b8203b9b.invoke()
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
at com.rishabh.saas.audit.config.AuditLogConsumerConfig$$EnhancerBySpringCGLIB$$4303e232.cassandraSession()
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 28 common frames omitted
Caused by: java.nio.file.NoSuchFileException: cassandra/migration
at jdk.zipfs/jdk.nio.zipfs.ZipPath.getAttributes(Unknown Source)
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.readAttributes(Unknown Source)
at java.base/java.nio.file.Files.readAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.getAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.visit(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.walk(Unknown Source)
at java.base/java.nio.file.FileTreeIterator.(Unknown Source)
at java.base/java.nio.file.Files.walk(Unknown Source)
at java.base/java.nio.file.Files.walk(Unknown Source)
at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:34)
at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:189)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:135)

Hello,

We are using version 2.1.2:

compile('org.cognitor.cassandra:cassandra-migration-spring-boot-starter:2.1.2')

Until last Friday migrator used to work and pickup cql files from cassandra/migration folder. This week it started treat cassandra/migration as if it exist in a zipped folder. Notice classes**!**/cassandra/migration/

PS: We are using the default migration folder

MigrationTask migration = new MigrationTask(database, new MigrationRepository());
migration.migrate();

below is the exception that we getting from the application:

17:38:10.783 [main] DEBUG org.cognitor.cassandra.migration.MigrationRepository - Scanning for cql migration scripts in cassandra/migration/
17:38:10.784 [main] DEBUG org.cognitor.cassandra.migration.MigrationRepository - Potential script folder: jar:file:/Users/myuser/Documents/sourcecode/myservice/build/libs/myservice-1.0.jar!/BOOT-INF/classes!/cassandra/migration/
17:38:10.784 [main] DEBUG org.cognitor.cassandra.migration.scanner.JarLocationScanner - Scanning in jar jar:file:/Users/myuser/Documents/sourcecode/myservice/build/libs/myservice-1.0.jar!/BOOT-INF/classes!/cassandra/migration/ in location cassandra/migration/
17:38:10.784 [main] DEBUG org.cognitor.cassandra.migration.scanner.JarLocationScanner - Trying to get existing filesystem for jar:file:/Users/myuser/Documents/sourcecode/myservice/build/libs/myservice-1.0.jar!/BOOT-INF/classes!/cassandra/migration/
17:38:10.789 [main] DEBUG org.cognitor.cassandra.migration.scanner.JarLocationScanner - Creating new filesystem for jar:file:/Users/myuser/Documents/sourcecode/myservice/build/libs/myservice-1.0.jar!/BOOT-INF/classes!/cassandra/migration/
Exception in thread "main" java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: org.cognitor.cassandra.migration.MigrationException: Error while scanning script folder for new scripts.
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:133)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:114)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:102)
at com.philips.ri.myservice.myserviceApplication.main(myserviceApplication.java:33)
... 8 more
Caused by: java.nio.file.NoSuchFileException: cassandra/migration
at jdk.zipfs/jdk.nio.zipfs.ZipPath.getAttributes(ZipPath.java:729)
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.readAttributes(ZipFileSystemProvider.java:293)
at java.base/java.nio.file.Files.readAttributes(Files.java:1755)
at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.base/java.nio.file.FileTreeIterator.(FileTreeIterator.java:71)
at java.base/java.nio.file.Files.walk(Files.java:3653)
at java.base/java.nio.file.Files.walk(Files.java:3707)
at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:34)
at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:181)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:131)
... 11 more

README describes that modifications are done by using QUORUM:

Migrations are executed with the Quorum consistency level to make sure that always a majority of nodes share the same schema information.

but it's incorrect - the schema changes are performed via different path than other operations, like, INSERT, etc. See https://stackoverflow.com/questions/41248615/set-consistency-level-for-cassandra-ddl for more information

to perform correct schema modifications you need to wait for schema agreement explicitly, see java driver doc, and this example.

I like to increase the timeouts for the migration tasks for my spring boot app.
The default timeout for my app is set to 2 seconds. I don't want to change that, as it will change the application performance.

But for the actual migration during startup, I don't mind a bigger timeout.
So it would be great to configure that specifically so that the startup does not fails.

Log message from the spring boot start:

Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'migrationTask' defined in class path resource [org/cognitor/cassandra/migration/spring/CassandraMigrationAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.cognitor.cassandra.migration.MigrationTask]: Factory method 'migrationTask' threw exception; nested exception is com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S

Cassandra does not support IF NOT EXISTS for ALTER table ADD ... queries, so it might be a good idea to be able to ignore failures in these queries.

Might be a global configuration where you specify the codes/parts of cassandra error message to be ignored.

The constant MIGRATION_TASK_BEAN_NAME appears to not be published in 2.5.0_v4.

Here is what I see with the 2.5.0_v4 dependency:

@Configuration
@EnableConfigurationProperties(CassandraMigrationConfigurationProperties.class)
@AutoConfigureAfter(CassandraAutoConfiguration.class)
@ConditionalOnClass(CqlSession.class)
public class CassandraMigrationAutoConfiguration {
    public static final String CQL_SESSION_BEAN_NAME = "cassandraMigrationCqlSession";
    private final CassandraMigrationConfigurationProperties properties;

    @Autowired
    public CassandraMigrationAutoConfiguration(CassandraMigrationConfigurationProperties properties) {
        this.properties = properties;
    }


    @Bean(initMethod = "migrate")
    @ConditionalOnBean(value = CqlSession.class)
    @ConditionalOnMissingBean(MigrationTask.class)
    public MigrationTask migrationTask(@Qualifier(CQL_SESSION_BEAN_NAME) CqlSession cqlSession) {

Here is what I see when I browse the GitHub code:

@Configuration
@EnableConfigurationProperties(CassandraMigrationConfigurationProperties.class)
@AutoConfigureAfter(CassandraAutoConfiguration.class)
@ConditionalOnClass(CqlSession.class)
public class CassandraMigrationAutoConfiguration {
    public static final String CQL_SESSION_BEAN_NAME = "cassandraMigrationCqlSession";
    public static final String MIGRATION_TASK_BEAN_NAME = "migrationTask";
    private final CassandraMigrationConfigurationProperties properties;

    @Autowired
    public CassandraMigrationAutoConfiguration(CassandraMigrationConfigurationProperties properties) {
        this.properties = properties;
    }

    @Bean(name = MIGRATION_TASK_BEAN_NAME, initMethod = "migrate")
    @ConditionalOnBean(value = CqlSession.class)
    @ConditionalOnMissingBean(MigrationTask.class)
    public MigrationTask migrationTask(@Qualifier(CQL_SESSION_BEAN_NAME) CqlSession cqlSession) {

More context around the log:

{"timestamp":"2023-04-28 17:01:13.476","level":"INFO","thread":"main","logger":"org.cognitor.cassandra.migration.Database","message":"Changing keyspace of the session to 'payment'","context":"default"}
{"timestamp":"2023-04-28 17:01:13.478","level":"WARN","thread":"s0-io-11","logger":"com.datastax.oss.driver.internal.core.session.PoolManager","message":"[s0] Detected a keyspace change at runtime (<none> => payment). This is an anti-pattern that should be avoided in production (see 'advanced.request.warn-if-set-keyspace' in the configuration).","context":"default"}
{"timestamp":"2023-04-28 17:01:13.623","level":"INFO","thread":"main","logger":"org.cognitor.cassandra.migration.MigrationRepository","message":"Found 1 migration scripts","context":"default"}

The application.conf docs here: https://java-driver.docs.scylladb.com/stable/manual/core/configuration/reference/README.html
Say:

 # Whether a warning is logged when a request (such as a CQL `USE ...`) changes the active
    # keyspace.
    # Switching keyspace at runtime is highly discouraged, because it is inherently unsafe (other
    # requests expecting the old keyspace might be running concurrently), and may cause statements
    # prepared before the change to fail.
    # It should only be done in very specific use cases where there is only a single client thread
    # executing synchronous queries (such as a cqlsh-like interpreter). In other cases, clients
    # should prefix table names in their queries instead.
    #
    # Note that CASSANDRA-10145 (scheduled for C* 4.0) will introduce a per-request keyspace option
    # as a workaround to this issue.
    #
    # Required: yes
    # Modifiable at runtime: yes, the new value will be used for keyspace switches occurring after
    #   the change.
    # Overridable in a profile: no
    warn-if-set-keyspace = true

The referenced per-request docs for Scylla (Cassandra compatible) are here
https://java-driver.docs.scylladb.com/stable/manual/core/statements/per_query_keyspace/

Is there a better way to get rid of this warning?

It would be great if the migration script commands would be executed in a way that the Cassandra RetryPolicy or any other retry method could be used.

Previously dateOf was deprecated.

WARN The function 'dateof' is deprecated. Use the function 'toTimestamp' instead.

Appears it was removed from 5.0 per changelog:

• Remove deprecated CQL functions dateOf and unixTimestampOf (CASSANDRA-18328)

I realized cassandra migration fails with the exception while playing with cassandra 5.0:

com.datastax.driver.core.exceptions.InvalidQueryException: Unknown function dateof called
at com.datastax.driver.core.Responses$Error.asException(Responses.java:181) ~[cassandra-driver-core-3.7.2.jar:na]
at com.datastax.driver.core.SessionManager$4.apply(SessionManager.java:250) ~[cassandra-driver-core-3.7.2.jar:na]
at com.datastax.driver.core.SessionManager$4.apply(SessionManager.java:219) ~[cassandra-driver-core-3.7.2.jar:na]
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1442) ~[guava-19.0.jar:na]
at com.google.common.util.concurrent.Futures$AsyncChainingFuture.doTransform(Futures.java:1433) ~[guava-19.0.jar:na]
at com.google.common.util.concurrent.Futures$AbstractChainingFuture.run(Futures.java:1408) ~[guava-19.0.jar:na]
at com.google.common.util.concurrent.Futures$2$1.run(Futures.java:1177) ~[guava-19.0.jar:na]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.0.56.Final.jar:4.0.56.Final]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

Makes sense to tweak the code to use toTimestamp as suggested in above warning. Per my observation the function is used for migration table timestamp columns.

Idea is to replace LGPL licensed parser with MIT licensed version from https://github.com/antlr/grammars-v4/blob/master/cql3/CqlParser.g4 (I contributed fixes, which allowed all tests to pass).

I created an experimental implementation here arixmkii@4c50310 It passes all tests with this parser (but I had to extract embedded cassandra).

This is a significant change, because it will put a minimal JDK requirement to 11 for build and run. Also it will need to use test containers or something similar as embedded cassandra is troublesome with JDK 11. Antlr4 itself is not the most lightweight thing and could be unacceptable in some scenarios.

For me it looks like this will need to have major version bump and I imagine the plan to go there like this:

• bump major version;
• refactor the into 2 jar: cassandra-migration (implementation with old parser, mixed license) and cassandra-migration-core (without any parsers and only abstract class, MIT licensed);
• add new jar cassandra-migration-antlr4 (with antlr4 parser, MIT licensed);
• document how to introduce alternative parsers on demand;
• dropping _v4 suffix and remove support for cassandra v3 driver at all;
• no updates (except compilation fixes) or potentially dropping springboot, as springboot 3.0 is now the new mainstream.

Further additions for new major could be:

• introduction of CLI application to use it as standalone application (useful for running migration in initContainers in k8s like envs).

Actual version:
<dependency>
<groupId>org.cognitor.cassandra</groupId>
<artifactId>cassandra-migration-spring-boot-stater</artifactId>
<version>2.1.0</version>
</dependency>

good one:
<dependency>
<groupId>org.cognitor.cassandra</groupId>
<artifactId>cassandra-migration-spring-boot-starter</artifactId>
<version>2.1.1</version>
</dependency>

I recently configured the library in my Spring Boot project and have two questions:

1. Is Cassandra migration supposed to throw an exception on an invalid migration and not stop the app from running?
2. Other beans are initialized before ending the migration scripts, is it on purpose or my local problem?

Hi, I started using your library in this setup:

• Custom company wide library which encapsulates your library and ads Spring Boot auto configuration to it (similar to your Spring Boot integration)
• Application running in Kubernetes which uses the company library to make use of the Cassandra migration

During startup it logs this:
2020-10-19 08:53:19.234 DEBUG 1 --- [ main] o.c.c.migration.MigrationRepository : Scanning for cql migration scripts in cassandra/migration/ 2020-10-19 08:53:19.234 DEBUG 1 --- [ main] o.c.c.migration.MigrationRepository : Potential script folder: jar:file:/app/application.jar!/BOOT-INF/classes!/cassandra/migration/ 2020-10-19 08:53:19.235 DEBUG 1 --- [ main] o.c.c.m.scanner.JarLocationScanner : Scanning in jar jar:file:/app/application.jar!/BOOT-INF/classes!/cassandra/migration/ in location cassandra/migration/ 2020-10-19 08:53:19.235 DEBUG 1 --- [ main] o.c.c.m.scanner.JarLocationScanner : Trying to get existing filesystem for jar:file:/app/application.jar!/BOOT-INF/classes!/cassandra/migration/ 2020-10-19 08:53:19.244 DEBUG 1 --- [ main] o.c.c.m.scanner.JarLocationScanner : Creating new filesystem for jar:file:/app/application.jar!/BOOT-INF/classes!/cassandra/migration/

Eventually it fails with:
Caused by: java.nio.file.NoSuchFileException: cassandra/migration at jdk.zipfs/jdk.nio.zipfs.ZipPath.getAttributes(ZipPath.java:742) at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.readAttributes(ZipFileSystemProvider.java:292) at java.base/java.nio.file.Files.readAttributes(Files.java:1763) at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) at java.base/java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:71) at java.base/java.nio.file.Files.walk(Files.java:3822) at java.base/java.nio.file.Files.walk(Files.java:3876) at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:34) at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:189) at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:135) ... 72 common frames omitted

I checked the docker container and can see the folder there:
root@mycontainer-7cbd86969d-lkwd9:/app# jar -tf application.jar ... BOOT-INF/classes/cassandra/ BOOT-INF/classes/cassandra/migration/ BOOT-INF/classes/cassandra/migration/01_create-ad-table.cql

The permissions of the file are set to read by everyone, also I would expect another error if the permissions were there problem.
I'm really looking forward to get some help here.

Thanks
Dennis

Hi there, I.m using the spring starter and have 2 scripts. The first one is to create a table and the second one is alter it.

00002_createTable.cql (we already have a 00001 version script which is altering another table)

CREATE TABLE IF NOT EXISTS table_name
(
    field1       text,
    field2      text,

    PRIMARY KEY ((field1), field2)
);

00003_addExpireTimestamp.cql

alter table "table_name" add expire_timestamp timestamp;

When running the service, I got error saying

Caused by: org.cognitor.cassandra.migration.MigrationException: Error during migration of script 00003_addExpireTimestamp.cql while executing 'alter table cass_content_of_interests add expire_timestamp timestamp;'

... ...


Caused by: com.datastax.oss.driver.api.core.servererrors.InvalidQueryException: Table 'namespace.table_name' doesn't exist

Looks like the create table scrip is not executed somehow. Am I missing something here?

Hi Patrick,

I want the ability to know if a script that already processed (and in the schema_version table) was changed, and if so return failure. (like the checksum mechanism in flyway).

The migration task go over all the scripts, calculate the script checksum.
if the script is in schema_version table, it compare the calculated checksum to the checksum value in the schema_version table
if the script is not in schema_version table, migrate the script.

what do you think?
if you don't have time, i can help with the development

Thanks,
Shay

Hi,

Are there any plans on supporting the new Cassandra library java-driver-core?

In my application I have run across an issue loading the cql files. I am using jboss and a .war file and unfortunately jboss is doing some magic that prevents the java.nio.FileSystem directory scanning from working correctly. I would like to override the scannerFactory in the MigrationFactory to get around this problem.

Thoughts? How would you go about changing the api to allow for this?

Hi Patrick,

Currently scripts with duplicate versions are silently ignored. What do you think of a change that would force the migration to fail when there are duplicate version in the migration directory? We could keep current behaviour as the default and switch on the 'force failure on duplicate versions' functionality via a flag. The reason for this change is that I'm evaluating your lib for a fairly large project with many committers and I want them to get feedback on build issues asap.

Regards,
Jakob

I have my .cql scripts written as following:

CREATE KEYSPACE IF NOT exists tagservice
WITH replication = ...

During migration on the startup, following exception is thrown:

Caused by: com.datastax.driver.core.exceptions.SyntaxError: line 1:45 missing K_WITH at 'replication' (...KEYSPACE IF NOT exists tagserviceWITH [replication] =...)

It looks like the line breaks are not treated as a separating whitespace. If those are manually added at the beginning of the line (spaces or tabs), the migration is performed succesfully.

Amazon managed Cassandra service execute DDL statements asynchronously. Meaning that we receive errors like keyspace or table (schema_migration) does not exist because it takes some time to create them asynchronously.
https://docs.aws.amazon.com/keyspaces/latest/devguide/functional-differences.html#functional-differences.table-keyspace-management
There are some queries that can be performed to see when the tables are created.
Do you have any idea how support can be added to this async execution in this project? I think that it is too complicated to parse the migration cql files and recognize create table statements, extract table name and then query for table creation finalization.

I have a need where I want to run only certain migrations in a given environment. The way I got around this problem in the past was simply to have 2 different version tables (schema_migrations, schema_migrations_v2)
This still seems like the easiest path forward. Thoughts?

I would be cool if migration scripts supported some sort of subqueries. For cases where you don't know the ID(s) of some records, but you want to perform an update. That update would be slow, but it can be on tables with a limited number of records and so the penalty would not be that big.

I implemented something to support subqueries by extending your classes, but since DbMigration is package-private, I had to use reflection so it's a bit ugly. A query in my implementation would look like this:

 UPDATE some_table SET some_column=1000 WHERE id=${{SELECT id FROM some_table WHERE code='non-primary-key-value' ALLOW FILTERING}};

Then I have something like that:

private static final String SUBSTITUTE_REGEX = "\\$\\{\\{(.+)\\}\\}";
private static final Pattern SUBSTITUTE_PATTERN = Pattern.compile(SUBSTITUTE_REGEX);
void transformMigrations(List migrations) {
    try {
        // DbMigration is not public, so we use type-unsafe list + reflection
        Field field = ReflectionUtils.findField(Class.forName("org.cognitor.cassandra.migration.DbMigration"), "migrationScript");
        ReflectionUtils.makeAccessible(field);
        for (Object migration : migrations) {
            String script = (String) field.get(migration);
            if (!script.contains("${{")) {
                continue;
            }
            String[] lines = script.split(";");
            StringBuilder sb = new StringBuilder();
            for (String line : lines) {
                Matcher matcher = SUBSTITUTE_PATTERN.matcher(line);
                if (matcher.find()) {
                    logger.info("Transforming query {}", line);
                    String subquery = matcher.group(1);
                    List<Map<String, Object>> result = cqlOperations.query(subquery, new ColumnMapRowMapper());
                    // assuming a single column select
                    for (Map<String, Object> row : result) {
                        sb.append(line.replaceAll(SUBSTITUTE_REGEX, row.values().iterator().next().toString())).append(";");
                    }
                } else {
                    sb.append(line).append(";");
                }
            }
            String transformedMigrationScript = sb.toString();
            logger.info("Transformed migration script {}", transformedMigrationScript);
            ReflectionUtils.setField(field, migration, transformedMigrationScript);
        }
    } catch (Exception ex) {
        throw new RuntimeException(ex);
    }
}

I tried running the maven tests but keep getting this error:

DatabaseTest.shouldApplyMigrationToDatabaseWhenMigrationsAndEmptyDatabaseGiven:47 » NoHostAvailable

Full exception:

FAILURE! - in org.cognitor.cassandra.it.migration.DatabaseTest shouldApplyMigrationToDatabaseWhenMigrationsAndEmptyDatabaseGiven(org.cognitor.cassandra.it.migration.DatabaseTest) Time elapsed: 14.179 sec <<< ERROR! com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried) at org.cognitor.cassandra.it.migration.DatabaseTest.shouldApplyMigrationToDatabaseWhenMigrationsAndEmptyDatabaseGiven(DatabaseTest.java:47) Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried) at org.cognitor.cassandra.it.migration.DatabaseTest.shouldApplyMigrationToDatabaseWhenMigrationsAndEmptyDatabaseGiven(DatabaseTest.java:47)

I also get this warning a lot:

WARN Cluster:2105 - You listed localhost/0:0:0:0:0:0:0:1:9142 in your contact points, but it wasn't found in the control host's system.peers at startup

And also lots of infos like this one:

INFO Cluster:1475 - New Cassandra host localhost/127.0.0.1:9142 added

Apparently the tests are running ok but something is breaking at some point, I have not used CassandraUnit before and was wondering if you could give a helping hand.

If you need more details just ask.

Hi,

As you may have notice, com.datastax.cassandra dependency it's no longer developed as it is migrated to com.datastax.oss. The Cassandra 4 driver comes with several changes (in comparison to the 3.x driver).

Most of the features are similar but some concepts are changed (new driver drops the intermediate Cluster concept in favor of a single CqlSession entry point).

In this case, it is not possible to execute the following, as the expected type is a Cluster

Database database = new Database(cluster, "nameOfMyKeyspace");

In future version, this may be updated, allowing to pass the new "CqlSession" Class.

PD: Great tool to manage migrations, thanks.

Schema migration in practice frequently includes not only schema migration, but also data changes. For instance in case of creating of additional table to existing one for new select support, apart from creation of new table, we also need to fill it with data from primary table.

What if, hypothetically, tool could accept some implementations of interface with two methods:

• getVersion, which specs, after which schema migration script it should be executed
• apply, data migration execution scenario itself

Eventually, on script execution, tool will get data migration, assigned to script and invoke it and after that, migration is considered as executed successfully

Feature request is based on personal implementation of data migration, but it is executed after schema migration with it's own tables, containing migrations and leaders information. In practice, if you use spring data repositories in data migration code, you should support old migrations, as models change alongside with schema, which leads to compilation problems. It is easier to delete them later. Alternatively, you can operate with cqlTemplate, playing around with non-mapped result set, in this case you write code only once, but code itself is more complex. In any case, with such proposal, user is not forced to some specific approach, it is up to him to decide, what and how to operate on schema migration #X .

I'm using cassandra-migration in a Ratpack application. The application is executed as a uber-jar just using java -jar myapplication.jar. When boostrapping the application the cassandra migration is executed causing the following exception:

Caused by: java.nio.file.FileSystemAlreadyExistsException
        at com.sun.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:113)
        at java.nio.file.FileSystems.newFileSystem(FileSystems.java:326)
        at java.nio.file.FileSystems.newFileSystem(FileSystems.java:276)
        at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:35)
        at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:164)
        at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:118)
        at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:101)
        at org.cognitor.cassandra.migration.MigrationRepository.<init>(MigrationRepository.java:88)

Apparently googling a little bit, sometimes it may be necessary to add a check to make sure the file system has not been created already.

I've cloned the code and did a local snapshot changing the following at JarLocationScanner:

    @Override
    public Set<String> findResourceNames(String location, URI locationUri) throws IOException {
        notNullOrEmpty(location, "location");
        notNull(locationUri, "locationUri");
        LOGGER.debug("Scanning in jar {} in location {}", locationUri, location);
        try(FileSystem fileSystem = getFileSystem(locationUri)) {
            final Path systemPath = fileSystem.getPath(location);
            return Files.walk(systemPath)
                    .filter(Files::isRegularFile)
                    .map(path -> normalizePath(path.toString()))
                    .collect(toSet());
        }
    }

    private FileSystem getFileSystem(URI locationUri) throws IOException {
        FileSystem fileSystem = FileSystems.getFileSystem(locationUri);
        if (fileSystem != null) {
            return fileSystem;
        }
        return FileSystems.newFileSystem(locationUri, emptyMap());
    }

And it fixes the problem.

We plan to use cassandra migrations in different packages. Right now this tool does not allow that due to the following:

1. This package has no basePackage option. Here in MigrationRepository

 Enumeration<URL> scriptResources = getClass().getClassLoader().getResources(scriptPath);

while in flyway

    @Override
    public List<ResolvedMigration> resolveMigrations() {
        List<ResolvedMigration> migrations = new ArrayList<>();

        for (Location location : locations) {
            if (!location.isClassPath()) {
                continue;
            }
            resolveMigrationsForSingleLocation(location, migrations);
        }

        Collections.sort(migrations, new ResolvedMigrationComparator());
        return migrations;
    }

    private void resolveMigrationsForSingleLocation(Location location, List<ResolvedMigration> migrations) {
        try {
            Class<?>[] classes = scanner.scanForClasses(location, getImplementedInterface());
            for (Class<?> clazz : classes) {
                M migration = ClassUtils.instantiate(clazz.getName(), scanner.getClassLoader());
                ConfigUtils.injectFlywayConfiguration(migration, configuration);

                ResolvedMigrationImpl migrationInfo = extractMigrationInfo(migration);
                migrationInfo.setPhysicalLocation(ClassUtils.getLocationOnDisk(clazz));
                migrationInfo.setExecutor(createExecutor(migration));

                migrations.add(migrationInfo);
            }
        } catch (Exception e) {
            throw new FlywayException("Unable to resolve " + getMigrationTypeStr() + " Java migrations in location " + location + " : " + e.getMessage(), e);
        }
    }

2. Migrations from different packages conflict one another.

For example, PackageA has migrations 10, 20, while PackageB has 10, 30. So migrations from PackageA won't be applied since their vertions are lower then from PackageB.

I use this library with spring boot 3 and java 17.
Configuration class CassandraMigrationAutoConfiguration is not initialised by spring context. Looks like it is just ignored. I have no any exception when application starts and migrations just not runs.
I have used it with spring boot 2.7.5 and java 17 and it worked.

I am trying to run multiple services of same type connecting to a Cassandra cluster. Looks like we are hitting a race condition when running the scripts. The server fails to come up with the following exception. Please let me know if any one has faced the same issue.

The version of the software being used is 2.2.0

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'migrationTask' defined in class path resource [org/cognitor/cassandra/migration/spring/CassandraMigrationAutoConfiguration.class]: Invocation of init method failed; nested exception is org.cognitor.cassandra.migration.MigrationException: Error during migration of script 2_Change-13418.cql while executing 'DROP TABLE conversation_message_subscriber;' at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:849) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) at Application.main(Application.java:32) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)Caused by: org.cognitor.cassandra.migration.MigrationException: Error during migration of script 2_Change-13418.cql while executing 'DROP TABLE conversation_message_subscriber;' at org.cognitor.cassandra.migration.Database.execute(Database.java:187) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.cognitor.cassandra.migration.MigrationTask.migrate(MigrationTask.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1903) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1846) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1774) ... 24 common frames omittedCaused by: com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table conversation_message_subscriber

Hi Patrick,

I'd like to implement an actuator endpoint to check migration status, similar to #39
This PR targets the v3 branch, and things changed quite a bit with the v4 driver so we can't just port this solution to the v4 branch.

Furthermore, since the session is closed after migrations, we definitely can't just create a Database bean as we can't reuse it after migrations.

The solution could be to manually re-create a Database with a custom cqlsession but that's not ideal, as it'd require to potentially create 2 identical manual sessions (one for migrations and one for actuator).
Fortunately for spring-data-cassandra users with same migration keyspace as the spring-data one, we can reuse the latter one.
But then, the Database class logic is to check the keyspace status and create it/create tables, which in the case of an actuator feature is useless and delay our Spring app startup as this process has already been handled by the migration (considering migrations are done before initializing the actuator bean).

Do you have an idea ?

On my Windows machine there is something wrong with two tests:

JarLocationScannerTest.shouldReturnTwoResourcesWhenJarFileWithOneScriptGiven 
JarLocationScannerTest.shouldThrowExceptionWhenNonExistingPathGiven

they are trying to open jar:file:C:/Users/MVASIL~1/AppData/Local/Temp/Test14094071718996507780.jar
on JarLocationScanner.getFileSystem() throwing:

java.lang.IllegalArgumentException: URI is not hierarchical

	at java.base/sun.nio.fs.WindowsUriSupport.fromUri(WindowsUriSupport.java:122)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.getPath(WindowsFileSystemProvider.java:97)
	at java.base/java.nio.file.Path.of(Path.java:203)
	at java.base/java.nio.file.Paths.get(Paths.java:98)
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:76)
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:151)
	at java.base/java.nio.file.FileSystems.getFileSystem(FileSystems.java:231)
	at org.cognitor.cassandra.migration.scanner.JarLocationScanner.getFileSystem(JarLocationScanner.java:44)
	at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:32)
	at org.cognitor.cassandra.migration.scanner.JarLocationScannerTest.shouldReturnTwoResourcesWhenJarFileWithOneScriptGiven(JarLocationScannerTest.java:47)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

The code is:
return FileSystems.getFileSystem(location);
where location is: jar:file:C:/Users/MVASIL~1/AppData/Local/Temp/Test17443126786644398458.jar
Note the jar:file:C

However it works with:
FileSystems.newFileSystem(URI.create("jar:file:///C:/Users/MVASIL~1/AppData/Local/Temp/Test17443126786644398458.jar"), new java.util.HashMap<>())
and getFileSystem() works after it was created with the above newFileSystem()
FileSystems.getFileSystem(URI.create("jar:file:///C:/Users/MVASIL~1/AppData/Local/Temp/Test17443126786644398458.jar"))

In a fat spring boot jar I get a failure:

Caused by: java.nio.file.NoSuchFileException: cassandra/migration
at jdk.zipfs/jdk.nio.zipfs.ZipPath.getAttributes(Unknown Source)
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.readAttributes(Unknown Source)
at java.base/java.nio.file.Files.readAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.getAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.visit(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.walk(Unknown Source)
at java.base/java.nio.file.FileTreeIterator.(Unknown Source)
at java.base/java.nio.file.Files.walk(Unknown Source)
at java.base/java.nio.file.Files.walk(Unknown Source)
at org.cognitor.cassandra.migration.scanner.JarLocationScanner.findResourceNames(JarLocationScanner.java:34)
at org.cognitor.cassandra.migration.MigrationRepository.scanForScripts(MigrationRepository.java:181)
at org.cognitor.cassandra.migration.MigrationRepository.(MigrationRepository.java:131)

This is probably because the jar has a specific structure: BOOT-INF/classes/cassandra/migration

To address that, maybe introduce multiple values for the script path properties - one that works in a fat-jar context, and another one that works in a regular "Run application" context

I tried to migrate the function :
CREATE OR REPLACE FUNCTION avgState ( state tuple<int,double>, val float ) CALLED ON NULL INPUT RETURNS tuple<int,double> LANGUAGE java AS 'if (val !=null) { state.setInt(0, state.getInt(0)+1); state.setDouble(1, state.getDouble(1) + val.doubleValue()); } return state;';

and because of the STATEMENT_DELIMITER = ";" the query was cutted in the middle.

Is there a way to baseline to a given version?

First of all, nice work 👍

My use case is that I would like to be able to create the keyspace as the first statement of my migrations, and the library assumes that the keyspace is already created prior to apply the migrations.

Database databaseConnection = new Database(cluster, 'mykeyspace')
MigrationTask migrationTask = new MigrationTask(
    databaseConnection, 
    new MigrationRepository())

migrationTask.migrate()

For instance, if I wanted to do integration tests, I would like not to create the keyspace by hand before running the migrations, and add it as part of the migration file:

-- 001_migration.cql
CREATE KEYSPACE mykeyspace
       WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 2 };

Is this something feasible ?