rolang / dumbo Goto Github PK
View Code? Open in Web Editor NEWSimple database migration tool for Scala + Postgres with skunk on JVM and Native
License: MIT License
Simple database migration tool for Scala + Postgres with skunk on JVM and Native
License: MIT License
E.g. for operations like CREATE INDEX CONCURRENTLY
...
Add test of the command line docker build before publishing the docker image.
Repeatable migrations have a description and a checksum, but no version. Instead of being run just once, they are (re-)applied every time their checksum changes.
This is very useful for managing database objects whose definition can then simply be maintained in a single file in version control. They are typically used for
- (Re-)creating views/procedures/functions/packages/...
- Bulk reference data reinserts
Within a single migration run, repeatable migrations are always applied last, after all pending versioned migrations have been executed. Repeatable migrations are applied in the order of their description.
It is your responsibility to ensure the same repeatable migration can be applied multiple times. This usually involves making use of CREATE OR REPLACE clauses in your DDL statements.
Naming:
Hi! I just noticed that loading the migration files from the resources directory doesn't work on windows.
In ResourcesReader getClass().getResourceAsStream(path.toString)
it uses the platform path separators which turns the resource path into \\db\migration\V001__init.sql
but java only understands /db/migration/V001__init.sql
so I get a NPE on getResourceAsStream.
fs2.CompositeFailure: Multiple exceptions were thrown (2), first java.lang.NullPointerException: Cannot invoke "java.io.InputStream.read(byte[], int, int)" because "is$1" is null
at fs2.CompositeFailure$.apply(CompositeFailure.scala:58)
at fs2.CompositeFailure$.apply(CompositeFailure.scala:45)
at fs2.Pull$.addError$1(Pull.scala:1173)
at fs2.Pull$.viewCont$1(Pull.scala:1180)
at fs2.Pull$.$anonfun$compile$22(Pull.scala:1217)
at get @ skunk.util.Pool$.free$1(Pool.scala:171)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Pull$.$anonfun$compile$21(Pull.scala:1215)
at update @ skunk.util.StatementCache$$anon$2.put(StatementCache.scala:48)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at update @ skunk.util.StatementCache$$anon$2.put(StatementCache.scala:48)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at fromSync @ de.bitmarck.bms.zpc.datahub.Main$.applicationResource(Main.scala:75)
at uncancelable @ fs2.Compiler$Target.uncancelable(Compiler.scala:165)
at flatMap @ fs2.Compiler$Target.flatMap(Compiler.scala:163)
at handleErrorWith @ fs2.Compiler$Target.handleErrorWith(Compiler.scala:161)
at modify @ skunk.util.Pool$.$anonfun$ofF$2(Pool.scala:100)
Caused by: java.lang.NullPointerException: Cannot invoke "java.io.InputStream.read(byte[], int, int)" because "is$1" is null
at fs2.io.package$.$anonfun$readInputStream$3(io.scala:48)
Skunk changed the tracing library in v 1.0.0-M1, making dumbo unusable with it
Hi @rolang! Big fan of Flyway and the Typelevel ecosystem! Thanks for putting this open source project together.
I'm using dumbo
for a pretty simple CLI application. It'd be great to be able to fine-tune the text output from dumbo
that's currently produced via Console[F].println
using a logging framework; as a dependency it'd be even better if it leveraged SLF4J via something like log4cats-slf4j
.
I'd be happy to take a crack at this myself if it's something you'd be interested in!
When testing this library we noticed that it was not possible to run it through sbt with scala 2.13.
It results in this exception:
java.nio.file.FileSystemNotFoundException: null
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:156)
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:142)
at java.base/java.nio.file.Path.of(Path.java:208)
at java.base/java.nio.file.Paths.get(Paths.java:98)
at dumbo.ResourceFilePath$.$anonfun$fromResourcesDir$3(ResourceFilePath.scala:25)
at delay @ dumbo.ResourceFilePath$.$anonfun$fromResourcesDir$2(ResourceFilePath.scala:24)
at delay @ dumbo.ResourceFilePath$.fromResourcesDir(ResourceFilePath.scala:22)
at flatMap @ dumbo.ResourceFilePath$.fromResourcesDir(ResourceFilePath.scala:22)
For this code:
Dumbo
.withResourcesIn[F]("db/migration")
.apply(
sessionResource = Session.single[F](
host = dbConf.host.value,
port = dbConf.port.value,
user = dbConf.user.value,
database = dbConf.db,
password = dbConf.password.value.some,
),
defaultSchema = schema.getOrElse("public"),
)
.runMigration
.flatMap { result =>
Async[F].delay(println(s"Migration completed with ${result.migrationsExecuted} migrations"))
}
After digging a bit it seems like Paths.get
does not handle jar files jar:file:/Users/.../target/bg-jobs/sbt_fac89327/job-21/target/92a8dfce/9d592533/test_2.13-27e70d62.jar!/db/migration
Any advice on how to proceed?
Scan given location for migration scripts recursively (as Flyway does).
E.g. include scripts from sub directories given structure like
migrations
V1__a.sql
subdir1
V2__b.sql
subdir2
V3__c.sql
Add support for reading migration files from embedded resources
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.