boostport / migration Goto Github PK
View Code? Open in Web Editor NEWSimple and pragmatic migrations for Go applications.
License: Apache License 2.0
Simple and pragmatic migrations for Go applications.
License: Apache License 2.0
Including code-quality checks should be a must for every go project.
We tend to use in our projects the following ones: goimports, golint, vet, megacheck, errcheck.
They han proven to be very useful.
Possibly add more?
I was thinking of adding a Makefile, refactor the code to pass all the checks, and possibly integrate that Makefile check as a Travis CI building stage?
I would take over this.
Thoughts?
Hi,
I have tried to implement a db migration setup with this package - without success to this point.
I experimented a bit and for my setup it seems like the Driver Implementations in the ./driver package don't implement the Driver interface.
My code doesn't recognize the driver instance I created using the postgres.NewFromDB()
method as a valid implementation of the driver interface.
When I open your code directly the driver implementations aren't recognized as valid implementations of the driver interface either. Whereas the mockDriver you implemented in the package root is recognized correctly.
I experimented a little bit and it seems like if I modify the driver interface slightly to explicitly import the package itself for the PlannedMigration parameter of the Migrate() method all interface implementations are recognized correctly.
To me this seems like a strange behaviour since there shouldn't be an issue with the original interface declaration and implementations as they are.
So the question is wether my setup is just misbehaving or if you want to adapt this change. Unfortunately this breakes the mockDriver implementation. So one would have to find a solution for that.
I still think that Intellij is getting something wrong. Please let me know if you have any advice on how to use this lib properly.
Kind regards,
Felix
Additional support for triggers within MySQL/MariaDB statements. Since triggers have a statement enclosed in the syntax which can contain a ;
it breaks the normal parsing of the migration.
Using a common double semi-colon ;;
that is a result of various sql dump methods a basic approach could be something like the following before driver/mysql.go:L71 :
import(
...
"regexp"
...
)
...
if strings.Contains(content, "END;;") {
triggerRegexp := regexp.MustCompile(`(?sm)^CREATE\sTRIGGER.*?END\;\;`)
triggerStmts := triggerRegexp.FindAllString(content, -1)
for i, _ := range triggerStmts {
if _, err := driver.db.Exec(triggerStmts[i]); err != nil {
return fmt.Errorf("Error executing statement: %s\n%s", err, triggerStmts[i])
}
// Remove the trigger statement from the migration content
content = strings.Replace(content, triggerStmts[i], "", -1)
}
}
Basic trigger example:
CREATE TRIGGER some_delightful_trigger
BEFORE UPDATE
ON `some_magical_table` FOR EACH ROW
BEGIN
INSERT INTO `some_magical_table_history` SET
`some_magical_table_id` = OLD.id,
`a_column` = OLD.a_column;
END;;
Experiencing some import problems with the package due to a malformed filepath in test-migrations/3_add_column.up..sql
there are multiple dots.
This is the error message I get is the following:
C:\Users\felix\source\felixcolaci\worf\shared>go mod tidy
go: extracting github.com/Boostport/migration v0.12.0
-> unzip C:\Users\felix\go\pkg\mod\cache\download\github.com\!boostport\migration\@v\v0.12.0.zip: malformed file path "test-migrations/3_add_column.up..sql": double dot
I have been wondering for quite some time why in our projects we end up with a lot of linter dependencies in go.mod. It seems they are included transitively through Boostport/migration. They seem to be imported in tools.go
. What is it? Would it be possible to keep go.mod clean of linter dependencies?
It seems that Golang migrations doesn't integrate well with other regular database
migrations.
To reproduce the issue I'm facing, imagine an scenario where I have a migrations
folder with a single sql migration:
I've migrated this up
using db
, so I have one row 1_step1
contents in the schema_migration
table.
Then I want to build a go
migration as a second step like this:
src := golang.NewSource()
src.AddMigration("2_step2", migration.Up, func() error {
fmt.Println("I'm migrating up")
}
src.AddMigration("2_step2", migration.Down, func() error {
fmt.Println("I'm migrating down")
}
I've used the postgres driver to return the contents of the schema_migration
table like this:
drv, err := postgres.New(dbURL)
if err != nil {
return err
}
applied := func() ([]string, error) {
return drv.Versions()
}
drv.Versions()
returning only 1 on this point, 1_step1
which is fine.
If I try to migrate the go
2_step2
over the first db
step_1
however it doesn't apply anything.
getMigrations()
returns only the go
source manually added files 2_step2
in this case.appliedMigrations
has only db
entries 1_step1
.migrationsToApply
is empty, so the go
migration 2_step2
migration never gets applied.Something I misunderstood or doing wrong?
Btw I have some improvements in mind for the docs, took me a while to understand some points.
CC\ @F21
I enjoy working with your simple and easy migration library, thanks for creating it. ๐
However I ran into an issue where I had named my migrations like this:
1-init.up.sql
Instead of:
1_init.up.sql
This was a frustrating bug to fix because it took me a while to see that I had used a hyphen instead of an underscore. The hyphen doesn't match the regex the library uses to look for migrations. I think requiring an underscore is a good idea, but it would be helpful to docment the actual regex that the migration file names must match.
Firstly, thank you for this wonderful library! We've been happily using it for over 2 years.
I have a feature request. Sometimes we have multiple instances of our application start a the same time, and they all try to run the same DB migration. Suffice to say, only one instance is successful and the others often crash. Worst case scenario, they could interfere with each other and break the migration.
I was thinking it would be useful to have an option to lock the schema_migration
table in order to avoid such issues (or even do it by default). This would be the order of operations:
schema_migration
schema_migration
This way if multiple applications try to run migrations at the same time, only one will get the lock. After the migration completes, the other ones can then get the lock, select versions, and realize that there's nothing to migrate anymore.
I'm thinking of either implementing locking in our applications or contributing the feature here, but I don't know when I'll have the time to do it. Perhaps someone else would like to implement it? For reference, we're using the postgres driver.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.