Support for golang migrations about migrate HOT 16 OPEN

hello any updates on this?

from migrate.

This would allow us to finally stop using goose. The non-static migrations could just be type .so golang plugins which implement an interface with an Up(input interface{}) and Down(input interface{}) method. Our driver for this is that we want migrations to be managed by our app, not a 3rd party binary, and we certainly don't want to be compiling code at runtime the way goose dynamic migrations do.

from migrate.

None AFAIK, feel free to pick it up!

from migrate.

I see the usefulness of running custom migrations in Go and/or bash and am not opposed it.

However, I don't have the bandwidth to implement it myself but will participate in design discussions and review the subsequent PR(s).

We may need to change the driver interface to support custom Go/bash migrations

from migrate.

Faced with the same problem where we need to run complex migrations involving data generation, ended up implementing our own migration framework.

from migrate.

@ahammond It's worth starting a new framework. The amount of actual work that is needed here is huge (i'm not blaming this framework, no) comparing to the amount of work needed to extract migrations framework from here: https://github.com/fnproject/fn/tree/master/api/datastore/sql/migrations

from migrate.

Using Go plugins is an interesting idea! It looks like plugins are currently only support in Linux and Mac OS.

From: https://golang.org/pkg/plugin/

Currently plugins are only supported on Linux and macOS. Please report any issues.

I'm not familiar with Go plugins (yet), so I'm not sure sure if plugins will work across platforms. e.g. built on Mac OS but used/run on Linux.

Also, I'm not sure about the requirements for this feature. Do people want to run Go migrations with SQL migrations? e.g. run 5 SQL migrations, then 1 Go migration, followed by 3 SQL migrations.

A current work around (if your DB supports it) would be to use a temporary procedure or function to perform the more complicated logic that's hard to do in SQL. e.g. for postgres, there's: https://www.postgresql.org/docs/current/server-programming.html

from migrate.

Plugins are still quite raw technology, let's be honest. With plugins, you'd have a problem when your plugin is built for another OS, etc. I'd really stick with *.go migrations (more safe)

I'm not familiar with Go plugins (yet), so I'm not sure if plugins will work across platforms. e.g. built on Mac OS but used/run on Linux.

no, linux plugins for linux, mac plugins for mac

Also, I'm not sure about the requirements for this feature. Do people want to run Go migrations with SQL migrations? e.g. run 5 SQL migrations, then 1 Go migration, followed by 3 SQL migrations.

No, there's no point having both, only *.go matters and gives a lot of options around the migration process.

from migrate.

@denismakogon can you please share an example on how to run it along this package?
Thanks!

from migrate.

Everything runs in docker containers these days. A user-interactive experience for these tools should be focused around debugging migrations, and to me, that means keeping the environment as close as possible to the production runtime environment. And that means a linux container.

from migrate.

@ahammond,
That's great, exactly as I thought.
Nevertheless, a real life example will become handy.

from migrate.

see: Yaegi for dynamic plugin :)

from migrate.

Is there any work going on implementing this feature?

from migrate.

I've ended up using this simple library that suit my needs
https://github.com/xakep666/mongo-migrate

from migrate.

What about simply making it easy to extend the CLI so that I could do,

func main() {
  custom_migrations = map[string]Migration{
    "183747version": my_function_handle
  }

  migration.CustomCLI{
    migrations: custom_migrations
  }.run()
}

Then you build this, it works identical to the normal CLI, but the migrations take into account this function as one of the steps.

Seems like it would be fairly simple and no importing .so files. If people like this idea, I could look at implementing.

from migrate.

Is there any updates for migrations with go function / extension ?
because I'm using goose right now.

from migrate.