A completely generic database migration cli tool ♊️

The benefit is:

• You can use it in any project
• You can use it with any language (given a gemini language runtime has been developed - see below)
• It will never touch or effect your application (so many migration tools these days think their so self important they force you to hook them directly into your app 🤷)

The downside is:

• You have to write some boilerplate
• You give up some features

should you use Gemini?

If your using Gemini on a non-python project there is probably some setup (dependency install, etc.) that Gemini needs to do in order to work with that language. Run gemini setup [runtime] and Gemini will take care of it for you. For example, gemini setup csx will setup C# by installing dotnet-scripts on your machine. Sound scary? You will be prompted for approval with a list of changes that would be made before any work is started.

The first thing you will need to do is add a geminiver file. This is the boiler plate. Because Gemini is generic it can't make assumptions about how to talk to your database. So we will use the geminiver file to write those functions. If your using python it will be geminiver.py if your using C# it will be geminiver.csx. Check out the examples to see how its done. You will need to add the logic that goes to the database and retrieves the latest version and the code that goes to the database and sets the version. Gemini will then use those functions to keep your database up to date.

So, you wrote some code in your geminiver... does it work? Lets see! Gemini ships with a command that will call your get version and set version functions to help you ensure all is running in proper order. Just run gemini dryrun.

gemini dryrun --runtime csx

Create your first migration script. This command will simply generate a migration script template for you. Since this is the first migration your making - and thereby none currently exist - you need to use --runtime to tell Gemini what langauge you want to use for your migration scripts. Most other commands - including subsequent migrate commands - will attempt to auto detect the language runtime by looking at what is already in the /migrations directory.

gemini migrate --message "add user table" --runtime py

Again, this only creates a template. You'll need to go in the file and script out the logic to create that table in whatever way fits with your application and organization.

The upgrade command will check the current version of your database and run all child migrations in order that it finds in the migrations directory.

gemini upgrade

Maybe your developing and you want to undo that last migration so you can make some changes to it and run it again? The rollback command will run the down function of the latest migration.

gemini rollback

The basic flow of logic/modules in Gemini is...

------------------
|      cli       |
--------v---------
|    command     |
--------v---------
|    runtime     |
--------v---------
|   migrations   |
------------------

This is - and should always be - a simple module that handles cli invokations, gathers any arguments, options, and flags, and then invokes the desired command.

Command modules are where the business logic (if you can call it that) lives. This is where the core logic you would expect in Gemini lives. This module contains the logic that decides if migrations should be run, how many, and when. It detects variances in the current version of the database and the version you might attempt to run. In order to support many languages it does all this through the use of the runtime module.

The runtime is funky... I'll admit. However, for its arguably hacky architecture the benefit is that you can now allow Gemini to handle any programming language. Every language has its own runtime language module. Each languages runtime module contains the logic and implementation for interacting with scripts and code of that particular language. The hacky part comes in when you factor in that this is not easy. There are great tools to help us out for a lot of languages but there is still a lot of string parsing and semi-wild dynamic script execution or command line tools used as a bridge to get the job done.

It is no longer Gemini's code executing. The runtime is now dynamically loading user provided scripts and executing the code inside them.

A language runtime is a python module that exposes a standard set of functions that have been implemented to interact with migrations scripts of a specific language. The runtime module must conform to the runtime protocol (protocol requirements listed below). For example, the C# language runtime module contains functions implemented specifically to read, parse, template, and execute .csx migration scripts. Lets say you have an api written in Scala (or maybe your weird and you just want to manage your database changes with Scala scripts) then you would create a runtimes/scala/runtime.py module and implement the required functions to read, parse, template, and execute Scala scripts.

• get_database_version()
• set_database_version(version: str)
• get_migrations()
• run_down(version: str)
• run_up(version: str)
• create_migration(new_version: str, parent_version: str, name: str)
• setup()

Gemini isn't for everyone and all projects. In order to make it generic enough to work in such a variety of projects we've had to let go of some standard database migration tool features. One example is dynamically building a migration script based on changes to your model. Theoretically, we could develop this feature... it would just mean adding to the boilerplate code every user needs to write. Each user would need to write project specific boiler plate so Gemini could interact with that projects models. On the other hand - if you have an older project with no migration management, or maybe just a directory of raw SQL scripts and want to add in a migration manager, or you have a variety of project with a variety of languages and you would like to unify the migration management for CI/CD and developer sake, or you prefer not to use a tool that forces itself into your application code - Gemini might be a good tool for you. So, there are the pros, cons, limitations, and features to using Gemini.