The command xmigra decldoc '!table' omits information about foreign key constraint keys:

• link to (required)
• on update
• on delete

To facilitate searching, it would be nice if xmigra history --known listed every term that can be searched when searching changes.

Reorganize this code as a gem that can be registered with rubygems.org.

If the tutorial step for merging changes to the structure directory is followed using a Git repository, it is no longer possible to generate a production script (xmigra upgrade --production) because of the change to the migration file made by xmigra unbranch. This is implemented in the check at

When a new migration is created that extends the production migration chain, this hook command (if specified) will be run to create a database backup. This will allow restoring the database to a state on the production migration chain to support the case of syncing in upstream changes:

• Sync-merge or rebase
• xmigra.rb unbranch
• Restore database from backup
• Run the generated upgrade script against the "production state" database

Since the hook script can be anything, it can either do the backup itself or prompt the user to do the backup.

To be more useful as a "plumbing" command, xmigra history could accept a --paths option which would change it from outputting SQL to outputting an ordered list of paths to migrations YAML files.

Index definition files don't get the filename metavariable replaced during upgrade script generation, so the xmigra new --index command should not inject it into the generated file and should, instead, put the correct string there.

While it's great that xmigra history automatically picks up that a declarative implementation migration changes its declarative subject, it would be nice to also include anything explicitly included in changes as does a plain migration.

This could easily be implemented in XMigra::DeclarativeMigration#changes by using super() | on the result of the current definition.

Add support for a new database system, "PostgreSQL", that outputs scripts in PG-SQL compatible with the PostgreSQL database. Emitted SQL should be compatible with PostgreSQL 8.4 or later.

Because it was just comparing the "new" (recovered/copied-from-history) declarative file to the migration file that implemented destruction of that declarative, XMigra incorrectly assessed the declarative file as :newer instead of the correct :unimplemented.

Add hook points for extension libraries (e.g. sql_loaded, final_script_chunk, etc.) and a command line option to load (require) an arbitrary extension library.

XMigra fundamentally organizes changes to the data-containment structure of a database as a chain of migration commands and there is no effective way or convincing argument here to change that. However, XMigra could offer some level of support to developers for comparing new versions of declarative-language structure files to their corresponding previous versions in the version control system and building a proposed migration off of those changes.

My initial thought is that the declarative files should be stored in the structure/declarative subdirectory of the XMigra project. Addition, deletion, or modification of those files would, for certain XMigra operations like upgrade, require a .decl.yaml file to exist in the structure directory that explicitly references the new version of the declarative file and gives SQL for implementing the change. XMigra would not enforce that the given SQL effected the change to the declarative file when run against the database -- only that one of these "special" migrations existed that claimed to do so. The files in the structure/declarative subdirectory should be named for the objects in the database they implement (as do the files in the access subdirectory).

The general flow is:

• Alice revises (or creates or deletes) a declarative file for one database element.
• Alice tries xmigra upgrade. XMigra exits with failure and notes that no existing migration implements the current state of the declarative file.
• Alice runs xmigra impdecl to generate migrations implementing the current version of any unimplemented declarations. Any migration XMigra doesn't know how to fully generate will contain a flag that will continue to cause upgrade script generation to fail until the migration is edited and the flag removed.
• Alice tries xmigra upgrade again. This time, the script generation is successful.

The special declarative-change-effector migrations (implementation migrations or .decl.yaml files) contain a superset of the top-level keys required for standard migrations. In addition, they contain (bold keys are required):

The .decl.yaml file contains the to realize hash because generation of the upgrade script in development mode does not require the files be committed to version control, and it would be important that the .decl.yaml files have a reference not only to the object name but also a hash of the contents to avoid the situation where Bob changes the declarative file, generates (and possibly edits) a migration to implement it, and then modifies the declarative file again but does not regenerate or edit the corresponding new migration file. Unless Bob is protected by XMigra checking the hash of the declarative file against the hash in the implementing migration, he could have a very bad day.

In addition to supporting addition, revision, and deletion, the structure/declarative system should also support the operations of adoption and renunciation, which allow existing structural elements to become declaratively managed or declaratively managed structural elements to cease declarative support without being eliminated from the database. xmigra impdecl run with the --adopt flag would generate an adoption script and similarly generate a renunciation script when run with --renounce.

xmigra impdecl would generate the new migration file using the object name and the identifier of the commit/revision of the previous version of the declarative file (or new for newly created or adopted declaratives, end for destroyed or renounced ones). It would properly set all the required fields (even those required for a normal migration) and include a diff from the previous version of the declarative to the current version in the migration under the delta key. It might generate SQL to implement the declarative change and it might add a implementation qualification key that would prevent generating an upgrade script.

The files in structure/declarative would also be YAML files with top-level keys TBD. The files would include information on the type of object as well as its structure, allowing piecewise implementation of SQL generation support in the XMigra tool.

Allow specification of XMigra migrations as ActiveRecord::Migration code.

Reversible migrations should store reversion steps in the database for application when/if a new upgrade script that modifies the old migration is run.

Extending the production chain in a git repository schema does not trigger the production-chain-extension hooks.

The select_for_install_sql method returns SQL that selects multiple rows in a scalar context if more than one migration has already been applied, causing upgrade failure.

Environment:

• Windows 10
• tortoise svn command line tools

svn --version
svn, version 1.9.3 (r1718519)
   compiled Dec 13 2015, 15:47:16 on x86-microsoft-windows

Build works fine with xmigra 1.5.1.

Repro:

• Upgraded to to xmigra 1.6.0
• Run same command displayed below.

Build command output:

running command "ruby "C:\Ruby21-x64\bin\xmigra" upgrade "--source=<Path to source>" -o "<output path>\conversation.2-debug-schema.sql""
The system cannot find the path specified.
Subversion command failed with exit code 1 (XMigra::VersionControlError)

There is a LIMIT 1 clause in the wrong part of the SQL statement in check_preceding_migrations_sql, leading to an error during upgrade.

Provide an "init" subcommand that initializes a directory as an XMigra schema.

The generated upgrade script errors out on EXECUTE UpgradeCommads; (should be EXECUTE UpgradeCommands;).

XMigra currently demands that migration files used to produce production upgrade scripts have been unmodified in the version control system since their introduction. This restriction is overly broad; not every part of the migration file contributes to the configuration of production servers, especially the "changes" section. It would be nice to be able to alter the "changes" section after committing. It would also be nice if mild formatting changes to the SQL (if SQL is used in the definition) that cannot change the meaning of the SQL commands (leaning conservatively) were acceptable, as reformatting may occasionally be desirable.

This would probably involve creating an SQL canonizer (overridable by RDBMS specifics) so SQL semantics could be compared (along with preceding state, which is a simple string compare) between the current version of the migration file and it's originally committed form. One potential issue is introduction of addition top level keys that help generate the SQL (especially through plugins): should they have hard immutability requirements? Can the immutability requirements be eased by plugins?

The support code for the !table declarative schema:

• is confused (when using Git version control) about indexed/cached files not being committed (it is considering them committed, but they have no commit hash since they aren't really committed)
• is failing to retrieve file content because previous changes removed the leading ./ from the path used in the git show command.

Issue #31 and the fix in PR #32 revealed a gap in maintaining the immutability of the migration chain in a Git repository network. A more reliable process would be:

• Merge (or directly commit) changes into the "main branch" (master or a version-specific branch)
• Use the new xmigra promote tool to update the central definition of production (preceded by checks that the new commit is consistent with the old one).

Setting this up would involve using a branch name different than master in the xmigra-master Git attribute for the database.yaml file -- possibly something like prod-db-chain. It will probably also be necessary to mark a branch as the "premaster" to prevent accidentally promoting unmerged development changes to the database schema's master branch.

Currently, XMigra only supports immediate creation or update of indexes during the main upgrade transaction. It would be nice if the features of the database that allow indexes to be created concurrently with normal access to the table were accessible/manageable through XMigra.

Database Support

• PostgreSQL
• MS SQL Server:
  • Guidelines for online index operations
  • How online index operations work

Considerations

• MS SQL Server does not support these operations for all license levels. For organizations that use the more expensive licenses with support for this feature, it is likely that indexes will need to be managed without it in some (lower) environments.
• PostgreSQL does not allow CONCURRENT index creation inside transactions.

xmigra impdecl often leaves a big mess if the declarative file has errors, especially if changes didn't start from a clean working tree/working copy. It would be nice if XMigra could back its changes out with xmigra impdecl --undo.

I get

undefined local variable or method `selection' for #<XMigra::Console::Menu:0x00000101953718> (NameError)

when I try to run xmigra init and select one of the database options.

The database.yaml file should support an option allowing specification of the schema name (rather than the hard-coded default of xmigra) within which the XMigra version tracking metadata is stored. In addition to general flexibility, this allows different parts of the same database to be managed by different XMigra schema source file trees.

The method vcs_file_modified? is invoked but not defined by the Git and Subversion VCS implementations.

Adding a new column with nullable: false in a declarative !table file does not include NOT NULL in the generated ALTER TABLE statement.

It's a pain for the user to have to create and open the rollback file manually. If the proposed --with-rollback flag is given to xmigra new when generating a migration (either with the --migration flag or with no file-type flag), create an empty rollback file for the migration and open it in the editor (if the migration is opened in the editor).

Stopping points are migrations with an SQL boolean expression attached; if the boolean expression is true, the sql of the migration is carried out, otherwise the upgrade is early-exited with a message about how to proceed.

Use of stopping points precludes single-transaction upgrade guarantees, but it enables functionality like non-blocking index creation (#25) and shared database management (#6).

The unbranch subcommand has too much logic in the Program class; that logic (although not the messages) should move to the MigrationConflict class.

It is a convenience for help and general text output to run through a pager program if it is specified in the environment and when the STDOUT is a TTY.

As a user, it's nice to get tab-completion on file paths. If I use rm, del, or a function of my editor to remove the declarative file before invoking xmigra impdecl for the destruction migration, I don't get tab completion. Rolling the deletion of the file into xmigra impdecl will solve that problem by allowing me to specify deletion of the file and generation of the implementing migration in one command (without repeating the path to the declarative file -- DRY and all that).