redbooth / departure Goto Github PK

Although we do have an account on Code Climate we're not seeing its status on the pull requests. It is something important to have.

So far we only have a little section in the README.md but without any instructions on how to proceed with a code change. This would help onboarding new contributors (hopefully).

Now github adds links to that file when opening a new PR.

There's an edge case concerning create table and adding and index.

When you run a create table migration that includes an index creation from a machine that doesn't have percona-toolkit installed, the table will be created but the index will not, as the former doesn't go through pt-online-schema-change.

We should check it and raise a PerconaMigrator::CommandNotFoundError accordingly, before running a CREATE TABLE.

This also applies to any other DDL statements that don't go through pt-online-schema-change.

When issuing a wrong alter table statement, pt-osc fails with a nasty Segmentation fault, which apart from being scary doesn't provide any details to help debug.

With the latest changes in Redbooth we should move the repo away from Redbooth's GitHub organization. I suggest we create a departure organization and we move it to it.

Thanks for releasing this gem!

I added support for rails 4.2.x in this fork

Could you create v4.2 branch so I can make pull request and get your opinions?

Hey,

Currently you are blocking usage of PERCONA_ARGS env variable when executing rake db:migrate here but we are actually running this command and would like to set —recursion-method none for all migrations. I was thinking about it and come up with 2 solutions:

1. To allow using it
2. To introduce new config variable in PerconaMigrator::Configuration, for example global_cmd_options that would have the same syntax as PERCONA_AGRS, leverage UserOptions class to parse it and change CliGenerator#all_options to sth like this:

def all_options
  env_options = UserOptions.new
  global_cmd_options = UserOptions.new(PerconaMigrator.configuration.global_cmd_options)
  options = DEFAULT_OPTIONS.merge(global_cmd_options).merge(env_options).to_a
  options.join(' ')
end

so we would have options overriding schema: default < global < env

Let me know what do you think and I'm happy to work on pull request implementing this :)

We need to solve the warning below when executing the test suite:

(...)
    when index is set to true
Warning: Using a password on the command line interface can be insecure.
Warning: Using a password on the command line interface can be insecure.
Warning: Using a password on the command line interface can be insecure.
        adds an index for the reference column
(...)

we might need to specify the password of the testing database in a config file.

With the rename to Departure we've changed the version numbers. This and the release process should be carefully documented.

Hi everyone! this week Percona reach us out to ask us to change the name of the gem so that it doesn't cause confusion on who is the actual author.

We open the question to everyone to suggest a new name for percona migrator. We've been thinking about removing migrator and use a single word. The ideas we have are: departure, exile or immigrane. Because exile has a bit negative connotation and there is already a gem called immigrant, departure seems to be our favorite.

thoughts?

Hey,

We noticed that currently mysql password is placed in logs when executing percona. I took a look at the code and also see 2 options to fix it:

1. filter out with regexp command_line.gsub(/-p .*? -/, '-p [filtered]') here - this won't be ideal as regexp is not the best tool to do this and it might break in case we change argument order etc.

2. Do refactor and make PerconaMigrator::Command accept cli_generator and sql instead of already generated string. That way we could add sanitize parameter to CliGenerator#generate and CliGenerator#parse_statement and push it down to ConnectionDetails object so we can do proper replacement in password_argument method

I would actually prefer 2nd option as it's more bulletproof, but wan to check with you before sending PR

Running migrations locally, I do not encounter this problem. However, when my demo and production servers run migrations during deployment (on VM running Linux, with migration triggered by Capistrano), the logs are always messed up. Here is a gist of one such migration -

https://gist.github.com/garettarrowood/42449b48709d85017a13bd5e522183ab

I think the issue is the read_nonblock(8) method here - https://github.com/redbooth/departure/blob/master/lib/departure/command.rb#L47 .

I'll open a PR, tie my repo to it, and see if just raising the number to 40 solves the problem. But before I do, can you provide some reference why a max length of 8 is used here? What problem might I create by changing it?

Upon further thought - it seems more likely logger.write_no_newline(data) is not working correctly.

Hi there,

This tool looks fantastic and it's exactly what my team is looking for. I'm having a bit of trouble getting it to work locally on my machine, however. I installed percona-toolkit via homebrew, but when I actually run a migration it's giving me the following error:

PerconaMigrator::Error: Cannot connect to MySQL: Cannot connect to MySQL because the Perl DBI module is not installed or not found.  Run 'perl -MDBI' to see the directories that Perl searches for DBI.  If DBI is not installed, try:
  Debian/Ubuntu  apt-get install libdbi-perl
  RHEL/CentOS    yum install perl-DBI
  OpenSolaris    pkg install pkg:/SUNWpmdbi

What should I hunt down to install to fix this issue? I did try installing libdbi via homebrew, but that did not have an effect.

Thanks!

The latest developments done in particular branches must be back ported to master. v4.2, v4.1 and v4.0 are way ahead of master.

This will be required to start implementing support for Rails 5.0 and 5.1

Hi folks,

I've been planning to add PerconaMigrator gem to our project and we noticed that the latest released gem v0.1.0.rc.6 does not have stdout logging implemented, I've though noticed that it was recently merged into master, here's a comparison since last release:
v0.1.0.rc.6...master

I was wondering if we could bump a minor version and release so we could benefit from the logging.

@sauloperez if we can make this release, it wouldn't need to add a general logging for whether PerconaMigrator is being used, making #31 not needed.

--max-load and --critical-load both accept an array of MySQL status variables as arguments, e.g.,

--critical-load Threads_running=500,Threads_created=1000

Unfortunately, departure currently doesn't parse this argument format correctly.

I believe many issues of this repo could be solved by newbies or first-timers. They are simple enough to jump in and get confidence and experience.

Some sites that showcase open source issues for newcomers are: https://www.codetriage.com/, http://up-for-grabs.net, https://contributor.ninja/, http://www.firsttimersonly.com/, https://yourfirstpr.github.io/ we could post them there.

To do that thought we should describe these issues in detail providing links to the particular files, methods, lines, etc. where things should be implemented.

I'm interested in working on Rails 5 support for percona_migrator. Do you have any preferences on methods for supporting multiple versions of Rails?

I have seen gems designate certain version numbers for certain versions of Rails. For example percona_migrator 1.x.x series is for Rails 4.x and percona_migrator 2.x.x is for Rails 5.x, etc.

I have also seen gems that detect which version of Rails is running and monkeypatch Rails with the correct strategy/adapter.

Is there anything to that I should be aware of before trying to start this work?

So far we don't pass the port number to pt-online-schema-change so setups using other than the default mysql port don't work. This is the case when doing a ssh tunnel to the database.

It would be great to set up rubocop to enforce a style guide. More info in the repo: https://github.com/bbatsov/rubocop