redbooth / departure Goto Github PK
View Code? Open in Web Editor NEWPercona's pt-online-schema-change runner for ActiveRecord migrations.
License: MIT License
Percona's pt-online-schema-change runner for ActiveRecord migrations.
License: MIT License
Although we do have an account on Code Climate we're not seeing its status on the pull requests. It is something important to have.
So far we only have a little section in the README.md but without any instructions on how to proceed with a code change. This would help onboarding new contributors (hopefully).
Now github adds links to that file when opening a new PR.
There's an edge case concerning create table and adding and index.
When you run a create table migration that includes an index creation from a machine that doesn't have percona-toolkit installed, the table will be created but the index will not, as the former doesn't go through pt-online-schema-change.
We should check it and raise a PerconaMigrator::CommandNotFoundError accordingly, before running a CREATE TABLE.
This also applies to any other DDL statements that don't go through pt-online-schema-change
.
When issuing a wrong alter table statement, pt-osc
fails with a nasty Segmentation fault
, which apart from being scary doesn't provide any details to help debug.
With the latest changes in Redbooth we should move the repo away from Redbooth's GitHub organization. I suggest we create a departure
organization and we move it to it.
Thanks for releasing this gem!
I added support for rails 4.2.x in this fork
Could you create v4.2 branch so I can make pull request and get your opinions?
Hey,
Currently you are blocking usage of PERCONA_ARGS
env variable when executing rake db:migrate
here but we are actually running this command and would like to set —recursion-method none
for all migrations. I was thinking about it and come up with 2 solutions:
PerconaMigrator::Configuration
, for example global_cmd_options
that would have the same syntax as PERCONA_AGRS
, leverage UserOptions
class to parse it and change CliGenerator#all_options
to sth like this:def all_options
env_options = UserOptions.new
global_cmd_options = UserOptions.new(PerconaMigrator.configuration.global_cmd_options)
options = DEFAULT_OPTIONS.merge(global_cmd_options).merge(env_options).to_a
options.join(' ')
end
so we would have options overriding schema: default < global < env
Let me know what do you think and I'm happy to work on pull request implementing this :)
We need to solve the warning below when executing the test suite:
(...)
when index is set to true
Warning: Using a password on the command line interface can be insecure.
Warning: Using a password on the command line interface can be insecure.
Warning: Using a password on the command line interface can be insecure.
adds an index for the reference column
(...)
we might need to specify the password of the testing database in a config file.
With the rename to Departure we've changed the version numbers. This and the release process should be carefully documented.
Hi everyone! this week Percona reach us out to ask us to change the name of the gem so that it doesn't cause confusion on who is the actual author.
We open the question to everyone to suggest a new name for percona migrator. We've been thinking about removing migrator
and use a single word. The ideas we have are: departure
, exile
or immigrane
. Because exile
has a bit negative connotation and there is already a gem called immigrant
, departure
seems to be our favorite.
thoughts?
Hey,
We noticed that currently mysql password is placed in logs when executing percona. I took a look at the code and also see 2 options to fix it:
filter out with regexp command_line.gsub(/-p .*? -/, '-p [filtered]')
here - this won't be ideal as regexp is not the best tool to do this and it might break in case we change argument order etc.
Do refactor and make PerconaMigrator::Command
accept cli_generator
and sql
instead of already generated string. That way we could add sanitize
parameter to CliGenerator#generate
and CliGenerator#parse_statement
and push it down to ConnectionDetails
object so we can do proper replacement in password_argument
method
I would actually prefer 2nd option as it's more bulletproof, but wan to check with you before sending PR
Running migrations locally, I do not encounter this problem. However, when my demo and production servers run migrations during deployment (on VM running Linux, with migration triggered by Capistrano), the logs are always messed up. Here is a gist of one such migration -
https://gist.github.com/garettarrowood/42449b48709d85017a13bd5e522183ab
I think the issue is the read_nonblock(8)
method here - https://github.com/redbooth/departure/blob/master/lib/departure/command.rb#L47 .
I'll open a PR, tie my repo to it, and see if just raising the number to 40 solves the problem. But before I do, can you provide some reference why a max length of 8 is used here? What problem might I create by changing it?
Upon further thought - it seems more likely logger.write_no_newline(data)
is not working correctly.
Hi there,
This tool looks fantastic and it's exactly what my team is looking for. I'm having a bit of trouble getting it to work locally on my machine, however. I installed percona-toolkit
via homebrew, but when I actually run a migration it's giving me the following error:
PerconaMigrator::Error: Cannot connect to MySQL: Cannot connect to MySQL because the Perl DBI module is not installed or not found. Run 'perl -MDBI' to see the directories that Perl searches for DBI. If DBI is not installed, try:
Debian/Ubuntu apt-get install libdbi-perl
RHEL/CentOS yum install perl-DBI
OpenSolaris pkg install pkg:/SUNWpmdbi
What should I hunt down to install to fix this issue? I did try installing libdbi
via homebrew, but that did not have an effect.
Thanks!
The latest developments done in particular branches must be back ported to master. v4.2, v4.1 and v4.0 are way ahead of master.
This will be required to start implementing support for Rails 5.0 and 5.1
Hi folks,
I've been planning to add PerconaMigrator gem to our project and we noticed that the latest released gem v0.1.0.rc.6 does not have stdout logging implemented, I've though noticed that it was recently merged into master, here's a comparison since last release:
v0.1.0.rc.6...master
I was wondering if we could bump a minor version and release so we could benefit from the logging.
@sauloperez if we can make this release, it wouldn't need to add a general logging for whether PerconaMigrator is being used, making #31 not needed.
--max-load
and --critical-load
both accept an array of MySQL status variables as arguments, e.g.,
--critical-load Threads_running=500,Threads_created=1000
Unfortunately, departure currently doesn't parse this argument format correctly.
I believe many issues of this repo could be solved by newbies or first-timers. They are simple enough to jump in and get confidence and experience.
Some sites that showcase open source issues for newcomers are: https://www.codetriage.com/, http://up-for-grabs.net, https://contributor.ninja/, http://www.firsttimersonly.com/, https://yourfirstpr.github.io/ we could post them there.
To do that thought we should describe these issues in detail providing links to the particular files, methods, lines, etc. where things should be implemented.
I'm interested in working on Rails 5 support for percona_migrator. Do you have any preferences on methods for supporting multiple versions of Rails?
I have seen gems designate certain version numbers for certain versions of Rails. For example percona_migrator 1.x.x series is for Rails 4.x and percona_migrator 2.x.x is for Rails 5.x, etc.
I have also seen gems that detect which version of Rails is running and monkeypatch Rails with the correct strategy/adapter.
Is there anything to that I should be aware of before trying to start this work?
So far we don't pass the port number to pt-online-schema-change
so setups using other than the default mysql port don't work. This is the case when doing a ssh tunnel to the database.
It would be great to set up rubocop to enforce a style guide. More info in the repo: https://github.com/bbatsov/rubocop
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.