marcusg / pg_backup Goto Github PK

View Code? Open in Web Editor NEW

5.0 1.0 3.0 32 KB

Create, restore, download and upload postgres dumps locally and on remote servers using capistrano

Home Page: https://github.com/marcusg/pg_backup

License: MIT License

Ruby 100.00%

capistrano dumps restore restoring-postgres-dumps postgresql rails database dump ruby synchronization

pg_backup's People

Contributors

Stargazers

Watchers

Forkers

logankoester jspraul beowulf2128

pg_backup's Issues

Add sync task

add simple sync task which includes

create dump on source server
download dump from source server
upload local dump to target server
load dump into target server

Capistrano unable to run backup due to RAILS_ENV

Ruby 2.3.3
Rails 5.0.1
Capistrano 3.7

Having followed the steps to enable Capistrano and pg_backup, everytime I try to run any backups, I get the following:

In Capfile:

require "pg_backup/integration/capistrano"

Then run

bundle exec cap staging pg_backup:dump:create

and it returns

00:30 pg_backup:dump:create
      01 bundle exec rake pg_backup:dump:create
      01 rake aborted!
      01 ActiveRecord::AdapterNotSpecified: '' database is not configured. Available: ["staging"]

Any ideas?

Add tests

with https://github.com/technicalpickles/capistrano-spec?

The dump:load command should clear the target first

Currently the dump:load command just does not the target database structure at all. This is quite bad when it comes to a real backup load with a different migration state. It's not enough to clear and reimport all the data.

The the dump creation MUST take care of including the whole schema and the load command MUST take care of dropping the database first, recreate it, load the schema and then the data from the dump.

Looks like pg_restore is not able to this by passing --clean and --create together, due to:

When this option is used, the database named with -d is used only to issue the initial DROP DATABASE and CREATE DATABASE commands. All data is restored into the database name that appears in the archive.

So we should drop+create the database first, with a psql statement.

I will come up with a PR for this soon. :)

DEBUG [3163879a] 	rake aborted!

Don't know how to build task 'pg_backup:dump:create' (see --tasks)

 DEBUG [3163879a] 	/home/me/app/shared/bundle/ruby/2.3.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli/exec.rb:74:in `load'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli/exec.rb:27:in `run'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli.rb:332:in `exec'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli.rb:20:in `dispatch'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/cli.rb:11:in `start'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/exe/bundle:34:in `block in <top (required)>'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/lib/bundler/friendly_errors.rb:100:in `with_friendly_errors'
/home/me/.rvm/gems/ruby-2.3.1@pre/gems/bundler-1.13.3/exe/bundle:26:in `<top (required)>'
/home/me/.rvm/gems/ruby-2.3.1@pre/bin/bundle:22:in `load'
/home/me/.rvm/gems/ruby-2.3.1@pre/bin/bundle:22:in `<main>'

 DEBUG [3163879a] 	(See full trace by running task with --trace)

The deploy file:

server 'server', user: 'user', roles: %w(web app db)

set :rails_env, 'staging'

set :default_env, rails_env: fetch(:rails_env)

set :environment, fetch(:rails_env)

set :deploy_to, '/home/me/app'
set :branch, 'staging'
.....

# Flow
before 'deploy:starting', 'database:backup_list' # shows list of dumps in remote dir
before 'deploy:started',  'database:backup'

namespace :database do
  task :backup_list do
    invoke 'pg_backup:dump:list'
  end

  task :backup do
    invoke 'pg_backup:dump:create'
  end

  task :download_backup do
    invoke 'pg_backup:dump:download'
  end
end

Any ideas?

Thank you.

Ensure local download directory before downloading remote dump

default rake task not loaded

dump dir is not linked in shared dir using capistrano

set :linked_dirs, fetch(:linked_dirs, []).push('dump')

Better dump name formatting

use Time.now.strftime("%Y%m%d%H%M%S") instead of Time.now.to_i

Use pg_backup helpers in custom Capistrano task

I am trying to use the PgBackup::Helpers::Database in a custom Capistrano task

require 'rake'
require 'pg_backup/helpers/database'

include PgBackup::Helpers::Database

namespace :database do
  task :dev_backup do
    on roles(:db) do
      within current_path do
        with rails_env: fetch(:environment) do
          dump_dir = "#{shared_path}/#{fetch(:pg_backup_remote_dump_dir)}"
          backup_file = "#{dump_dir}/dev_database.backup"
          if !test("[ -d #{dump_dir} ]")
            error "Folder '#{dump_dir}' does not exits!"
          else
            with_database_config do |host, db, user, pw|
              ` PGPASSWORD=#{pw} pg_dump --host #{host} --username #{user} --clean --format=c --no-owner --no-acl --exclude-table-data 'table' #{db} > #{backup_file} `
            end
          end
        end
      end
    end
  end
end

but looks like ActiveRecord constant is not loaded

uninitialized constant PgBackup::Helpers::Database::ActiveRecord

any hints about how to resolve this?

Thx.

marcusg / pg_backup Goto Github PK

pg_backup's People

Contributors

Stargazers

Watchers

Forkers

pg_backup's Issues

Add sync task

Capistrano unable to run backup due to RAILS_ENV

Add tests

The dump:load command should clear the target first

Add dump rotation

List dumps with filename

Capistrano: Don't know how to build task 'pg_backup:dump:create'

Ensure local download directory before downloading remote dump

default rake task not loaded

Better dump name formatting

Use pg_backup helpers in custom Capistrano task

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent