shahriyarr / mysql-autoxtrabackup Goto Github PK

View Code? Open in Web Editor NEW

139.0 10.0 79.0 817 KB

MySQL-AutoXtraBackup commandline tool written in Python 3 based on Percona XtraBackup

Home Page: https://autoxtrabackup.azepug.az/

License: MIT License

Python 90.88% Shell 7.47% Dockerfile 1.65%

mysql-autoxtrabackup azerbaijan-mysql-user-group percona-xtrabackup python3 azerbaijan azerbaijan-python-user-group

mysql-autoxtrabackup's Introduction

MySQL-AutoXtrabackup

MySQL AutoXtrabackup commandline tool written in Python 3. The source code fully typed with hints - structured the project mostly similar to FastAPI and Pydantic

For community from Azerbaijan MySQL User Community: Python Azerbaijan Community.

For any question please open an issue here.

What this project is about?

The idea for this tool, came from my hard times after accidentally deleting the table data. There was a full backup and 12 incremental backups. It took me 20 minutes to prepare necessary commands for preparing backups. If you have compressed + encrypted backups you need also, decrypt and decompress, which is going to add extra time for preparing backups. Then I decided to automate this process. In other words, preparing necessary commands for backup and prepare stage were automated.

We have nice CLI with necessary options:

autoxtrabackup --help
Usage: autoxtrabackup [OPTIONS]

Options:
  --dry-run                       Enable the dry run.
  --prepare                       Prepare/recover backups.
  --run-server                    Start the FastAPI app for serving API
  --backup                        Take full and incremental backups.
  --version                       Version information.
  --defaults-file TEXT            Read options from the given file  [default: /
                                  home/shako/.autoxtrabackup/autoxtrabackup.cn
                                  f]

  --tag TEXT                      Pass the tag string for each backup
  --show-tags                     Show backup tags and exit
  -v, --verbose                   Be verbose (print to console)
  -lf, --log-file TEXT            Set log file  [default: /home/shako/.autoxtr
                                  abackup/autoxtrabackup.log]

  -l, --log, --log-level [DEBUG|INFO|WARNING|ERROR|CRITICAL]
                                  Set log level  [default: INFO]
  --log-file-max-bytes INTEGER    Set log file max size in bytes  [default:
                                  1073741824]

  --log-file-backup-count INTEGER
                                  Set log file backup count  [default: 7]
  --help                          Print help message and exit.

If you think, CLI is not for you. We have experimental feature where you can start API server and take backups using API call(ATTENTION: FastAPI involved)

sudo `which autoxtrabackup` --run-server
INFO:     Started server process [30238]
INFO:     Waiting for application startup.
app started
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:5555 (Press CTRL+C to quit)

$ curl -X POST http://127.0.0.1:5555/backup
{"result":"Successfully finished the backup process"}

For the rest please read the full documentation.

Development:

Current major version is >= 2.0 - so if you want to help, please do changes on this branch and then kindly send PR :) I also encourage you to upgrade from older version as the code base fully updated. Do you have an idea, question please open an issue.

Read full documentation here:

MySQL-AutoXtrabackup documentation!

mysql-autoxtrabackup's People

Contributors

Stargazers

Watchers

mysql-autoxtrabackup's Issues

When using non-root --user in bck.cnf, it will fail to continue

If you edit, bck.cnf configuration file and change --user=root for another user, it will fail as follows:

  Creating archive folder+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+
  Created+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
  Something is wrong with your user name or password!!!
  Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-FAILED

The main reason is hard-coded user name.
Will fix it to get user name from config file.

Error if you have single quote in password like 'Pas12345

I am using Mysql 5.7.16
Password parameter in /etc/bck.cnf defined as below.
mysql_password='Pas12345

Command

./autoxtrabackup --backup -v -l DEBUG

Error log.

[root@server bin]# ./autoxtrabackup --backup -v -l DEBUG
<pid.PidFile object at 0x7f9b7fc33048> entering setup
<pid.PidFile object at 0x7f9b7fc33048> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f9b7fc33048> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Running mysqladmin command -> /usr/bin/mysqladmin --defaults-file=/etc/my.cnf --user=root --password='Pas12345 status --host=127.0.0.1 --port=3306
<pid.PidFile object at 0x7f9b7fc33048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "./autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 509, in all_backup
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/general_conf/check_env.py", line 384, in check_all_env
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/general_conf/check_env.py", line 36, in check_mysql_uptime
  File "/opt/Python-3.5.3/lib/python3.5/shlex.py", line 273, in split
    return list(lex)
  File "/opt/Python-3.5.3/lib/python3.5/shlex.py", line 263, in __next__
    token = self.get_token()
  File "/opt/Python-3.5.3/lib/python3.5/shlex.py", line 90, in get_token
    raw = self.read_token()
  File "/opt/Python-3.5.3/lib/python3.5/shlex.py", line 166, in read_token
    raise ValueError("No closing quotation")
ValueError: No closing quotation
<pid.PidFile object at 0x7f9b7fc33048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

Then I enclosed password with double quotes like below.

mysql_password="'Pas12345"

As you can see I got error during connecting to mysql instance.

[root@rtpdbs bin]# ./autoxtrabackup --backup -v -l DEBUG
<pid.PidFile object at 0x7f10fab07048> entering setup
<pid.PidFile object at 0x7f10fab07048> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f10fab07048> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Running mysqladmin command -> /usr/bin/mysqladmin --defaults-file=/etc/my.cnf --user=root --password="'My162636@" status --host=127.0.0.1 --port=3306
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
mysqladmin: connect to server at '127.0.0.1' failed
error: 'Host '127.0.0.1' is not allowed to connect to this MySQL server'
Server is NOT Up+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
<pid.PidFile object at 0x7f10fab07048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f10fab07048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
[root@rtpdbs bin]# vi /etc/bck.conf
[root@rtpdbs bin]# ./autoxtrabackup --backup -v -l DEBUG
<pid.PidFile object at 0x7f09a1ef9048> entering setup
<pid.PidFile object at 0x7f09a1ef9048> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f09a1ef9048> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Running mysqladmin command -> /usr/bin/mysqladmin --defaults-file=/etc/my.cnf --user=root --password="'Pass12345" status --host=localhost --port=3306
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory does not exist.+-+-+-+-+-+-+-+-+-+-+-+-OK
Creating full backup directory...+-+-+-+-+-+-+-+-+-++-+-+-+-+OK
Created+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Increment directory does not exist.+-+-+-+-+-+-+-+-+-++-+-+-+OK
Creating increment backup directory.+-+-+-+-+-+-+-+-+-++-+-+-OK
Created
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
###############################################################
#You have no backups : Taking very first Full Backup! - - - - #
###############################################################
Trying to flush logs
Log flushing completed
The following backup command will be executed /usr/bin/xtrabackup --defaults-file=/etc/my.cnf --user=root --password='"'Pas12345"'  --target-dir=/backup/full/2017-02-17_11-51-26 --backup --host=localhost --port=3306
Starting /usr/bin/xtrabackup
FULL BACKUP FAILED!
/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/bin/sh: -c: line 1: syntax error: unexpected end of file
<pid.PidFile object at 0x7f09a1ef9048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f09a1ef9048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

But tool trying to take backup and failed. Because it is adding another single quotes.

Enable debug output of running commands.

Currently the full output is not shown by logger as:

2017-04-04 11:32:29 DEBUG    Trying to decrypt backup
2017-04-04 11:32:29 DEBUG    Running decrypt command -> /home/shahriyar.rzaev/Percona_Xtrabackups/xb_2.4/usr/local/xtrabackup/bin/xtrabackup --decrypt=AES256 --encrypt-key='VVTBwgM4UhwkTTV98fhuj+D1zyWoA89K' --target-dir=/home/shahriyar.rzaev/backup_dir/ps_5.6_master/full/2017-04-04_10-22-11
2017-04-04 11:32:32 DEBUG    0404 11:32:32 completed OK!
2017-04-04 11:32:32 DEBUG    Decrypted!
2017-04-04 11:32:32 DEBUG    Trying to decompress backup
2017-04-04 11:32:32 DEBUG    Running decompress command -> /home/shahriyar.rzaev/Percona_Xtrabackups/xb_2.4/usr/local/xtrabackup/bin/xtrabackup --decompress=TRUE --target-dir=/home/shahriyar.rzaev/backup_dir/ps_5.6_master/full/2017-04-04_10-22-11
2017-04-04 11:32:40 DEBUG    0404 11:32:40 completed OK!
2017-04-04 11:32:40 DEBUG    Decompressed
2017-04-04 11:32:40 DEBUG    Running prepare command -> /home/shahriyar.rzaev/Percona_Xtrabackups/xb_2.4/usr/local/xtrabackup/bin/xtrabackup --prepare --apply-log-only --target-dir=/home/shahriyar.rzaev/backup_dir/ps_5.6_master/full/2017-04-04_10-22-11
2017-04-04 11:32:43 DEBUG    0404 11:32:43 completed OK!

It will be better to show the full output of running command with some check or by default.

"IndexError: list index out of range"

[root@c-two-mysql MySQL-AutoXtraBackup]# autoxtrabackup --help
Usage: autoxtrabackup [OPTIONS]

Options:
  --prepare  Prepare/recover backups.
  --backup   Take full and incremental backups.
  --partial  Recover specified table (partial recovery).
  --version  Version information.
  --help     Show this message and exit.

[root@c-two-mysql MySQL-AutoXtraBackup]# autoxtrabackup --backup
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.4.3/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.0', 'console_scripts', 'autoxtrabackup')()
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/mysql_autoxtrabackup-1.0-py3.4.egg/autoxtrabackup.py", line 34, in all_procedure
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/mysql_autoxtrabackup-1.0-py3.4.egg/master_backup_script/backuper.py", line 25, in __init__
  File "/root/.pyenv/versions/3.4.3/lib/python3.4/site-packages/mysql_autoxtrabackup-1.0-py3.4.egg/general_conf/generalops.py", line 13, in __init__
IndexError: list index out of range

Same error with Python 3.3.2.

Prepare new full documentation - first version

The necessary documentation is available here:

http://mysql-autoxtrabackup.readthedocs.io/en/latest/index.html

Why in Mac I'm getting this frequent error ? ERROR! MySQL server PID file could not be found!

After all these years , it's 2016 now ! And yet I need to face issue with MYSQL bugs and errors . When will MYSQL be stable completely ?

Bismoy.Murasing$ mysql.server restart
ERROR! MySQL server PID file could not be found!
Starting MySQL
.......................................^C

Improve error/debug messaging; i.e logging mechanism and messages should be changed

The missing thing with logging that there is no clear indication of message types.
For eg, if it is WARN, ERROR or DEBUG message or not.
The messages should be displayed like:
[timestamp][DEBUG][Original message]

Enable --dry-run option for only showing commands to be executed

It is helpful to have --dry-run option for only showing commands which are going to be executed.
It will eliminate the need to run backup to take backup and prepare commands.
Under some circumstances we only need to have those commands on our hand to fill reports or to show to somebody.

Add support for removing original files after decompression and decrypting backups

With new, upcoming PXB release -> 2.4.6 and 2.3.7,
There will be new option --remove-original to remove .qp, .xbcrypt and
.qp.xbcrypt files after decryption and decompression.

For this there can be new option inside bck.conf indicating:
remove_original = TRUE/FALSE or 0/1
If enabled add --remove-original option for decompress/decrypt command

Handle the KeyErrors if the specified config file is not valid: KeyError: 'MySQL'

The result:

sudo /home/sh/REPOS/python3_5_3/bin/autoxtrabackup --defaults_file=/tmp/fff.cnf --backup -v -l DEBUG
<pid.PidFile object at 0x7f3e6acda868> entering setup
<pid.PidFile object at 0x7f3e6acda868> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f3e6acda868> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f3e6acda868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/sh/REPOS/python3_5_3/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.4', 'console_scripts', 'autoxtrabackup')()
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/autoxtrabackup.py", line 76, in all_procedure
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/master_backup_script/backuper.py", line 35, in __init__
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/general_conf/generalops.py", line 19, in __init__
  File "/usr/lib/python3.5/configparser.py", line 956, in __getitem__
    raise KeyError(key)
KeyError: 'MySQL'
<pid.PidFile object at 0x7f3e6acda868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

why are you guys not allowing root as my password ? why this restriction suddenly ? UPDATE mysql.user SET authentication_string=password('root') where User='root';

mysql> UPDATE mysql.user SET authentication_string=password('1111') where User='root';
ERROR 1819 (HY000): Your password does not satisfy the current policy requirements

Update README and add sample run results

Should update the README and add sample run results for:

backup
prepare
partial
Functionalities

Error with MariaDB server while taking incremental backup

Related BUG report is -> https://bugs.launchpad.net/percona-xtrabackup/+bug/1444541
Workaround is adding _--incremental-force-scan_ option, while taking incremental backup if server is MariaDB.
Fix should be applied soon.
Error message:

    innobackupex: Created backup directory /home/MySQL-AutoXtraBackup/backup_dir/inc/2015-08-30_12-33-43

      150830 12:33:43 innobackupex: Starting ibbackup with command: xtrabackup --defaults-file="/etc /my.cnf" --defaults-group="mysqld" --backup --suspend-at-end --target-dir=/home/MySQL-AutoXtraBackup/backup_dir/inc/2015-08-30_12-33-43 --innodb_log_file_size="5242880" --innodb_data_file_path="ibdata1:10M:autoextend" --tmpdir=/var/tmp --extra-lsndir='/var/tmp' --incremental-basedir='/home/MySQL-AutoXtraBackup/backup_dir/full/2015-08-30_12-33-25' --suspend-at-start
     innobackupex: Waiting for ibbackup (pid=3885) to suspend
     innobackupex: Suspend file '/home/MySQL-AutoXtraBackup/backup_dir/inc/2015-08-30_12-33-43/xtrabackup_suspended_1'

    xtrabackup version 2.2.12 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 8726828)
    incremental backup from 1597945 is enabled.
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to /var/lib/mysql
    xtrabackup: open files limit requested 0, set to 1024
xtrabackup: using the following InnoDB configuration:
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 2
xtrabackup: innodb_log_file_size = 5242880
>> log scanned up to (1597945)
xtrabackup: Generating a list of tablespaces
xtrabackup: Creating suspend file '/home/MySQL-AutoXtraBackup/backup_dir/inc/2015-08-30_12-33-43/xtrabackup_suspended_1' with pid '3885'

150830 12:33:44 innobackupex: Continuing after ibbackup has suspended
DBD::mysql::db do failed: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CHANGED_PAGE_BITMAPS' at line 1 at /usr/bin/innobackupex line 3052.
innobackupex: got a fatal error with the following stacktrace: at /usr/bin/innobackupex line 3055.
 main::mysql_query('HASH(0x2bb12c8)', 'FLUSH NO_WRITE_TO_BINLOG CHANGED_PAGE_BITMAPS') called at /usr/bin/innobackupex line 1978
 main::backup() called at /usr/bin/innobackupex line 1609
innobackupex: Error:
Error executing 'FLUSH NO_WRITE_TO_BINLOG CHANGED_PAGE_BITMAPS': DBD::mysql::db do failed: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CHANGED_PAGE_BITMAPS' at line 1 at /usr/bin/innobackupex line 3052.
150830 12:33:44 innobackupex: Waiting for ibbackup (pid=3885) to finish

Update README files to reflect new documentation

There are unneeded README files and they are should be removed or changed.

error while using --partial

[root@emin home]# autoxtrabackup --partial
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
Type Database name: lkgfd
Type Table name: sdfnl
Sorry, There is no such Database or Table in backup directory
Or maybe table storage engine is not InnoDB
Traceback (most recent call last):
  File "/usr/local/bin/autoxtrabackup", line 9, in <module>
load_entry_point('mysql-autoxtrabackup==1.0', 'console_scripts', 'autoxtrabackup')()
  File "/usr/local/lib/python3.3/site-packages/click-4.0-py3.3.egg/click/core.py", line 664, in __call__
return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.3/site-packages/click-4.0-py3.3.egg/click/core.py", line 644, in main
rv = self.invoke(ctx)
  File "/usr/local/lib/python3.3/site-packages/click-4.0-py3.3.egg/click/core.py", line 837, in invoke
return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.3/site-packages/click-4.0-py3.3.egg/click/core.py", line 464, in invoke
return callback(*args, **kwargs)
  File "/usr/local/lib/python3.3/site-packages/mysql_autoxtrabackup-1.0-py3.3.egg/autoxtrabackup.py", line 39, in all_procedure
  File "/usr/local/lib/python3.3/site-packages/mysql_autoxtrabackup-1.0-py3.3.egg/partial_recovery/partial.py", line 323, in final_actions
TypeError: 'bool' object is not subscriptable

Add support for passing extra arguments to backup and prepare commands

For eg, if somebody needs something like,
--binlog-info=ON --galera-info etc options to be passed to xtrabackup.

It can be passed to xtrabackup for eg,
[Backup]
xtra_options='--binlog-info=ON --galera-info'

And also any other option can be passed as well in that manner.

Version check is not correct for partial recovery section

The result of run:

/opt/Python-3.5.3/bin/autoxtrabackup --partial -v -l DEBUG
<pid.PidFile object at 0x7f7d5f166868> entering setup
<pid.PidFile object at 0x7f7d5f166868> create pidfile: /var/run/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f7d5f166868> check pidfile: /var/run/MySQL-AutoXtraBackup/autoxtrabackup.pid
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
Type Database name: dbtest
Type Table name: t1
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
MySQL per table space enabled!
Your MySQL server is not supported
MySQL version must be >= 5.6
<pid.PidFile object at 0x7f7d5f166868> closing pidfile: /var/run/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f7d5f166868> closing pidfile: /var/run/MySQL-AutoXtraBackup/autoxtrabackup.pid

In fact mysql version is:

mysql --version
mysql  Ver 14.14 Distrib 5.7.16-10, for Linux (x86_64) using  6.2

Expand AutoXtraBackup to use backup compression

Implement compressed backup feature.
Available options to test and add:

# xtrabackup --help | grep -i compress
xtrabackup version 2.4.5 based on MySQL server 5.7.13 Linux (x86_64) (revision id: e41c0be)
  --compress[=name]   Compress individual backup files using the specified
                      compression algorithm. Currently the only supported
                      i.e. the one used when --compress is used without an
  --compress-threads=# 
                      Number of threads for parallel data compression. The
  --compress-chunk-size=# 
                      Size of working buffer(s) for compression threads in
  --decompress        Decompresses all files with the .qp extension in a backup
                      previously made with the --compress option.
compress                          (No default value)
compress-threads                  1
compress-chunk-size               65536
decompress                        FALSE

Full backup failed, but tool try to take incremental backup.

I tried several times to take full backup, but failed with login error.
Now backup tool tells that I have fresh Full backup and it will try to get incremental based on full.

[root@server inc]# autoxtrabackup --backup -v -l DEBUG
<pid.PidFile object at 0x7f03c8d58048> entering setup
<pid.PidFile object at 0x7f03c8d58048> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f03c8d58048> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Running mysqladmin command -> /usr/bin/mysqladmin --defaults-file=/etc/my.cnf --user=yusif --password=Pas12345@ status --host=localhost --port=3306
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
################################################################
You have a full backup that is less than 86400 seconds old. - -#
We will take an incremental one based on recent Full Backup - -#
################################################################
Installed Server is MySQL, will continue as usual.
The following backup command will be executed /usr/bin/xtrabackup --defaults-file=/etc/my.cnf --user=yusif --password='My162636@'  --target-dir=/backup/inc/2017-02-17_12-41-33 --incremental-basedir=/backup/inc/2017-02-17_12-21-32 --backup --host=localhost --port=3306
INCREMENT BACKUP FAILED!
xtrabackup: Error: cannot open /backup/inc/2017-02-17_12-21-32/xtrabackup_checkpoints
xtrabackup: error: failed to read metadata from /backup/inc/2017-02-17_12-21-32/xtrabackup_checkpoints
<pid.PidFile object at 0x7f03c8d58048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f03c8d58048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

After final actions, implement the option to install this tool via pip

Maybe it is better idea to share this project via pip.
It will ease installing process as well.

raise KeyError(key) KeyError: 'mkdir_command'

/opt/Python-3.5.3/bin/autoxtrabackup --backup
Traceback (most recent call last):
  File "/opt/Python-3.5.3/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.1', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.1-py3.5.egg/autoxtrabackup.py", line 54, in all_procedure
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.1-py3.5.egg/general_conf/generalops.py", line 75, in __init__
  File "/opt/Python-3.5.3/lib/python3.5/configparser.py", line 1230, in __getitem__
    raise KeyError(key)
KeyError: 'mkdir_command'

Add new option to project --defaults-file=

At this moment the project reading /etc/bck.conf from start.
It is quite common that somebody will test different mysql versions.
They will have multiple configs, it will be great if they could pass it to autoxtrabackup:
The sample call will be something like:

autoxtrabackup --defualts-file=/home/mysql_5_6_33_backup.conf --backup -v -l DEBUG

For now the only config file is, by default located in -> /etc/bck.conf

autoxtrabackup --partial traceback error

when mysql not installed or down, autoxtrabackup --partial gives traceback error:

lpiexam@lpiexam-VirtualBox:/$ autoxtrabackup --partial
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/mysql_connector_python-2.0.3-                       py3.4.egg/mysql/connector/network.py", line 469, in open_connection
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.0', 'console_scripts', 'autoxtrabackup')()
  File "/usr/local/lib/python3.4/dist-packages/click-4.0-py3.4.egg/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/click-4.0-py3.4.egg/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.4/dist-packages/click-4.0-py3.4.egg/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.4/dist-packages/click-4.0-py3.4.egg/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-    py3.4.egg/autoxtrabackup.py", line 38, in all_procedure
  File "/usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-py3.4.egg/partial_recovery/partial.py", line 32, in __init__
  File "/usr/local/lib/python3.4/dist-packages/mysql_connector_python-2.0.3-py3.4.egg/mysql/connector/__init__.py", line 162, in connect
  File "/usr/local/lib/python3.4/dist-packages/mysql_connector_python-2.0.3-py3.4.egg/mysql/connector/connection.py", line 129, in __init__
  File "/usr/local/lib/python3.4/dist-packages/mysql_connector_python-2.0.3-py3.4.egg/mysql/connector/connection.py", line 454, in connect
  File "/usr/local/lib/pyt6hon3.4/dist-packages/mysql_connector_python-2.0.3-py3.4.egg/mysql/connector/connection.py", line 417, in _open_connection
  File "/usr/local/lib/python3.4/dist-packages/mysql_connector_python-2.0.3-    py3.4.egg/mysql/connector/network.py", line 472, in open_connection
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on 'localhost:3306'     (111 Connection refused)
Exception ignored in: <bound method PartialRecovery.__del__ of     <partial_recovery.partial.PartialRecovery object at 0x7f711aaae2b0>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-    py3.4.egg/partial_recovery/partial.py", line 37, in __del__
AttributeError: 'PartialRecovery' object has no attribute 'cnx'

AttributeError: 'Logger' object has no attribute 'deberrorug'

In default conf file start stop commands is as below.

start_mysql_command=service mysql start
stop_mysql_command=service mysql stop

if we have different service name than mysql we will get error like this

2017-02-27 11:34:09 DEBUG    ####################################################################################################
2017-02-27 11:34:09 DEBUG    Shutting Down MySQL server: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -#
2017-02-27 11:34:09 DEBUG    ####################################################################################################
2017-02-27 11:34:12 DEBUG    <pid.PidFile object at 0x7f5e01c95ea8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/opt/Python-3.5.3/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.6', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/backup_prepare/prepare.py", line 666, in prepare_backup_and_copy_back
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/backup_prepare/prepare.py", line 624, in copy_back
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/backup_prepare/prepare.py", line 463, in shutdown_mysql
AttributeError: 'Logger' object has no attribute 'deberrorug'
2017-02-27 11:34:12 DEBUG    <pid.PidFile object at 0x7f5e01c95ea8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

Add support for xbstream usage

https://www.percona.com/doc/percona-xtrabackup/2.4/xbstream/xbstream.html
https://www.percona.com/doc/percona-xtrabackup/2.4/xbstream/xbstream.html
https://www.percona.com/doc/percona-xtrabackup/2.4/innobackupex/streaming_backups_innobackupex.html

We should find a way to implement this functionality. It will ease to test and use it directly.

bck.conf - keep commented only 1 pair start/stop commands

If we forget to comment below strings for start and stop commands, restore will stuck. In my case tool was trying to start MariaDB server. Can we keep only one pair of start - stop commands uncommented in config file?

[Commands]
start_mysql_command=service mysqld start
stop_mysql_command=service mysqld stop
#systemd_start_mysql=systemctl start mysqld.service
#systemd_stop_mysql=systemctl stop mysqld.service
#systemd_start_mariadb=systemctl start mariadb.service
#systemd_stop_mariadb=systemctl stop mariadb.service
#mariadb_cluster_bootstrap=service mysql bootstrap
chown_command=chown -R mysql:mysql

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
####################################################################################################
Shutting Down MySQL server: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -#
####################################################################################################
Stopping mysqld:                                           [  OK  ]
####################################################################################################
Moving MySQL datadir to /tmp/mysql: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -#
####################################################################################################
Emptied /tmp/mysql directory ...
Moved datadir to /tmp/mysql ...
Creating an empty data directory ...
Datadir is Created! ...
####################################################################################################
Copying Back Already Prepared Final Backup: - - - - - - - - - - - - - - - - - - - - - - - - - - - -#
####################################################################################################
####################################################################################################
Data copied back successfully! - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
####################################################################################################
####################################################################################################
New copied-back data now owned by specified user! - - - - - - - - - - - - - - - - - - - - - - - - - - -#
####################################################################################################
####################################################################################################
Starting MySQL/MariaDB server:
####################################################################################################

configparser.ParsingError: Source contains parsing errors: '/home/sh/txt.ff'

If config file contains something like:

sh@sh-ubuntu:~$ cat txt.ff
[MySQL]
kshfskjrksjnr

The result:

/home/sh/REPOS/python3_5_3/bin/autoxtrabackup --backup --defaults_file=/home/sh/txt.ff -v -l DEBUG
<pid.PidFile object at 0x7f63b116d868> entering setup
<pid.PidFile object at 0x7f63b116d868> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f63b116d868> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f63b116d868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/sh/REPOS/python3_5_3/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.4', 'console_scripts', 'autoxtrabackup')()
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/autoxtrabackup.py", line 76, in all_procedure
    b = Backup(config=defaults_file)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/master_backup_script/backuper.py", line 35, in __init__
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/general_conf/generalops.py", line 17, in __init__
  File "/usr/lib/python3.5/configparser.py", line 696, in read
    self._read(fp, filename)
  File "/usr/lib/python3.5/configparser.py", line 1107, in _read
    raise e
configparser.ParsingError: Source contains parsing errors: '/home/sh/txt.ff'
	[line  2]: 'ffhjjf\n'
<pid.PidFile object at 0x7f63b116d868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

Implement partial database/table backup

This is another upcoming plan to add into tool.
There can be some option to backup specified database and specified table.
It will be great to hear some other suggestions based on this.

Expand AutoXtraBackup to create encrypted backups

Available options:

[root@tokubackup ~]# xtrabackup --help | grep -i encrypt
xtrabackup version 2.4.5 based on MySQL server 5.7.13 Linux (x86_64) (revision id: e41c0be)
  --encrypt=name      Encrypt individual backup files using the specified
                      encryption algorithm.
  --encrypt-key=name  Encryption key to use.
  --encrypt-key-file=name 
                      File which contains encryption key to use.
  --encrypt-threads=# Number of threads for parallel data encryption. The
  --encrypt-chunk-size=# 
                      Size of working buffer(S) for encryption threads in
                      backup previously made with --encrypt option.

Decrypt:

[root@tokubackup ~]# xtrabackup --help | grep -i decrypt
xtrabackup version 2.4.5 based on MySQL server 5.7.13 Linux (x86_64) (revision id: e41c0be)
  --decrypt=name      Decrypts all files with the .xbcrypt extension in a
decrypt                           NONE

Failed with --defaults-file option; should add check for config file prior passing to parser

The result of run:

sh@sh-ubuntu:~$ /home/sh/REPOS/python3_5_3/bin/autoxtrabackup --defaults_file=fgdjgdfg --backup
Traceback (most recent call last):
  File "/home/sh/REPOS/python3_5_3/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.4', 'console_scripts', 'autoxtrabackup')()
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/autoxtrabackup.py", line 70, in all_procedure
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/master_backup_script/backuper.py", line 474, in all_backup
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/general_conf/check_env.py", line 386, in check_all_env
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/general_conf/check_env.py", line 23, in check_mysql_uptime
AttributeError: 'CheckEnv' object has no attribute 'mysqladmin'

define mysql ipaddress config with bind-address [my.cnf]

Hi,

Thanks for repo, i think you should define variable for remote ipaddress with bind-address in my.cnf config file , now this is a static '127.0.0.1',

https://github.com/ShahriyarR/MySQL-AutoXtraBackup/blob/master/master_backup_script/backuper.py#L106

/bin/sh: /usr/bin/mysqladmin: No such file or directory; seems to be the config file was parsed incorrectly

Result:

[shahriyar.rzaev@qaserver-04 ~]$ autoxtrabackup --backup -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf 
<pid.PidFile object at 0x7fc7247864a8> entering setup
<pid.PidFile object at 0x7fc7247864a8> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7fc7247864a8> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup/Innobackupex exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
################################################################
You have a full backup that is less than 86400 seconds old. - -#
We will take an incremental one based on recent Full Backup - -#
################################################################
mysqladmin ver command Failed
/bin/sh: /usr/bin/mysqladmin: No such file or directory
<pid.PidFile object at 0x7fc7247864a8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 593, in all_backup
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 336, in inc_backup
UnboundLocalError: local variable 'args' referenced before assignment
<pid.PidFile object at 0x7fc7247864a8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

Missing unit tests from project

Currently, I have not developed any kind of unit tests.
This is lacking crucial thing, which should be addressed ASAP.

Installing on debian

I'm trying to install on debian(jessie)
The installation stops on:

Installed /usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-py3.4.egg
Processing dependencies for mysql-autoxtrabackup==1.0
Searching for mysql-connector-python>=2.0.2
Reading https://pypi.python.org/simple/mysql-connector-python/
No local packages or download links found for mysql-connector-python>=2.0.2
error: Could not find suitable distribution for Requirement.parse('mysql-connector-python>=2.0.2')

I have already installed mysql-connector-python from :
https://dev.mysql.com/doc/connector-python/en/connector-python-installation-binary.html

Why is it trying ...
Reading https://pypi.python.org/simple/mysql-connector-python/

instead of:
https://pypi.python.org/pypi/mysql-connector-python/2.0.2

Can someone help me with this ?

configparser.MissingSectionHeaderError: File contains no section headers.

If the file contains dummy data inside.
For eg:

sh@sh-ubuntu:~$ cat txt.ff
kshfskjrksjnr

Result:

/home/sh/REPOS/python3_5_3/bin/autoxtrabackup --backup --defaults_file=/home/sh/txt.ff -v -l DEBUG
<pid.PidFile object at 0x7f431935c868> entering setup
<pid.PidFile object at 0x7f431935c868> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f431935c868> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7f431935c868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/sh/REPOS/python3_5_3/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.4', 'console_scripts', 'autoxtrabackup')()
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/autoxtrabackup.py", line 76, in all_procedure
    b = Backup(config=defaults_file)
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/master_backup_script/backuper.py", line 35, in __init__
  File "/home/sh/REPOS/python3_5_3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.4-py3.5.egg/general_conf/generalops.py", line 17, in __init__
  File "/usr/lib/python3.5/configparser.py", line 696, in read
    self._read(fp, filename)
  File "/usr/lib/python3.5/configparser.py", line 1077, in _read
    raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '/home/sh/txt.ff', line: 1
'sdjhvfsdmfbf\n'
<pid.PidFile object at 0x7f431935c868> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

mysqladmin failed to connect with SSL user

$ autoxtrabackup --backup -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.6_master_bck.conf
<pid.PidFile object at 0x7feb8c5934a8> entering setup
<pid.PidFile object at 0x7feb8c5934a8> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7feb8c5934a8> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Warning: Using a password on the command line interface can be insecure.
/home/shahriyar.rzaev/Percona_Servers/5.6.35/bin/mysqladmin: connect to server at '127.0.0.1' failed
error: 'Access denied for user 'jeffrey'@'localhost' (using password: YES)'
Server is NOT Up+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
<pid.PidFile object at 0x7feb8c5934a8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7feb8c5934a8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

The reason for this seems to be that mysqladmin is not reading SSL things under [client] from MySQL's config file.
The MySQL's config file should be passed as --defaults-file should be passed to mysqladmin.

UnboundLocalError: local variable 'args' referenced before assignment

sudo autoxtrabackup --backup -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf
<pid.PidFile object at 0x7fdc15fa50e8> entering setup
<pid.PidFile object at 0x7fdc15fa50e8> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
<pid.PidFile object at 0x7fdc15fa50e8> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Running mysqladmin command -> /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf --user=jeffrey --password=msandbox status --host=localhost --port=20192
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
################################################################
You have a full backup that is less than 86400 seconds old. - -#
We will take an incremental one based on recent Full Backup - -#
################################################################
mysqladmin ver command Failed
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)'
Check that mysqld is running and that the socket: '/tmp/mysql.sock' exists!
<pid.PidFile object at 0x7fdc15fa50e8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 594, in all_backup
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 327, in inc_backup
UnboundLocalError: local variable 'args' referenced before assignment
<pid.PidFile object at 0x7fdc15fa50e8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

Missing License

What license is this project released under?

Partial recovery MySQL connection problem

Failed to use --partial option as it is failed to connect to MySQL instance:

[shahriyar.rzaev@qaserver-04 ~]$ sudo autoxtrabackup --partial -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf
2017-02-20 16:03:17 DEBUG    <pid.PidFile object at 0x7fb14bf14048> entering setup
2017-02-20 16:03:17 DEBUG    <pid.PidFile object at 0x7fb14bf14048> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-20 16:03:17 DEBUG    <pid.PidFile object at 0x7fb14bf14048> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-20 16:03:17 ERROR    MySQL server connection problem. 
2017-02-20 16:03:17 ERROR    Check if your MySQL server is up and running or if there is no firewall blocking connection.
2017-02-20 16:03:17 ERROR    2003: Can't connect to MySQL server on '127.0.0.1:3306' (111 Connection refused)
2017-02-20 16:03:17 DEBUG    <pid.PidFile object at 0x7fb14bf14048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-20 16:03:17 DEBUG    <pid.PidFile object at 0x7fb14bf14048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

AttributeError: 'Backup' object has no attribute 'archive_dir'; if backup is timedout

$ sudo autoxtrabackup --backup -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf
2017-02-27 08:58:11 DEBUG    <pid.PidFile object at 0x7fee2db010e8> entering setup
2017-02-27 08:58:11 DEBUG    <pid.PidFile object at 0x7fee2db010e8> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-27 08:58:11 DEBUG    <pid.PidFile object at 0x7fee2db010e8> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-27 08:58:11 DEBUG    Running mysqladmin command -> /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf --user=jeffrey --password=msandbox status --host=localhost --port=20192
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
2017-02-27 08:58:12 DEBUG    Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
2017-02-27 08:58:12 DEBUG    /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
2017-02-27 08:58:12 DEBUG    /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
2017-02-27 08:58:12 DEBUG    MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
2017-02-27 08:58:12 DEBUG    Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
2017-02-27 08:58:12 DEBUG    Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
2017-02-27 08:58:12 DEBUG    Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
2017-02-27 08:58:12 DEBUG    Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
2017-02-27 08:58:12 DEBUG    Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
2017-02-27 08:58:12 DEBUG    ################################################################
2017-02-27 08:58:12 DEBUG    Your full backup is timeout : Taking new Full Backup!- - - - - #
2017-02-27 08:58:12 DEBUG    ################################################################
2017-02-27 08:58:15 DEBUG    <pid.PidFile object at 0x7fee2db010e8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.6', 'console_scripts', 'autoxtrabackup')()
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.6-py3.5.egg/master_backup_script/backuper.py", line 549, in all_backup
AttributeError: 'Backup' object has no attribute 'archive_dir'
2017-02-27 08:58:15 DEBUG    <pid.PidFile object at 0x7fee2db010e8> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

ValueError: unconverted data remains

if we rename full backup directory name or create dummy folder under full directory.
Renamed folder

[root@server full]# ll
total 4
drwxr-xr-x 6 root root 4096 Feb 17 14:47 2017-02-17_14-23-01_old

Backup command

mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
<pid.PidFile object at 0x7f6b16345048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/opt/Python-3.5.3/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 540, in all_backup
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 53, in last_full_backup_date
  File "/opt/Python-3.5.3/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/opt/Python-3.5.3/lib/python3.5/_strptime.py", line 346, in _strptime
    data_string[found.end():])
ValueError: unconverted data remains: _old
<pid.PidFile object at 0x7f6b16345048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

With dummy folder

mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-OK
Full Backup directory exists.+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+OK
Increment directory exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-OK
<pid.PidFile object at 0x7f0bab490048> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/opt/Python-3.5.3/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 160, in all_procedure
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 540, in all_backup
  File "/opt/Python-3.5.3/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/master_backup_script/backuper.py", line 53, in last_full_backup_date
  File "/opt/Python-3.5.3/lib/python3.5/_strptime.py", line 510, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/opt/Python-3.5.3/lib/python3.5/_strptime.py", line 343, in _strptime
    (data_string, format))
ValueError: time data 'test' does not match format '%Y-%m-%d_%H-%M-%S'

1819 (HY000): Your password does not satisfy the current policy requirements

I have a problem with running autoxtrabackup --backup option. it throws an error indicating something related to password policy: 1819 (HY000): Your password does not satisfy the current policy requirements

Here is full output:

[root@ELSHAD_ThinkPad ~]# autoxtrabackup --backup
Warning: Using a password on the command line interface can be insecure.
Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
/usr/bin/mysql exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-OK
/usr/bin/mysqladmin exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
MySQL configuration file exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+OK
Xtrabackup/Innobackupex exists+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-OK
Main backup directory does not exist+-+-+-+-+-+-+-+-+-++-+-+-+-
Creating Main Backup folder+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+
Created+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Full Backup directory does not exist.+-+-+-+-+-+-+-+-+-+-+-+-OK
Creating full backup directory...+-+-+-+-+-+-+-+-+-++-+-+-+-+OK
Created+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-OK
Increment directory does not exist.+-+-+-+-+-+-+-+-+-++-+-+-+OK
Creating increment backup directory.+-+-+-+-+-+-+-+-+-++-+-+-OK
Created
Creating Test User (test_backup)+-+-+-+-+-+-+-+-+-++-+-+-+-+-OK
1819 (HY000): Your password does not satisfy the current policy requirements
Check status: STATUS+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-FAILED

Unexpected behaviour after mysqlfrm run; seems to be the wrong create statement extracted

[shahriyar.rzaev@qaserver-04 MySQL-AutoXtraBackup]$ sudo autoxtrabackup --partial -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf
2017-02-21 16:52:47 DEBUG    <pid.PidFile object at 0x7f35b5893f48> entering setup
2017-02-21 16:52:47 DEBUG    <pid.PidFile object at 0x7f35b5893f48> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-21 16:52:47 DEBUG    <pid.PidFile object at 0x7f35b5893f48> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-21 16:52:47 DEBUG    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
Type Database name: dbtest
Type Table name: t1
2017-02-21 16:52:50 DEBUG    Running mysqladmin command -> /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf --user=jeffrey --password=msandbox status --host=localhost --port=20192
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
2017-02-21 16:52:50 DEBUG    Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
2017-02-21 16:52:50 DEBUG    Checking if innodb_file_per_table is enabled
2017-02-21 16:52:50 DEBUG    Will run following command ->/home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysql --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf -ujeffrey --password=msandbox --host=localhost --port=20192 -e "select @@global.innodb_file_per_table"
2017-02-21 16:52:50 DEBUG    innodb_file_per_table is enabled!
2017-02-21 16:52:50 DEBUG    Checking MySQL version
2017-02-21 16:52:50 DEBUG    MySQL Version is, 5.7.17-11-log
2017-02-21 16:52:50 DEBUG    You have correct version of MySQL
2017-02-21 16:52:50 DEBUG    Checking if database exists in MySQL
2017-02-21 16:52:50 DEBUG    Database exists!
2017-02-21 16:52:50 DEBUG    Checking if table exists in MySQL Server
2017-02-21 16:52:50 DEBUG    Table does not exist in MySQL Server.
2017-02-21 16:52:50 DEBUG    You can not restore table, with not existing tablespace file(.ibd)!
2017-02-21 16:52:50 DEBUG    We will try to extract table create statement from .frm file, from backup folder
2017-02-21 16:52:50 DEBUG    Running mysqlfrm tool
2017-02-21 16:52:50 DEBUG    Success
2017-02-21 16:52:50 ERROR    Failed to create table from .frm file!
2017-02-21 16:52:50 ERROR    /bin/sh: dbtest: command not found
/bin/sh: t1: command not found
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1064 (42000) at line 2: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(
  uid=0(root) gid=0(root) groups=0(root) int(11) NOT NULL 
) ENGINE=InnoDB' at line 1
2017-02-21 16:52:50 DEBUG    <pid.PidFile object at 0x7f35b5893f48> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-21 16:52:50 DEBUG    <pid.PidFile object at 0x7f35b5893f48> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

[shahriyar.rzaev@qaserver-04 MySQL-AutoXtraBackup]$ sudo autoxtrabackup --partial -v -l DEBUG --defaults_file=/home/shahriyar.rzaev/AutoXtrabackup_Configs/ps_5.7_master_bck.conf
2017-02-21 15:16:38 DEBUG    <pid.PidFile object at 0x7f661748ff48> entering setup
2017-02-21 15:16:38 DEBUG    <pid.PidFile object at 0x7f661748ff48> create pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-21 15:16:38 DEBUG    <pid.PidFile object at 0x7f661748ff48> check pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
2017-02-21 15:16:38 DEBUG    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
Type Database name: dbtest
Type Table name: t1
2017-02-21 15:16:50 DEBUG    Running mysqladmin command -> /home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysqladmin --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf --user=jeffrey --password=msandbox status --host=localhost --port=20192
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
2017-02-21 15:16:50 DEBUG    Server is Up and running+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+OK
2017-02-21 15:16:50 DEBUG    Checking if innodb_file_per_table is enabled
2017-02-21 15:16:50 DEBUG    Will run following command ->/home/shahriyar.rzaev/Percona_Servers/5.7.17/bin/mysql --defaults-file=/home/shahriyar.rzaev/sandboxes/rsandbox_Percona-Server-5_7_17/master/my.sandbox.cnf -ujeffrey --password=msandbox --host=localhost --port=20192 -e "select @@global.innodb_file_per_table"
2017-02-21 15:16:51 DEBUG    innodb_file_per_table is enabled!
2017-02-21 15:16:51 DEBUG    Checking MySQL version
2017-02-21 15:16:51 DEBUG    MySQL Version is, 5.7.17-11-log
2017-02-21 15:16:51 DEBUG    You have correct version of MySQL
2017-02-21 15:16:51 DEBUG    Checking if database exists in MySQL
2017-02-21 15:16:51 ERROR    There is no such database!
2017-02-21 15:16:51 ERROR    Create Specified Database in MySQL Server, before restoring single table
We can create it for you do you want? (yes/no): yes
2017-02-21 15:16:57 DEBUG    Creating specified database
2017-02-21 15:16:57 DEBUG    dbtest database created
2017-02-21 15:16:57 DEBUG    Checking if table exists in MySQL Server
2017-02-21 15:16:57 ERROR    Table does not exist in MySQL Server.
2017-02-21 15:16:57 ERROR    You can not restore table, with not existing tablespace file(.ibd)!
2017-02-21 15:16:57 ERROR    We will try to extract table create statement from .frm file, from backup folder
2017-02-21 15:16:57 DEBUG    Running mysqlfrm tool
2017-02-21 15:16:57 ERROR    Failed
2017-02-21 15:16:57 ERROR    /bin/sh: /usr/bin/mysqlfrm: No such file or directory
2017-02-21 15:16:57 DEBUG    <pid.PidFile object at 0x7f661748ff48> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid
Traceback (most recent call last):
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/bin/autoxtrabackup", line 11, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.5', 'console_scripts', 'autoxtrabackup')()
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/click-6.7-py3.5.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/autoxtrabackup.py", line 168, in all_procedure
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/partial_recovery/partial.py", line 374, in final_actions
  File "/home/shahriyar.rzaev/virtualenvs/py_3_5_3_xtrabackup/lib/python3.5/site-packages/mysql_autoxtrabackup-1.4.5-py3.5.egg/partial_recovery/partial.py", line 182, in check_table_exists_on_mysql
TypeError: expected string or bytes-like object
2017-02-21 15:16:57 DEBUG    <pid.PidFile object at 0x7f661748ff48> closing pidfile: /tmp/MySQL-AutoXtraBackup/autoxtrabackup.pid

AttributeError: 'GeneralClass' object has no attribute 'pid_dir'

/opt/Python-3.3.2/bin/autoxtrabackup --backup
Traceback (most recent call last):
  File "/opt/Python-3.3.2/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.4.1', 'console_scripts', 'autoxtrabackup')()
  File "/opt/Python-3.3.2/lib/python3.3/site-packages/click-6.7-py3.3.egg/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/Python-3.3.2/lib/python3.3/site-packages/click-6.7-py3.3.egg/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/Python-3.3.2/lib/python3.3/site-packages/click-6.7-py3.3.egg/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/Python-3.3.2/lib/python3.3/site-packages/click-6.7-py3.3.egg/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/Python-3.3.2/lib/python3.3/site-packages/mysql_autoxtrabackup-1.4.1-py3.3.egg/autoxtrabackup.py", line 57, in all_procedure
AttributeError: 'GeneralClass' object has no attribute 'pid_dir'

Replace innobackupex with xtrabackup

For recent versions innobackupex is symlink to xtrabackup.
And any new additions are only added to xtrabackup.
That's why it is crucial to replace innobackupex usage with xtrabackup.

AttributeError: 'Backup' object has no attribute ‘myuseroption’

Reported using mail:

I’m trying to configure and run MySQL-AutoXtraBackup on Percona Cluster.

At this stage, I have already installed AutoXtraBackup:

sudo /opt/Python-3.3.2/bin/autoxtrabackup --help
Usage: autoxtrabackup [OPTIONS]

Options:
  --prepare  Prepare/recover backups.
  --backup   Take full and incremental backups.
  --partial  Recover specified table (partial recovery).
  --version  Version information.
  --help     Show this message and exit.

sudo /opt/Python-3.3.2/bin/autoxtrabackup --version
Developed by Shahriyar Rzayev from Azerbaijan MUG(http://mysql.az)
Link : https://github.com/ShahriyarR/MySQL-AutoXtraBackup
Email: [email protected]
Based on Percona XtraBackup: https://launchpad.net/percona-xtrabackup
MySQL-AutoXtrabackup Version 1.0

When I run the first Backup:

# autoxtrabackup --backup
Traceback (most recent call last):
  File "/usr/local/bin/autoxtrabackup", line 9, in <module>
    load_entry_point('mysql-autoxtrabackup==1.0', 'console_scripts', 'autoxtrabackup')()
  File "/usr/local/lib/python3.4/dist-packages/click-6.2-py3.4.egg/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/click-6.2-py3.4.egg/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.4/dist-packages/click-6.2-py3.4.egg/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.4/dist-packages/click-6.2-py3.4.egg/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-py3.4.egg/autoxtrabackup.py", line 34, in all_procedure
  File "/usr/local/lib/python3.4/dist-packages/mysql_autoxtrabackup-1.0-py3.4.egg/master_backup_script/backuper.py", line 32, in __init__
AttributeError: 'Backup' object has no attribute ‘myuseroption’

I would appreciate very much your help, to figure it out what the problem is.

Thank you