twindb / backup Goto Github PK
View Code? Open in Web Editor NEWTwinDB Backup
Home Page: https://twindb-backup.readthedocs.io/en/stable/
License: Other
TwinDB Backup
Home Page: https://twindb-backup.readthedocs.io/en/stable/
License: Other
By default the tool copied the backups to SSH destination.
However it produces errors complaining about missing aws creds:
An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: The AWS Access Key Id you provided does not exist in our records.
make_bucket failed: s3://twindb-backups/ An error occurred (InvalidAccessKeyId) when calling the CreateBucket operation: The AWS Access Key Id you provided does not exist in our records.
I would like to use this software locally only, but am not able to deploy SSH keys for connecting back to localhost in a particular scenario.
Since it's already implemented to have a local copy saved, would it be possible to enable a local destination as a primary location (rather than s3
or ssh
)?
I did start hacking on this, and hope to submit a PR, but wanted to open it up with the off-chance that a better coder might tackle it before I can.
[root@storage ~]# twindb-backup status { "daily": {}, "hourly": {}, "monthly": {}, "weekly": {}, "yearly": {} }
[root@mysql ~]# twindb-backup ls { .... }
Configs of twindb-backup are cloned.
When storing a local copy twindb-backup
should create directory for a host it takes backup from.
Let's say keep_local_copy=/var/log/mysql/backup
, hostname is ip-10-10-10-10
.
If /var/log/mysql/backup/ip-10-10-10-10
doesn't exist the script will crash with traceback:
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.9.3', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in _call_
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.9.3-py2.7.egg/twindb_backup/cli.py", line 53, in backup
run_backup_job(cfg, run_type)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.9.3-py2.7.egg/twindb_backup/backup.py", line 140, in run_backup_job
raise err
IOError: [Errno 2] No such file or directory: '/var/log/mysql/backup//ip-10-10-10-10/status'
List of backups:
[root@ip-10-0-52-101 centos]# twindb-backup ls | grep ip-10-0-52-101/hourly/mysql
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_18_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_19_00_05.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_20_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_21_00_05.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_22_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_23_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_00_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_01_17_14.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_01_28_40.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_02_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_03_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_04_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_05_00_05.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_06_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_07_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_08_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_09_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_10_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_11_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_12_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_13_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_14_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_15_00_03.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_16_00_04.xbstream.gz
s3://***/ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_17_00_04.xbstream.gz
From status file:
[root@ip-10-0-52-101 centos]# twindb-backup status | grep ip-10-0-52-101/hourly/mysql
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_19_00_05.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_22_00_04.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-01_23_00_03.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_01_17_14.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_01_28_40.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_03_00_04.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_05_00_05.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_07_00_03.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_09_00_04.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_10_00_03.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_12_00_04.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_14_00_04.xbstream.gz": {
"ip-10-0-52-101/hourly/mysql/mysql-2018-04-02_16_00_04.xbstream.gz": {
During upgrade or after uninstalling the package there are orphaned files left
$ sudo dpkg -i /tmp/twindb-backup_2.10.0-1_amd64.deb
(Reading database ... 86480 files and directories currently installed.)
Preparing to unpack .../twindb-backup_2.10.0-1_amd64.deb ...
Unpacking twindb-backup (2.10.0-1) over (2.8.2-1) ...
twindb-backup has been uninstalled!
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/MySQLdb/constants': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/MySQLdb': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/docs/bcdoc': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/docs': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/chardet': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/urllib3/packages/ssl_match_hostname': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/urllib3/packages': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/urllib3/util': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/urllib3/contrib': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages/urllib3': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests/packages': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored/requests': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore/vendored': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg/botocore': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/botocore-1.4.93-py2.7.egg': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/psutil-5.0.1-py2.7-linux-x86_64.egg/psutil': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/psutil-5.0.1-py2.7-linux-x86_64.egg': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.2-py2.7.egg/twindb_backup/destination': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.2-py2.7.egg/twindb_backup/source': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.2-py2.7.egg/twindb_backup': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.2-py2.7.egg': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/boto3-1.4.3-py2.7.egg/boto3/docs': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/boto3-1.4.3-py2.7.egg/boto3/resources': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/boto3-1.4.3-py2.7.egg/boto3': Directory not empty
dpkg: warning: unable to delete old directory '/opt/twindb-backup/embedded/lib/python2.7/site-packages/boto3-1.4.3-py2.7.egg': Directory not empty
Setting up twindb-backup (2.10.0-1) ...
Skipping setting up of /etc/twindb/twindb-backup.cfg as it already exists.
Skipping setting up of /etc/cron.d/twindb-backup as it already exists.
Thank you for installing twindb-backup!
clone doesn't log properly
Traceback (most recent call last):
File "/opt/rh/python27/root/usr/lib64/python2.7/logging/__init__.py", line 859, in emit
msg = self.format(record)
File "/opt/rh/python27/root/usr/lib64/python2.7/logging/__init__.py", line 732, in format
return fmt.format(record)
File "/opt/rh/python27/root/usr/lib64/python2.7/logging/__init__.py", line 471, in format
record.message = record.getMessage()
File "/opt/rh/python27/root/usr/lib64/python2.7/logging/__init__.py", line 335, in getMessage
msg = msg % self.args
TypeError: %d format: a number is required, not str
Logged from file clone.py, line 126
twindb-backup
depends on aws streaming feature that was introduces around 1.4.4.
If aws is older than 1.4.4 twindb-backup will fail with something like
2017-01-25 19:58:02,733: ERROR: base_destination._save():71: aws s3 cp - s3://<BUCKET>/ip-172-30-0-245/daily/files/_etc-2017-01-25_19_58_01.tar.gz exited with error code 1
2017-01-25 19:58:02,734: INFO: base_destination._save():73: [Errno 2] No such file or directory: '/etc/twindb/-'
Completed 1 part(s) with ... file(s) remaining
Solution might be either check aws version and notify that it's too old or maybe just use boto3?
Ubuntu Xenial is an LTS release. We should build twindb-backup
packages for it.
Hi,
I need to store my backups in google cloud storage, which is also popular. I can implement it, if you point me to amazon-s3 files.
The script doesn't gracefully handle full disk errors. Also, it probably leaves backup copies in tmp dir.
# twindb-backup restore mysql --dst /var/log/mysql/mysql_3307_tmp --cache /var/log/mysql/cache s3://.../.../mysql-2017-05-22_09_30_02.xbstream.gz
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.12.0', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.12.0-py2.7.egg/twindb_backup/cli.py", line 115, in restore_mysql
restore_from_mysql(cfg, backup_copy, dst, Cache(cache))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.12.0-py2.7.egg/twindb_backup/restore.py", line 365, in restore_from_mysql
restore_from_mysql_incremental(stream, dst_dir, config)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.12.0-py2.7.egg/twindb_backup/restore.py", line 190, in restore_from_mysql_incremental
inc_dir = tempfile.mkdtemp()
File "/opt/twindb-backup/embedded/lib/python2.7/tempfile.py", line 325, in mkdtemp
dir = gettempdir()
File "/opt/twindb-backup/embedded/lib/python2.7/tempfile.py", line 269, in gettempdir
tempdir = _get_default_tempdir()
File "/opt/twindb-backup/embedded/lib/python2.7/tempfile.py", line 212, in _get_default_tempdir
("No usable temporary directory found in %s" % dirlist))
IOError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root']
After twindb-backup runs a verification job it leaves temporary files in /tmp
# twindb-backup verify mysql ${backup_copy} --dst /mnt/data/verify
...
[root@ip-10-0-1-151 tmp]# ll
total 8
-rw-r--r--. 1 root root 57 Jan 12 09:00 percona-version-check
drwx------. 11 root root 4096 Jan 12 19:08 tmpas6x1T
If xtrabackup isn't present on the system twindb-backup
will exit with error 2016-12-05 02:30:55,823: ERROR: restore.restore_from_mysql_full():68: Failed to unarchive s3://****/daily/mysql/mysql-2016-12-01_01_13_55.xbstream.gz: [Errno 2] No such file or directory
twindb-backup restore mysql s3://****/daily/mysql/mysql-2016-12-01_01_13_55.xbstream.gz
Install Percona Xtrabackup
# yum install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm
# yum install percona-xtrabackup
List xtrabackup as a dependency
During uninstall the twindb-backup RPM fails to delete a cron config.
# yum remove twindb-backup-2.15.3-1.x86_64
...
Running transaction
Erasing : twindb-backup-2.15.3-1.x86_64 1/1
warning: file /opt/twindb-backup/twindb-backup.cron: remove failed: No such file or directory
twindb-backup has been uninstalled!
Verifying : twindb-backup-2.15.3-1.x86_64
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/cli.py", line 135, in backup
binlogs_only=binlogs_only
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/backup.py", line 371, in run_backup_job
backup_everything(run_type, cfg, binlogs_only=binlogs_only)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/backup.py", line 317, in backup_everything
backup_binlogs(run_type, config)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/backup.py", line 261, in backup_binlogs
dst.delete(copy.key + ".gz")
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/destination/ssh.py", line 114, in delete
self.execute_command(cmd)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/destination/ssh.py", line 234, in execute_command
background=background
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.16.1-py2.7.egg/twindb_backup/ssh/client.py", line 135, in execute
% (cmd, exit_code))
twindb_backup.ssh.exceptions.SshClientException: rm localhost.localdomain/binlog/mysql-bin.000001.gz exited with code 1
It seems that SshClient uses invalid path to binlog backup. Similar problem was in previous version of util with mysql backups.
the script waits for wsrep_local_recv_queue go to zero, but even though it touches zero the script doesn't exit until it times out.
# twindb-backup --debug backup hourly
...
2016-11-02 15:43:47,981: DEBUG: backup.disable_wsrep_desync():110: Timeout expired. Disabling wsrep_desync
...
Python: 2.7.12
I am now getting this error on one server running 2.14.1
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.14.1', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/cli.py", line 72, in backup
run_backup_job(cfg, run_type)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/backup.py", line 254, in run_backup_job
backup_everything(run_type, cfg)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/backup.py", line 207, in backup_everything
backup_mysql(run_type, config)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/backup.py", line 148, in backup_mysql
dst.status(status)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/destination/base_destination.py", line 122, in status
return self._write_status(status)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.1-py2.7.egg/twindb_backup/destination/ssh.py", line 137, in _write_status
proc = Popen(cmd)
File "/opt/twindb-backup/embedded/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/opt/twindb-backup/embedded/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 7] Argument list too long
There is a small bug in the source:
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.14.0-py2.7.egg/twindb_backup/source/mysql_source.py", line 447
I get a python error:
AttributeError: 'int' object has no attribute 'lower'
Lines 446 & 447:
return (str(row['wsrep_on']).lower() == "1" or
row['wsrep_on'].lower() == 'on')
Should be fixed by doing this:
return (str(row['wsrep_on']).lower() == "1" or
str(row['wsrep_on']).lower() == 'on')
Can you please update the code with this change? Otherwise, I have to edit this file everytime it updates.
Hi guys,
I'm reading the docs here: https://twindb-backup.readthedocs.io/
I see that full/incremental types of backups are supported, but I can't find in the docs where and how should I set such type.
Thanks.
Best, Michael Rikmas
On the destination a MySQL files should change owner to mysql.
As long as only root can do it, chown should be run with sudo
self._ssh_client.execute("sudo chown -R mysql %s" % datadir)
TwinDB Backup can keep few latest copies on the same server it takes backups from. The feature is enabled by keep_local_path
options.
The local copies have their own retention policy managed by [retention_local]
section in /etc/twindb/twindb-backup.conf
.
There is a report from a user that the local retention policy in version 2.15.0 is not respected and the tool fills local disk with backup copies.
A workaround to this is to disable keep_local_path
.
We need to reproduce the bug, write a unit test (possibly integration test as well) and fix the bug.
Inetrgration test in travis failed bc S3 raised an exception. See log
2.14.1
twindb-backup clone mysql
doesn't work with hostnames.
twindb-backup --config ~/.twindb-backup.cfg clone mysql hostname1 hostname2
2017-10-13 11:56:28,003: ERROR: clone.clone_mysql():85: Failed to stop MySQL on <twindb_backup.destination.ssh.Ssh object at 0x7f5ae5a30d50>
Traceback (most recent call last):
File "/home/okuzminskyi/.virtualenvs/backup/bin/twindb-backup", line 11, in <module>
load_entry_point('twindb-backup', 'console_scripts', 'twindb-backup')()
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/okuzminskyi/src/backup/twindb_backup/cli.py", line 231, in clone_mysql_backup
netcat_port=netcat_port)
File "/home/okuzminskyi/src/backup/twindb_backup/clone.py", line 107, in clone_mysql
src.clone_config(dst)
File "/home/okuzminskyi/src/backup/twindb_backup/source/remote_mysql_source.py", line 66, in clone_config
self._save_cfg(dst, cfg_path)
File "/home/okuzminskyi/src/backup/twindb_backup/source/remote_mysql_source.py", line 71, in _save_cfg
server_id = struct.unpack("!I", socket.inet_aton(dst.host))[0]
socket.error: illegal IP address string passed to inet_aton
Custom SSH port is passed as port
option in the ssh
section of the config.
Support both options ssh_port
and port
for the destination, describe the in documentation.
https://twindb-backup.readthedocs.io/en/master/usage.html#ssh-settings
Start/stop MySQL with sudo.
def _mysql_service(dst, action):
"""Start or stop MySQL service
:param dst: Destination server
:type dst: Ssh
:param action: string start or stop
:type action: str
"""
for service in ['mysqld', 'mysql']:
try:
return dst.execute_command(
"service %s %s" % (service, action),
quiet=True
)
except SshClientException:
pass
raise TwinDBBackupError('Failed to %s MySQL on %r'
% (action, dst))
When travis builds packages from a master branch it uploads them to package cloud.
centos/6
is wrong name. it should be el/6
clone mysql
attempts to stop MySQL on the destination. It shouldn't.
Instead it should check if MySQL is running and raise an exception if it is.
Upgrading the package from a prior version does not execute postinst
[root@stoage ~]# twindb-backup --debug verify mysql /backup_path/hostname/hourly/mysql/mysql-2018-09-02_13_00_03.xbstream.gz --dst /tmp/verify
2018-09-02 11:46:06,165: DEBUG: cli.verify():235: Restore: <ConfigParser.ConfigParser instance at 0x7f873e8ffc68>
2018-09-02 11:46:06,165: DEBUG: cli.verify_mysql():250: mysql: <ConfigParser.ConfigParser instance at 0x7f873e8ffc68>
2018-09-02 11:46:06,166: INFO: restore.restore_from_mysql():307: Restoring /backup_path/hostname/hourly/mysql/mysql-2018-09-02_13_00_03.xbstream.gz in /tmp/verify
2018-09-02 11:46:06,167: DEBUG: configuration.get_destination():29: Destination in the config ssh
2018-09-02 11:46:06,299: DEBUG: client.execute():110: Executing command: bash -c 'if test -s /backupbgpercona/nbxp003.nbxp.local/status; then echo exists; else echo not_exists; fi'
2018-09-02 11:46:06,403: DEBUG: client.get_remote_handlers():163: Try to get remote handlers: cat /backupbgpercona/nbxp003.nbxp.local/status
2018-09-02 11:46:06,411: DEBUG: mysql_status.candidate_parent():127: Looking a parent candidate for hourly run
2018-09-02 11:46:06,411: DEBUG: mysql_status.candidate_parent():133: Checking 17 hourly copies
2018-09-02 11:46:06,411: DEBUG: mysql_status.candidate_parent():133: Checking 2 daily copies
2018-09-02 11:46:06,411: DEBUG: mysql_status.candidate_parent():138: Found parent MySQLCopy(hostname/daily/mysql/mysql-2018-09-02_00_00_03.xbstream.gz)
2018-09-02 11:46:06,411: DEBUG: restore.restore_from_mysql():367: Full parent copy is hostname/daily/mysql/mysql-2018-09-02_00_00_03.xbstream.gz
2018-09-02 11:46:06,412: DEBUG: restore.restore_from_mysql_full():73: Not decrypting the stream
2018-09-02 11:46:06,413: DEBUG: base._revert_stream():50: Running gunzip -c
2018-09-02 11:46:06,415: DEBUG: restore._extract_xbstream():117: Running /opt/twindb-backup/embedded/bin/xbstream -x
2018-09-02 11:46:06,415: DEBUG: restore._extract_xbstream():118: Working directory: /tmp/verify
2018-09-02 11:46:06,416: DEBUG: restore._extract_xbstream():119: Xbstream binary: /opt/twindb-backup/embedded/bin/xbstream
2018-09-02 11:46:06,549: DEBUG: ssh._write_to_pipe():131: Executing cat hostname/daily/mysql/mysql-2018-09-02_00_00_03.xbstream.gz
2018-09-02 11:46:06,553: DEBUG: ssh._write_to_pipe():137: closing channel
2018-09-02 11:46:06,556: ERROR: base._revert_stream():59: gunzip -c exited with non-zero code.
2018-09-02 11:46:06,556: ERROR: base._revert_stream():60:
gzip: stdin: unexpected end of file
Line:
2018-09-02 11:46:06,549: DEBUG: ssh._write_to_pipe():131: Executing cat hostname/daily/mysql/mysql-2018-09-02_00_00_03.xbstream.gz
Expected:
/backup_path/hostname/daily/mysql/mysql-2018-09-02_00_00_03.xbstream.gz
[root@storage ~]# twindb-backup --debug backup hourly
2018-09-01 18:30:06,219: DEBUG: backup.run_backup_job():269: hourly
2018-09-01 18:30:06,698: DEBUG: backup.set_open_files_limit():205: Setting max files limit to 1048577
2018-09-01 18:30:06,698: DEBUG: _init_.get_directories_to_backup():120: Directories to backup []
2018-09-01 18:30:06,704: DEBUG: configuration.get_destination():29: Destination in the config ssh
2018-09-01 18:30:07,061: DEBUG: client.execute():110: Executing command: bash -c 'if test -s /backup_dir/hostname/status; then echo exists; else echo not_exists; fi'
2018-09-01 18:30:07,380: DEBUG: client.get_remote_handlers():163: Try to get remote handlers: cat /backup_dir/hostname/status
2018-09-01 18:30:07,471: DEBUG: mysql_status.candidate_parent():127: Looking a parent candidate for hourly run
2018-09-01 18:30:07,472: DEBUG: mysql_status.candidate_parent():133: Checking 45 hourly copies
2018-09-01 18:30:07,472: DEBUG: mysql_status.candidate_parent():138: Found parent MySQLCopy(hostname/hourly/mysql/mysql-2018-08-28_17_00_02.xbstream.gz)
2018-09-01 18:30:07,473: DEBUG: mysql_status.candidate_parent():127: Looking a parent candidate for hourly run
2018-09-01 18:30:07,473: DEBUG: mysql_status.candidate_parent():133: Checking 45 hourly copies
2018-09-01 18:30:07,473: DEBUG: mysql_status.candidate_parent():138: Found parent MySQLCopy(hostname/hourly/mysql/mysql-2018-08-28_17_00_02.xbstream.gz)
2018-09-01 18:30:07,473: DEBUG: backup.backup_mysql():141: Creating source
{'backup_type': 'incremental', 'parent_lsn': 97968913006, 'dst': <twindb_backup.destination.ssh.Ssh object at 0x7f3034d98e10>, 'xtrabackup_binary': u'/opt/twindb-backup/embedded/bin/xtrabackup'}
2018-09-01 18:30:07,474: DEBUG: backup._backup_stream():56: keep_local_path is not present in the config file
2018-09-01 18:30:07,474: DEBUG: base_source._get_name():55: Suffix = xbstream.gz
2018-09-01 18:30:07,474: DEBUG: ssh.execute_command():217: Executing: mkdir -p "/backup_dir/hostname/hourly/mysql"
2018-09-01 18:30:07,771: DEBUG: client.execute():110: Executing command: mkdir -p "/backup_dir/hostname/hourly/mysql"
2018-09-01 18:30:08,100: DEBUG: client.get_remote_handlers():163: Try to get remote handlers: cat - > /backup_dir/hostname/hourly/mysql/mysql-2018-09-01_18_30_07.xbstream.gz
2018-09-01 18:30:08,131: DEBUG: mysql_source.is_galera():407: Galera is not supported or not enabled
2018-09-01 18:30:08,131: DEBUG: mysql_source.get_stream():138: Running /opt/twindb-backup/embedded/bin/xtrabackup --defaults-file=/etc/twindb/.my.cnf --stream=xbstream --host=127.0.0.1 --backup --target-dir . --incremental-basedir . --incremental-lsn=97968913006
2018-09-01 18:30:08,135: DEBUG: mysql_source.is_galera():407: Galera is not supported or not enabled
2018-09-01 18:30:08,135: DEBUG: mysql_source.get_stream():144: Running /opt/twindb-backup/embedded/bin/xtrabackup --defaults-file=/etc/twindb/.my.cnf --stream=xbstream --host=127.0.0.1 --backup --target-dir . --incremental-basedir . --incremental-lsn=97968913006
Please add support for MariaBackup (drop in replacement for xtrabackup) to support MariaDB 10.2+
Doing a backup: twindb-backup backup daily
# twindb-backup backup daily
Traceback (most recent call last):
File "/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.17.0', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.17.0-py2.7.egg/twindb_backup/cli.py", line 130, in backup
binlogs_only=binlogs_only
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.17.0-py2.7.egg/twindb_backup/backup.py", line 367, in run_backup_job
binlogs_only=binlogs_only
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.17.0-py2.7.egg/twindb_backup/backup.py", line 307, in backup_everything
backup_binlogs(run_type, twindb_config)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.17.0-py2.7.egg/twindb_backup/backup.py", line 216, in backup_binlogs
last_binlog=last_copy.name if last_copy else None
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.17.0-py2.7.egg/twindb_backup/backup.py", line 268, in binlogs_to_backup
cursor.execute("SHOW BINARY LOGS")
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/cursors.py", line 170, in execute
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/cursors.py", line 328, in _query
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/connections.py", line 517, in query
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/connections.py", line 732, in _read_query_result
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/connections.py", line 1075, in read
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/connections.py", line 684, in _read_packet
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/protocol.py", line 220, in check_error
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/PyMySQL-0.9.3-py2.7.egg/pymysql/err.py", line 109, in raise_mysql_exception
pymysql.err.InternalError: (1381, u'You are not using binary logging')
A backup job failed because of xtrabackup error:
170103 21:39:05 Executing FLUSH NO_WRITE_TO_BINLOG TABLES...
170103 21:39:05 Executing FLUSH TABLES WITH READ LOCK...
Error: failed to execute query FLUSH TABLES WITH READ LOCK: Lock wait timeout exceeded; try restarting transaction
The tool didn't save a status record (which is expected), but it also left a backup copy in S3 which is in unknown state, I don't know if it's usable or not.
The tool should exit gracefully and do some clean-up.
I was trying to install twins
After the install process ran, I ran yum install twindb-backup (per the docs)
Here is the output:
# yum install twindb-backup
Loaded plugins: priorities, security, ulninfo
Setting up Install Process
536 packages excluded due to repository priority protections
No package twindb-backup available.
Error: Nothing to do
Is anyone having this issue? Seems that the RPM repo is empty.
innobackupex needs to read MySQL owned files - e.g. should be run as root.
Implement mysqldump as a backups source.
Run mysqldump only if full backup is scheduled.
Command line --version
option should be present.
# twindb-backup --version
Error: no such option: --version
twindb-backup
sets wsrep_desync
to ON when it takes a backup from Percona XtraDB Cluster. if the backup job fails on whatever reason wsrep_desync
if not reverted to OFF.
Next twindb-backup
run will produce a warning:
/usr/lib/python2.7/site-packages/twindb_backup/backup.py:49: Warning: 'wsrep_desync' is already ON.
c.execute("set global wsrep_desync=ON")
Integration test test-integration-backup-ssh fails on vagrant.
[root@master1 twindb_backup]# make test-integration-backup-ssh
py.test -xsv tests/integration/backup/ssh
========================================================================================== test session starts ===========================================================================================
platform linux2 -- Python 2.7.5, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 -- /usr/bin/python2
cachedir: .cache
rootdir: /twindb_backup, inifile:
plugins: cov-2.5.1
collected 2 items
tests/integration/backup/ssh/test_backup.py::test_backup 2017-11-30 17:26:35,831: INFO: conftest.container_network():75: Created subnet 172.16.0.0/16
2017-11-30 17:26:35,832: DEBUG: conftest.container_network():76: {u'Warning': u'', u'Id': u'e7fb42258171763826220923b33b614247c8729dbd71020be2a8a72eb9fd6304'}
2017-11-30 17:27:05,322: DEBUG: conftest.get_container():108: {'EndpointsConfig': {'test_network': {'IPAMConfig': {'IPv4Address': '172.16.3.1'}}}}
2017-11-30 17:27:05,625: INFO: conftest.get_container():120: Created container {'ip': '172.16.3.1', u'Id': u'7839c484bd203b2ae3f43fe50cbfc7538cb003c136f53f7a3772c9a85cbf8747', u'Warnings': None}
2017-11-30 17:27:06,337: INFO: conftest.get_container():123: Started {'ip': '172.16.3.1', u'Id': u'7839c484bd203b2ae3f43fe50cbfc7538cb003c136f53f7a3772c9a85cbf8747', u'Warnings': None}
PASSED
tests/integration/backup/ssh/test_restore.py::test_restore CMD : twindb-backup --debug --config /tmp/pytest-of-root/pytest-2/test_restore0/twindb-backup.cfg ls | grep etc | grep tmp | awk -F/ '{ print $NF}' | sort | tail -1
2017-11-30 18:27:25,797: DEBUG: ls.list_available_backups():26: Running find /var/backup/local -type f
2017-11-30 18:27:25,802: DEBUG: configuration.get_destination():28: Destination in the config ssh
Traceback (most recent call last):
File "/bin/twindb-backup", line 11, in <module>
load_entry_point('twindb-backup', 'console_scripts', 'twindb-backup')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/twindb_backup/twindb_backup/cli.py", line 104, in list_backups
list_available_backups(cfg)
File "/twindb_backup/twindb_backup/ls.py", line 34, in list_available_backups
for copy in dst.find_files(dst.remote_path, run_type):
File "/twindb_backup/twindb_backup/destination/ssh.py", line 136, in find_files
with self._ssh_client.get_remote_handlers(cmd) as (_, cout, _):
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/twindb_backup/twindb_backup/ssh/client.py", line 104, in get_remote_handlers
with self._shell() as shell:
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/twindb_backup/twindb_backup/ssh/client.py", line 55, in _shell
raise SshClientException(err)
twindb_backup.ssh.exceptions.SshClientException: [Errno None] Unable to connect to port 22 on 172.16.3.1
CMD : twindb-backup --debug --config /tmp/pytest-of-root/pytest-2/test_restore0/twindb-backup.cfg ls | grep /tmp/backup | grep | head -1
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
2017-11-30 18:27:28,439: DEBUG: ls.list_available_backups():26: Running find /var/backup/local -type f
2017-11-30 18:27:28,444: DEBUG: configuration.get_destination():28: Destination in the config ssh
Traceback (most recent call last):
File "/bin/twindb-backup", line 11, in <module>
load_entry_point('twindb-backup', 'console_scripts', 'twindb-backup')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/twindb_backup/twindb_backup/cli.py", line 104, in list_backups
list_available_backups(cfg)
File "/twindb_backup/twindb_backup/ls.py", line 34, in list_available_backups
for copy in dst.find_files(dst.remote_path, run_type):
File "/twindb_backup/twindb_backup/destination/ssh.py", line 136, in find_files
with self._ssh_client.get_remote_handlers(cmd) as (_, cout, _):
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/twindb_backup/twindb_backup/ssh/client.py", line 104, in get_remote_handlers
with self._shell() as shell:
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/twindb_backup/twindb_backup/ssh/client.py", line 55, in _shell
raise SshClientException(err)
twindb_backup.ssh.exceptions.SshClientException: [Errno None] Unable to connect to port 22 on 172.16.3.1
Url:
Command output:
2017-11-30 18:27:30,024: INFO: cli.restore_file():180: No backup copy specified. Choose one from below:
2017-11-30 18:27:30,024: INFO: ls.list_available_backups():22: Local copies:
2017-11-30 18:27:30,095: INFO: ls.list_available_backups():33: hourly copies:
[Errno None] Unable to connect to port 22 on 172.16.3.1
(<class 'twindb_backup.ssh.exceptions.SshClientException'>, SshClientException(NoValidConnectionsError(None, 'Unable to connect to port 22 on 172.16.3.1'),), <traceback object at 0x3eec6c8>)
FAILED
================================================================================================ FAILURES ================================================================================================
______________________________________________________________________________________________ test_restore ______________________________________________________________________________________________
backup_server = {'Id': 'c7347718b4496d64fc38d2221956657bbc82d6bf75d0ce1e1869024acedda739', 'Warnings': None, 'ip': '172.16.3.1'}
config_content_ssh = '\n[source]\nbackup_dirs={BACKUP_DIR}\nbackup_mysql=no\n\n[destination]\nbackup_destination=ssh\nkeep_local_path=/var/...\n\n[intervals]\n\n# Run intervals\n\nrun_hourly=yes\nrun_daily=yes\nrun_weekly=yes\nrun_monthly=yes\nrun_yearly=yes\n'
tmpdir = local('/tmp/pytest-of-root/pytest-2/test_restore0')
@pytest.mark.timeout(600)
def test_restore(backup_server, config_content_ssh, tmpdir):
my_cnf = tmpdir.join('.my.cnf')
my_cnf.write("""
[client]
user=dba
password=qwerty
""")
config = tmpdir.join('twindb-backup.cfg')
id_rsa = tmpdir.join('id_rsa')
id_rsa.write("""-----BEGIN RSA PRIVATE KEY-----
MIIEoAIBAAKCAQEAyXxAjPShNGAedbaEtltFI6A7RlsyI+4evxTq6uQrgbJ6Hm+p
HBXshXQYXDyVjvytaM+6GKF+r+6+C+6Wc5Xz4lLO/ZiSCdPbyEgqw1JoHrgPNpc6
wmCtjJExxjzvpwSVgbZg3xOdqW1y+TyqeUkXEg/Lm4VZhN1Q/KyGCgBlWuAXoOYR
GhaNWqcnr/Wn5YzVHAx2yJNrurtKLVYVMIkGcN/6OUaPpWqKZLaXiK/28PSZ5GdT
DmxRg4W0pdyGEYQndpPlpLF4w5gNUEhVZM8hWVE29+DIW3XXVYGYchxmkhU7wrGx
xZR+k5AT+7g8VspVS8zNMXM9Z27w55EQuluNMQIBIwKCAQAzz35QIaXLo7APo/Y9
hS8JKTPQQ1YJPTsbMUO4vlRUjPrUoF6vc1oTsCOFbqoddCyXS1u9MNdvEYFThn51
flSn6WhtGJqU0BPxrChA2q0PNqTThfkqqyVQCBQdCFrhzfqPEaPhl1RtZUlzSh01
IWxVGgEn/bfu9xTTQk5aV9+MZQ2XKe4BGzpOZMI/B7ivRCcthEwMTx92opr52bre
4t7DahVLN/2Wu4lxajDzCaKXpjMuL76lFov0mZZN7S8whH5xSx1tpapHqsCAwfLL
k49lDdR8aN6oqoeK0e9w//McIaKxN2FVxD4bcuXiQTjihx+QwQOLmlHSRDKhTsYg
4Q5bAoGBAOgVZM2eqC8hNl5UH//uuxOeBKqwz7L/FtGemNr9m0XG8N9yE/K7A5iX
6EDvDyVI51IlIXdxfK8re5yxfbJ4YevenwdEZZ2O8YRrVByJ53PV9CcVeWL4p6f/
I56sYyDfXcnDTEOVYY0mCfYUfUcSb1ExpuIU4RvuQJg6tvbdxD9FAoGBAN4/pVCT
krRd6PJmt6Dbc2IF6N09OrAnLB3fivGztF5cp+RpyqZK4ve+akLoe1laTg7vNtnF
l/PZtM9v/VT45hb70MFEHO+sKvGa5Yimxkb6YCriJOcLxTysSgFHKz7v+8BqqoHi
qY4fORGwPVDv28I8jKRvcuNHendV/Rdcuk79AoGAd1t1q5NscAJzu3u4r4IXEWc1
mZzClpHROJq1AujTgviZInUu1JqxZGthgHrx2KkmggR3nIOB86/2bdefut7TRhq4
L5+Et24VzxKgSTD6sJnrR0zfV3iQvMxbdizFRBsaSoGyMWLEdHn2fo4xzMem9o6Q
VwNsdMOsMQhA1rsxuiMCgYBr8wcnIxte68jqxC1OIXKOsmnKi3RG7nSDidXF2vE1
JbCiJMGD+Hzeu5KyyLDw4rgzI7uOWKjkJ+obnMuBCy3t6AZPPlcylXPxsaKwFn2Q
MHfaUJWUyzPqRQ4AnukekdINAJv18cAR1Kaw0fHle9Ej1ERP3lxfw6HiMRSHsLJD
nwKBgCIXVhXCDaXOOn8M4ky6k27bnGJrTkrRjHaq4qWiQhzizOBTb+7MjCrJIV28
8knW8+YtEOfl5R053SKQgVsmRjjDfvCirGgqC4kSAN4A6MD+GNVXZVUUjAUBVUbU
8Wt4BxW6kFA7+Su7n8o4DxCqhZYmK9ZUhNjE+uUhxJCJaGr4
-----END RSA PRIVATE KEY-----
""")
content = config_content_ssh.format(
PRIVATE_KEY=str(id_rsa),
BACKUP_DIR="/etc",
HOST_IP=backup_server['ip']
)
config.write(content)
runner = CliRunner()
result = runner.invoke(main, [
'--debug',
'--config', str(config),
'backup', 'daily'
])
if result.exit_code != 0:
print('Command output:')
print(result.output)
print(result.exception)
print(result.exc_info)
assert result.exit_code == 0
cmd = 'twindb-backup --debug --config %s ls | grep etc | grep tmp ' \
'| awk -F/ \'{ print $NF}\' | sort | tail -1' % str(config)
print('CMD : %s' % cmd)
basename = check_output(args=cmd, shell=True).rstrip()
cmd = 'twindb-backup --debug --config %s ls ' \
'| grep /tmp/backup | grep %s | head -1' % (str(config), basename)
print('CMD : %s' %cmd)
url = check_output(cmd, shell=True).rstrip()
print('Url:')
print(url)
restore_dir = tmpdir.mkdir('restore')
runner = CliRunner()
result = runner.invoke(main,
['--config', str(config),
'restore', 'file', url,
"--dst", str(restore_dir)]
)
if result.exit_code != 0:
print('Command output:')
print(result.output)
print(result.exception)
print(result.exc_info)
> assert result.exit_code == 0
E AssertionError: assert -1 == 0
E + where -1 = <Result SshClientException(NoValidConnectionsError(None, 'Unable to connect to port 22 on 172.16.3.1'),)>.exit_code
tests/integration/backup/ssh/test_restore.py:91: AssertionError
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================= 1 failed, 1 passed in 3659.69 seconds ==================================================================================
make: *** [test-integration-backup-ssh] Error 2
[root@master1 twindb_backup]#
Looks like a regression in behavior, where taking the output of twindb-backup ls
s3 URLs results in a failed restore:
TwinDB-backup version: 2.15.10
Python version: 2.7.5
Kernel/OS: 3.10.0-862.2.3.el7.x86_64 / CentOS 7.3
root@**redactedhostname**:/tmp/mysql# twindb-backup --debug restore mysql s3://s3-bucket/**redactedhostname**/daily/mysql/mysql-2018-08-30_00_00_08.xbstream.gz
2018-09-06 02:17:09,295: DEBUG: cli.restore():177: restore: <ConfigParser.ConfigParser instance at 0x7f6707a865f0>
2018-09-06 02:17:09,296: DEBUG: cli.restore_mysql():189: mysql: <ConfigParser.ConfigParser instance at 0x7f6707a865f0>
2018-09-06 02:17:09,296: DEBUG: configuration.get_destination():29: Destination in the config s3
2018-09-06 02:17:10,977: ERROR: cli.restore_mysql():210: Copy **redactedhostname**/daily/mysql/mysql-2018-08-30_00_00_08.xbstream.gz not found
root@**redactedhostname**:/tmp/mysql# twindb-backup ls | grep s3://s3-bucket/**redactedhostname**/daily/mysql/mysql-2018-08-30_00_00_08.xbstream.gz
s3://s3-bucket/**redactedhostname**/daily/mysql/mysql-2018-08-30_00_00_08.xbstream.gz
Possibly related to #137 ?
Last known working 2.15.7
On a MySQL 5.7.23 host, while performing the hourly backup to an s3 bucket, twindb-backup hs been reporting "WARNING: mysql_source.apply_retention_policy():232: Copy **redacted host**/hourly/mysql/.... not found
Looking into this further, it appears as though this was introduced in the 2.15.8 release, and continues to exhibit the behavior through tonight's 2.15.10 release.
Looking at the debug logs, it looks as though mutliple S3 bucket URLs are being concatenated together, instead of just one into the URL string for the s3.delete
call:
from twindb-backup backup --debug hourly output:
deleting s3://bucketname/s3://bucketname
I've verified the presence of the files on the S3 bucket, but the call to s3.delete is mangled:
2018-09-04 06:49:28,081: DEBUG: mysql_source.apply_retention_policy():227: Deleting remote file s3://mysql-backups-bucket/mysql001/hourly/mysql/mysql-2018-09-01_13_00_05.xbstream.gz
2018-09-04 06:49:28,092: DEBUG: s3.delete():271: deleting s3://mysql-backups-bucket/s3://mysql-backups-bucket/mysql001/hourly/mysql/mysql-2018-09-01_13_00_05.xbstream.gz
[root@xxx ~]# twindb-backup verify mysql /path/to/hourly/mysql/mysql-2018-08-29_19_00_03.xbstream.gz --dst /tmp/verifw
2018-08-29 19:22:47,119: INFO: restore.restore_from_mysql():305: Restoring /path/to/hourly/mysql/mysql-2018-08-29_19_00_03.xbstream.gz in /tmp/verifw
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.15.7', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.7-py2.7.egg/twindb_backup/cli.py", line 239, in verify_mysql
print(verify_mysql_backup(cfg, dst, backup_copy, hostname))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.7-py2.7.egg/twindb_backup/verify.py", line 48, in verify_mysql_backup
restore_from_mysql(config, url, dst_path, tmp_dir)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.7-py2.7.egg/twindb_backup/restore.py", line 340, in restore_from_mysql
if status[key].type == "full":
AttributeError: 'NoneType' object has no attribute 'type'
Yes, some systems don't have ssh
client. As long as twindb-backup
depends on it, let's list it as a dependency.
# twindb-backup --debug ls
2017-01-14 18:43:23,388: INFO: ls.list_available_backups():11: Local copies:
2017-01-14 18:43:23,388: INFO: ls.list_available_backups():22: hourly copies:
2017-01-14 18:43:23,388: DEBUG: util.get_destination():14: Destination in the config ssh
2017-01-14 18:43:23,389: DEBUG: ssh.find_files():87: Running ssh -l root -o StrictHostKeyChecking=no -o PasswordAuthentication=no -p 22 -i /root/.ssh/id_rsa 127.0.0.1 find /tmp/backup/*/hourly -type f
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.8.1', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.1-py2.7.egg/twindb_backup/cli.py", line 60, in ls
list_available_backups(cfg)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.1-py2.7.egg/twindb_backup/ls.py", line 25, in list_available_backups
for copy in dst.find_files(dst.remote_path, run_type):
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.8.1-py2.7.egg/twindb_backup/destination/ssh.py", line 88, in find_files
proc = Popen(cmd, stdout=PIPE, stderr=PIPE)
File "/opt/twindb-backup/embedded/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/opt/twindb-backup/embedded/lib/python2.7/subprocess.py", line 1335, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
[root@9bf0f4a6a870 ~]# ssh
bash: ssh: command not found
For s3 backup copies make a command that would share the backup copy and return URL to download it.
malformed status file causes the tool crash
2018-01-23 16:44:11,617: DEBUG: backup.run_backup_job():233: daily
2018-01-23 16:44:12,473: DEBUG: backup.set_open_files_limit():169: Setting max files limit to 1048577
2018-01-23 16:44:12,474: DEBUG: __init__.get_directories_to_backup():118: Not backing up files
2018-01-23 16:44:12,474: DEBUG: configuration.get_destination():28: Destination in the config ssh
2018-01-23 16:44:12,475: DEBUG: base_source._get_name():50: Suffix = xbstream
2018-01-23 16:44:12,476: DEBUG: ssh.execute_command():266: Executing: mkdir -p "/backups_mysql/www.xxx.com/daily/mysql"
2018-01-23 16:44:12,873: DEBUG: client.execute():72: Executing mkdir -p "/backups_mysql/www.xxx.com/daily/mysql"
2018-01-23 16:44:13,350: DEBUG: mysql_source.get_stream():123: Running innobackupex --defaults-file=/etc/twindb/my.cnf --stream=xbstream --host=127.0.0.1 .
2018-01-23 16:44:13,355: DEBUG: mysql_source.get_stream():129: Running innobackupex --defaults-file=/etc/twindb/my.cnf --stream=xbstream --host=127.0.0.1 .
2018-01-23 18:02:23,974: DEBUG: mysql_source.get_stream():143: Successfully streamed innobackupex output
2018-01-23 18:02:23,975: DEBUG: mysql_source._update_backup_info():163: innobackupex error log file /tmp/tmpCi7ULD
2018-01-23 18:02:24,271: DEBUG: client.execute():72: Executing bash -c 'if test -s /backups_mysql/www.xxx.com/status; then echo exists; else echo not_exists; fi'
Traceback (most recent call last):
File "/usr/bin/twindb-backup", line 9, in <module>
load_entry_point('twindb-backup==2.15.3', 'console_scripts', 'twindb-backup')()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/click-6.7-py2.7.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/cli.py", line 82, in backup
run_backup_job(cfg, run_type, lock_file=lock_file)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/backup.py", line 235, in run_backup_job
backup_everything(run_type, cfg)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/backup.py", line 186, in backup_everything
backup_mysql(run_type, config)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/backup.py", line 119, in backup_mysql
status = prepare_status(dst, src, run_type, src_name, backup_start)
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/backup.py", line 136, in prepare_status
status = dst.status()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/destination/base_destination.py", line 120, in status
return self._read_status()
File "/opt/twindb-backup/embedded/lib/python2.7/site-packages/twindb_backup-2.15.3-py2.7.egg/twindb_backup/destination/ssh.py", line 230, in _read_status
return json.loads(base64.b64decode(stdout.read()))
File "/opt/twindb-backup/embedded/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/opt/twindb-backup/embedded/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/opt/twindb-backup/embedded/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Unterminated string starting at: line 1 column 170178 (char 170177)
A documentation page http://twindb-backup.readthedocs.io/en/master/installation.html has a broken link https://github.com/twindb/twindb_backup
$ twindb-backup --config ~/.twindb-backup.cfg clone mysql a.b.c.d e.f.g.h
2017-10-13 13:23:25,730: ERROR: clone.clone_mysql():85: Failed to stop MySQL on <twindb_backup.destination.ssh.Ssh object at 0x7f87b599dcd0>
Traceback (most recent call last):
File "/home/okuzminskyi/.virtualenvs/backup/bin/twindb-backup", line 11, in <module>
load_entry_point('twindb-backup', 'console_scripts', 'twindb-backup')()
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], **kwargs)
File "/home/okuzminskyi/.virtualenvs/backup/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/okuzminskyi/src/backup/twindb_backup/cli.py", line 231, in clone_mysql_backup
netcat_port=netcat_port)
File "/home/okuzminskyi/src/backup/twindb_backup/clone.py", line 130, in clone_mysql
binlog, position = dst_mysql.apply_backup(datadir)
File "/home/okuzminskyi/src/backup/twindb_backup/source/remote_mysql_source.py", line 166, in apply_backup
free_mem = int(stdout_.read()) * 1024
ValueError: invalid literal for int() with base 10: ''
2016-11-08 02:56:43,467: ERROR: base_destination._save():68: aws s3 cp - s3://<bucket>/daily/<hostname>/files/<filename>-2016-11-08_00_00_07.tar.gz exited with error code 1
2016-11-08 02:56:43,467: ERROR: base_destination._save():72: upload failed: ./- to s3://<bucket>/daily/<hostname>/files/<filename>-2016-11-08_00_00_07.tar.gz
An error occurred (InvalidArgument) when calling the UploadPart operation: Part number must be an integer between 1 and 10000, inclusive
python-psutil.x86_64 should be listed as dependency
Please implement Azure Blob storage as an alternative to AWS S3 destination.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.