Automate backups with borg in a more convenient and reliable way! These scripts are some small and handy shell scripts to automate the backup process with BorgBackup. They are POSIX-compatible, so they should run with all shells. You're free to modify them for your needs!
They add some convienent features around borg, regarding environments with only one client.
- Local lock system: Cirumvent the issue of stale lock files.
- Automated retries: When backups stop mid-way they are automatically restarted.
- Simple configuration: Using shell-scripts you can configure each backup and then execute it the order/way you want.
- Status information: You can use a login script to get a notice when backups failed.
- For servers & desktops: The script is also uable on desktops. But rest assured: We'll never forget servers.
- Optional & adjustable: You do not have to use all features and you can adjust them in a simply way.
When the backup process is interrupted, sometimes the remote borg repository stays locked. That's why further backups will fail.
The issue has been addressed in borg v1.1.0 (currently beta), but I have not tested it and until it works there, here is my workaround.
Borg's current PID is written into a file. As long as the file exists the backup is considered to be "locked". At the end of the backup process (no matter whether it was succesful or not), the local lock is being removed, permitting further backups to start. The "local lock" is more reliable than the "remote lock" system, currently implemented in stable versions of borg.
Sometimes backups stop mid-way. This can have different reasons (e.g. an unreliable network). However we still want our data to be backed up. That's why the script integrates a retry mechanism to retry the backup in case of a failure (for up to three times by default). Between the retry attempts the script pauses some minutes by default, to wait for your server to restart or the connection to reestablish. This is also a workaround for the "connection closed by remote" issue, which seems to affect some users.
Additionally the script has the ability to write stats of the last executed backup. cronsizecache.sh
outputs the size of backups stored on the backup server. Both can be used (in conjunction with a script to check the last backup time) to monitor your backups and to automatically notify you, in case a backup failed/didn't work.
Rest assured your backups are safe and recover themselve if possible. (If they don't, you'll get to know that.)
The whole script can, of course, run on servers, but some features making it also suitable for desktops. It can display notifications about started and finished backups, including its result and it can even ask the user, whether to retry the backup in case it failed. Or you ask the user about the password for your backup interactively, in a GUI window.
It has been tested with GNOME, but should work on any system, where zenity
is installed. And: You can always easily adjust it to work with a different program.
It is even possible to configure it to show the notification as a "standard" user while the backup is actually running as root.
For examples, see the wiki page about more ways for GUI integration.
The main "work" is done by borgcron.sh
. This script can be used to execute a single backup.
The configuration files per repository/backup are saved in the config
directory.
The file borgcron_starter.sh
is the file you should call from cron or when debugging manually. Using it, you can execute multiple backups by passing the config names to it.
Also, you can of course not use some features outlined here. That's why the whole functionality is broken into multiple scripts and configurable in a single config file per backup.
- easy to understand, easy to modify
- POSIX-compatible
- pretty logs
- passphrase can be protected better, because it's saved in an external file
- pruning after the backup
- script to dump databases before backing up
- privilege-separation (login scripts can have higher privilege than backup process)
- tested in production (but no guarantees, use at your own risk! 😉)
- logging if backup is interruped by signals (e.g. at shutdown)
- tested with more than 30 automated unit tests (see Travis-CI badge above)
- more to come…
borgcron_starter.sh
– Cycles through backup files and interprets passed parameters.borgcron.sh
Main script. Actually executes the backup and runs borg.config
– Directory for config filesexample-backup.sh
– Example configuration file for a backup. Please use this template to add your backup(s).
tools
– additional scriptschecklastbackup.sh
– Script, which you can execute at login (add it to your.bashrc
file or so). It notifies you when a backup has failed. Otherwise it remains silent.cronsizecache.sh
– Small one-liner to cache the size of the dir where backups are stored. (useful for remote backup servers) You can then include the result withcat
in your login script.databasedump.sh
– Dumps one or several databases into a dir/file. Make sure that this script and the dump dir are only readable by your backup user. Script might have to be executed with higher privileges (i.e. root) for creating the backup.
system
– Various system scripts, you may need for your setup.tests
– Scripts for unit testing, etc. See Readme inside of it, not needed in production.
Setup instructions can be found in the wiki.
If you find a vulnerability drop me a mail at my mail address listed in my GitHub profile. You can find my public key here and on keybase.io. My fingerprint is 7046 C1B2 8644 9EAF 9F3F F5C1 8F16 2AE4 4088 F1BE
.