vy-let / yoink Goto Github PK

Not all backups need to preserve owner and group. We'll want to have an option to turn off those options, thus not requiring sudo on either side.

Some backup areas need to run more frequently than others. For instance, you may want to pull an hourly database backup, but only pull large media backups once a day.

On the server side, we could add a priority flag on each hook and backup target, to separate them out into regular, high-priority, and low-priority buckets. A command-line switch would be passed up to the server so that it could select the proper sets.

• If run without a priority flag, all high-priority and default-priority targets should back up.
• If run with --high-priority, only high-priority targets should back up.
• If run with --low-priority, all targets should back up.

The task runner on the client system would be responsible for setting which flag at the appropriate times.

Rsync has built-in support for backing up old copies of changed and deleted files as they come down the wire. I didn't add those flags in initially because my backup-to destination itself is backed-up incrementally to a time machine disk. So I wouldn't really be the target audience for such a feature, but I imagine it would be quite useful.

If multiple requesting parties (those running yoink.rb) want to back up from the same servers, they may trivially do so; however, they may be prone to trampling on each other with their pre and post hooks.

We need to add a lockfile for the duration of each backup so that requesting parties may queue themselves, perhaps by poll-retrying at a random interval.

At the very least, backups from separate hosts can run in parallel, since they necessarily operate on disjoint sets of files. We need to have a config on the client specifying how many hosts to run at the same time.

As a separate issue, we may want to investigate the possibility of multiple parallel transfer sources per-host. Since we don't explicitly tie the side-effects of before-hooks to specific transfer areas, we'll want to make sure that no transfers start before pre-hooks are done, and that post-hooks don't start until all transfers are done.