Comments (15)
@tlaurion In the beta1 (wyng) branch I changed the date-time code so that if you specified a date-time later than the latest session, it wouldn't skip the volume but instead include the specified sessions (minus the latest). So this should now work:
sudo wyng --all-before --session=<current-date-time> prune <volname>
# or
sudo wyng --session=20191101-120000,20200117-231500 prune <volname>
BTW, I'm now up to beta2
which has had more testing than beta1. I highly recommend switching over to it since it has more features and safeguards than sparsebak. And it would really help me get a true release underway to have some feedback from the latest branch... You should only need to rename both '/var/lib/sparsebak' and the dest path to 'wyng.backup' to make the switch. (You can also make a no-copy backup of your archive in reasonable time with a command like cp -rl sparsebak sparsebak-backup1
). Let me know if you have any questions about it.
from wyng-backup.
Ref commit 36cb98d
from wyng-backup.
@tlaurion To make it easier, here is a working example that uses the current date-time whenever its run:
sudo wyng --all-before --session=$(date +%Y%m%d-%H%M%S) prune <volname>
from wyng-backup.
@tasket :
I transitioned away of sparsebak by doing as described in precedent post in dom0:
sudo mv /var/lib/sparsebak /var/lib/wyng.backup
Then took a less important backup and did, prior to looping in all preexisting backup:
sudo ~/sparsebak/wyng --all-before --session=$(date +%Y%m%d-%H%M%S) prune vm-build-x230-coreboot-48-private
Which results in never ending (ctrl-c'ed because no disk activity):
Pruning Volume : vm-build-x230-coreboot-48-private
Preserving latest session.
Removing 1: S_20191103-090906
Then:
Completing prior operation in progress: merge vm-build-x230-coreboot-48-private
from wyng-backup.
Just realized I havent't moved sparsebak backup directory on destination disk to wyng.backup
, just on dom0. Wiped wyng.backup
on destination disk (backup AppVM) and retried.
Now I get:
Completing prior operation in progress: merge vm-build-x230-coreboot-48-private
Traceback (most recent call last):
File "/home/user/sparsebak/wyng", line 2234, in <module>
aset.in_process[3], clear_sources=bool(aset.in_process[2]))
File "/home/user/sparsebak/wyng", line 1772, in merge_sessions
+" && python3 "+tmpdir+"/rpc/dest_helper.py merge" +(" --resume" if resume else "")
File "/home/user/sparsebak/wyng", line 751, in do_exec
raise subprocess.CalledProcessError(err.returncode, err.args)
subprocess.CalledProcessError: Command '['/usr/bin/qvm-run', '-p', 'backup', '/bin/sh /tmp/wyng/rpc/tmp2j4w_ydh']' returned non-zero exit status 1
from wyng-backup.
Hmmm. Even trying to delete that backup tries to continue resuming past operation on that volume. Any insight?
from wyng-backup.
Yeah, that's why I suggested doing a cp -rl
snapshot first, in case the rename to wyng.backup didn't go right. Actually if you have a backup of the /var/lib/sparsebak dir you could restore a copy of that too.
What's going on is that the prune op crashed at a point I don't cover yet in recovery sanity tests. And its not yet sophisticated enough to see that it has already re-tried the op.
from wyng-backup.
@tlaurion If you look at the volume's dir on dest, and there is a "merge" subdir with contents in it, then its part-way into the move/remove operations. The data won't come back out of that merge dir until the op is complete. There may also be a "merge.log" in /var/lib/wyng.backup/default/volname.
If you're really stuck with this, I can recreate the problem by renaming my dest archive and then trying to start a prune op.
from wyng-backup.
I tried to recreate the problem with:
cd /dest
mv wyng.backup sparsebak
wyng --all-before --session=$(date +%Y%m%d-%H%M%S) prune
rm -r wyng.backup
mv sparsebak wyng.backup
wyng --all-before --session=$(date +%Y%m%d-%H%M%S) prune
However, the in_process flag did not take and there was no attempt to resume the prior op.
from wyng-backup.
What needs to be done to discard merge attempt in case it tries to resume operation? Delete the whole backup, or there is something that can be attempted?
from wyng-backup.
@tlaurion If you're importing from the archive that had the prune interruption (and the in_process status), you may still have a bit of detective work to do before the import. Let me know if on the dest volume there is a 'wyng.backup/default/in_process' file and/or a 'wyng.backup/default/VOLNAME/merge*' dir.
I think beta3 can recover the situation now, but I only did a few tests. Here is the decision matrix:
In the case there is no 'merge' or 'merge-init' dir, the merge procedure will be aborted and the in_process status cleared. If there is a 'merge-init' dir, it will try to return the session data files to their original locations before aborting and clearing in_process.
In the case there is a 'merge' dir, it will have to try to complete the merge procedure. (If this fails for some reason, it can be rescued with normal shell commands like cp, rm, etc. I can provide details.)
from wyng-backup.
@tlaurion There is a bug in merge that I'm dealing with now, so please hold off pruning with beta3 until I post a fix.
from wyng-backup.
@tlaurion OK, that bug is now fixed in beta3.
from wyng-backup.
@tlaurion If you're importing from the archive that had the prune interruption (and the in_process status), you may still have a bit of detective work to do before the import. Let me know if on the dest volume there is a 'wyng.backup/default/in_process' file and/or a 'wyng.backup/default/VOLNAME/merge*' dir.
[user@backup QubesOS]$ cd wyng.backup/default/
[user@backup default]$ find ./ -name merge*
[user@backup default]$ find ./ -name in_process*
[user@backup default]$
I think beta3 can recover the situation now, but I only did a few tests. Here is the decision matrix:
In the case there is no 'merge' or 'merge-init' dir, the merge procedure will be aborted and the in_process status cleared. If there is a 'merge-init' dir, it will try to return the session data files to their original locations before aborting and clearing in_process.
In the case there is a 'merge' dir, it will have to try to complete the merge procedure. (If this fails for some reason, it can be rescued with normal shell commands like cp, rm, etc. I can provide details.)
from wyng-backup.
Since the problematic volume was vm-build-x230-coreboot-48-private, I also deleted that non-completed pruned session.
Same result when trying to arch-init with --from in #50
from wyng-backup.
Related Issues (20)
- libsodium rec: Limit messages to protect authentication
- Restore lvm backups on brtfs? HOT 5
- Enable 'monitor' to function without authentication
- Bug: Wyng may ignore truncated volumes HOT 1
- Upgrade from old backup: "Not a Wyng format" HOT 1
- Error when using `send` with `--autoprune` HOT 1
- Automated password and passphrase input HOT 6
- Autoprune overzealous removal of sessions (v0.8beta) HOT 1
- Verify incorrectly reports some good volumes as bad HOT 1
- wyng/wyng-util-qubes: sparse/sparse-write restore/receive operations broken (0.7beta 20230721 with wyng 0.8beta 20230826) HOT 6
- Error when using Wyng with rclone
- Cannot receive to a different LV name HOT 2
- Feature request: Handle I/O errors more gracefully and continue iterating volumes HOT 1
- Memory leak? HOT 14
- Alt deduplication mode to handle extremely large archives
- AttributeError: 'LvmVolume' object has no attribute 'lv_time'. HOT 4
- Bad argument parsing HOT 1
- KeyError on volume delete HOT 5
- Bug: lvcreate error 5 when sending HOT 46
- Features supporting wyng-util-qubes v0.9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wyng-backup.