Comments (14)
grke
commented on August 20, 2024
Hello,
I think it is getting to the end of setting up it's file descriptors for resuming, but then hits the end of file when doing so, generating an error and exiting.
I think this is happening on the /current/manifest.gz file, and it's trying to get to the next entry after f:0071:C:/Workdir/blablabla.avi
I think it's a bug, because it ought to be able to figure out what is happening and do the right thing.
You might be able to persuade it to unstick itself by truncating /working/changed file to a position where it is on some file that's more than one entry prior to the last one in /current/manifest.gz.
from burp.
jochym
commented on August 20, 2024
I have the same problem. The backup is unable to continue with the same message. The working directory has no manifest but the "last good entry" is in changed (twice with different prefixes:
t006Bt/D:/Foto...
y0069D:/Foto...
and in counters_d:
ny0069D:/Foto...
Is there any way I can salvage this backup?
from burp.
grke
commented on August 20, 2024
Hello,
What are your burp versions, and are you able to copy/paste the equivalent section of log that rongten did above?
from burp.
jochym
commented on August 20, 2024
Thanks for the reply.
Version: 2.4.0 on both ends
As for logs. I have decided after few days to just remove all broken backups and start anew after last known good one. So, unfortunately, I no longer have those. The new backup is at 79%, and I hope it will get to the end.
This happened to me on more than one occasion (the same message). Could you provide any clues how to deal with this in general for not completely clueless user? Is there any technical paper describing the structure of these files? We should be able to get them to the consistent state manually by just removing inconsistent entries? No?
from burp.
jochym
commented on August 20, 2024
The run unfortunately did not finish. Here is the log:
2023-06-15 19:01:01 +0200: burp[883958] Found interrupted backup.
2023-06-15 19:01:01 +0200: burp[883958] Will resume on the next backup request.
2023-06-15 19:01:01 +0200: burp[883958] Backup resumed
2023-06-15 19:01:01 +0200: burp[883958] Client version: 2.4.0
2023-06-15 19:01:01 +0200: burp[883958] Protocol: 1
2023-06-15 19:01:01 +0200: burp[883958] Using librsync hash blake2
2023-06-15 19:01:01 +0200: burp[883958] Client is Windows
2023-06-15 19:01:01 +0200: burp[883958] Begin phase2 (receive file data)
2023-06-15 19:01:01 +0200: burp[883958] Begin phase1 (read previous file system scan)
2023-06-15 19:01:04 +0200: burp[883958] Setting up resume positions...
2023-06-15 19:01:07 +0200: burp[883958] last good entry: y:0032:D:/Foto/My Lightroom Photos/2021/Lwow/IMG_2587.CR2
2023-06-15 19:01:07 +0200: burp[883958] setting pos_phase1
2023-06-15 19:01:08 +0200: burp[883958] setting pos_current
2023-06-15 19:01:10 +0200: burp[883958] End of file in forward_past_entry()
2023-06-15 19:01:10 +0200: burp[883958] End phase2 (receive file data)
2023-06-15 19:01:10 +0200: burp[883958] error in backup phase 2
2023-06-15 19:01:10 +0200: burp[883958] Backup failed
Is there any way to resume this? It was at 93% of 2.5TB transfer. That is a major frustration...
from burp.
jochym
commented on August 20, 2024
@grke I am willing to do some substantial debugging of this issue, since it took 10 days to transfer the data and I have some programming experience. Where should I start?
from burp.
jumbo-grke
commented on August 20, 2024
Hello,
Your backup was interrupted somehow, and the burp server is trying to resume.
The error is happening in src/server/resume.c.
It has figured out that the last good entry in the changed file list is "D:/Foto/My Lightroom Photos/2021/Lwow/IMG_2587.CR2".
It has figured out the correct place to resume in the phase1 file list (setting pos_phase1 didn't error).
It tried to figure out the correct place to resume in the previous backup's manifest.gz file, but got to the end of the file before it found the right location (setting pos_current ... End of file in forward_past_entry).
I think maybe it is because the last entry in the changed last is alphabetically later than the last entry in the previous backup's manifest.
Instead of erroring when it reaches the end of file, it's quite possible that everything will work fine if it just returns success.
ie, in the function forward_past_entry(), instead of:
logp("End of file in %s()\n", __func__);
goto error;
Do:
logp("End of file in %s()\n", __func__);
sbuf_free(&sb);
return 0;
from burp.
jochym
commented on August 20, 2024
Thanks @jumbo-grke .
First question: does this change need to be on the server or client side? (I guess server?)
from burp.
jochym
commented on August 20, 2024
Looks good @jumbo-grke !
I have modified 2.4.0 the way you advised, and here is the result:
2023-06-19 10:38:23 +0200: burp[1120819] Client version: 2.4.0
2023-06-19 10:38:23 +0200: burp[1120819] Protocol: 1
2023-06-19 10:38:23 +0200: burp[1120819] Using librsync hash blake2
2023-06-19 10:38:23 +0200: burp[1120819] Client is Windows
2023-06-19 10:38:23 +0200: burp[1120819] Begin phase2 (receive file data)
2023-06-19 10:38:24 +0200: burp[1120819] Begin phase1 (read previous file system scan)
2023-06-19 10:38:26 +0200: burp[1120819] Setting up resume positions...
2023-06-19 10:38:28 +0200: burp[1120819] last good entry: y:0032:D:/Foto/My Lightroom Photos/2021/Lwow/IMG_2587.CR2
2023-06-19 10:38:28 +0200: burp[1120819] setting pos_phase1
2023-06-19 10:38:30 +0200: burp[1120819] setting pos_current
2023-06-19 10:38:32 +0200: burp[1120819] End of file in forward_past_entry()
2023-06-19 10:38:32 +0200: burp[1120819] Did not find restart point in previous backup forward_past_entry()
2023-06-19 10:38:33 +0200: burp[1120819] setting entry counter_d
2023-06-19 10:38:34 +0200: burp[1120819] setting entry counter_n
* 2023-06-19 10:38:35 +0200: burp[1120819] End phase1 (read previous file system scan)
The backup resumed from the same place and is currently running! The message at 2023-06-19 10:38:32 is added by me.
Thanks! I think this should be included in the standard code, since this behaviour will probably trigger in large backups after a long break or failure.
from burp.
grke
commented on August 20, 2024
Thanks for the fast feedback.
I think I will need to do some extra checking before adding it, so I will leave the issue open for now.
from burp.
jochym
commented on August 20, 2024
Sure. The backup is still running with no visible problems. I will check at the end if it will verify correctly. I understand your reluctance to rapid modifications of stable code. Just note that this problem showed up twice with my system (the backup failed and was non-running for some time, while the large data addition happened). This may be more common than expected.
from burp.
arjan-s
commented on August 20, 2024
Hi, do you have this planned for a new release? I'm experiencing exactly the same problem with 2.4.0 on both sides (windows client, linux server). Thanks!
from burp.
arjan-s
commented on August 20, 2024
I patched my server as described above, and encountered more errors. The backup is continuing now though, so I'm doubting whether that'll be reliable or not. I think I need to do a verify for that.
Anyway, here are the errors:
2023-08-07 17:03:01 +0200: burp[2403] Found interrupted backup.
2023-08-07 17:03:01 +0200: burp[2403] Will resume on the next backup request.
2023-08-07 17:03:01 +0200: burp[2403] Backup resumed
2023-08-07 17:03:01 +0200: burp[2403] Client version: 2.4.0
2023-08-07 17:03:01 +0200: burp[2403] Protocol: 1
2023-08-07 17:03:01 +0200: burp[2403] Using librsync hash blake2
2023-08-07 17:03:01 +0200: burp[2403] Client is Windows
2023-08-07 17:03:01 +0200: burp[2403] Begin phase2 (receive file data)
2023-08-07 17:03:02 +0200: burp[2403] Begin phase1 (read previous file system scan)
2023-08-07 17:03:06 +0200: burp[2403] Setting up resume positions...
2023-08-07 17:03:08 +0200: burp[2403] Tried to read 71 bytes, got 60 by pass 1
2023-08-07 17:03:08 +0200: burp[2403] Error in fzp_read_ensure, called from do_iobuf_fill_from_fzp: 60 bytes, eof
2023-08-07 17:03:08 +0200: burp[2403] Error attempting to read after t0046 in do_iobuf_fill_from_fzp (t:70)
2023-08-07 17:03:08 +0200: burp[2403] Error after y:0044:H:/zzz/zzz/zzz/zzz/zzz.JPG in get_last_good_entry()
2023-08-07 17:03:08 +0200: burp[2403] last good entry: y:0044:H:/zzz/zzz/zzz/zzz/zzz.JPG
2023-08-07 17:03:08 +0200: burp[2403] setting pos_phase1
2023-08-07 17:03:09 +0200: burp[2403] setting pos_current
2023-08-07 17:03:12 +0200: burp[2403] End of file in forward_past_entry()
2023-08-07 17:03:14 +0200: burp[2403] setting entry counter_d
2023-08-07 17:03:15 +0200: burp[2403] Tried to read 69 bytes, got 32 by pass 1
2023-08-07 17:03:15 +0200: burp[2403] Error in fzp_read_ensure, called from do_iobuf_fill_from_fzp: 32 bytes, eof
2023-08-07 17:03:15 +0200: burp[2403] Error attempting to read after y0044 in do_iobuf_fill_from_fzp (y:68)
2023-08-07 17:03:15 +0200: burp[2403] Error in forward_past_entry_counter(), but continuing
2023-08-07 17:03:15 +0200: burp[2403] setting entry counter_n
2023-08-07 17:03:18 +0200: burp[2403] End phase1 (read previous file system scan)
from burp.
jumbo-grke
commented on August 20, 2024
Hello,
Yes, I would like to release this soon.
The errors that you are seeing your last message are because the working files in your previous attempt at a backup are shorter than expected. Burp doing it's best to truncate them at a sensible position so that it can continue.
from burp.
Related Issues (20)
- Question: is dedup_group still valid with protocol 1 only supported? HOT 2
- Start on server boot HOT 2
- Burp Server not Restoring To Burp Windows Client HOT 1
- Burp stops in phase 2 when encountering crazy filename HOT 6
- Burp on aarch64 (arm64): supported? HOT 2
- burp fails cert validation w/ OpenSSL 3.0 HOT 4
- SSL error triggered when max_children is hit HOT 1
- embedded yajl is vulnerable to CVE-2017-16516 and CVE-2022-24795 HOT 2
- Only restore newer files
- About 3.1.4 release HOT 2
- error in readdir: Unknown error HOT 5
- EVP_CipherInit_ex failed - 2.4.0 on ARCH HOT 2
- Windows Backup - Unchanged files detected as deleted / backed up repeatedly HOT 14
- Configuring 2 clients for 2 servers on one instance HOT 2
- Question : why speed is slow on first backup ? HOT 3
- [BUG] version 3.1.4 both server/client bug, client on Windows 11 HOT 10
- Restore Windows Server 2022 inetpub files error HOT 3
- Summary e-Mail is always blank HOT 3
- Help me with the error: error in rs_async: 100 IO error HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from burp.