Comments (2)
The current bees algorithm loops too many times when many blocks in many files are identical. This is fairly common in BMP files. The exception just breaks the loop. If the loop is allowed to run until completion it can take several days to process each file.
Can you say more about what exactly is going wrong with the filesystem?
from bees.
When it first happened I asked on the BTRFS mailing list for help to recover (https://www.spinics.net/lists/linux-btrfs/msg75143.html) but I was using zstd compression so couldn't use any of the normal tools, so I gave up. That instance of corruption was particularly bad, as it was only noticed after the FS went read only.
The second time (during the rebuild) I was being a bit more proactive about running scrub every day or two, which is how I found the corruption. There were no IO errors or other kernel messages beforehand. I wasn't capturing the output from bees so I don't know if it emitted any errors.
I tried to fix the FS, but btrfs check couldn't do it. Then I tried to btrfs check --init-extent-tree which seemed to be working, but I stopped it after millions of errors and 26 hours.
I'm rebuilding for the 3rd time now and no errors so far. I haven't deleted any snapshots with bees running (I remember a while back that would crash bees). Just create snapshot, rsync data, and repeat.
I may have also rebooted while bees was running, and if the drive didn't unmount in time then a bunch of dirty metadata would get lost.
Basically I have no idea where the problem is at the moment, still looking. I doubt it's bees though, more likely the kernel dedupe ioctls. If I have anything to report I'll let you know
from bees.
Related Issues (20)
- System service enable problem HOT 6
- Enhancement: exporting live stats HOT 1
- Used size increased by ~400GB while defragmenting HOT 2
- Mount prevent second run HOT 1
- bees seemingly cannot catch up with snapper snapshot creation HOT 16
- Initial run of bees appears to have resulted in data corruption HOT 7
- bees breaks existing reflinks? HOT 4
- bees "--loadavg=3" option causes load crazy 20+ and OOM kill HOT 1
- beesd script can't handle BEESHOME on a non-btrfs filesystem HOT 4
- Stopping one of multiple beesd services removes the /run/bees directory HOT 4
- 执行bee run之后一直卡出不动 HOT 2
- Lookup root 256 ino xxx failed: Bad address HOT 1
- Documentation: modern Ubuntu install HOT 1
- [Feature Request] Unmount mount points if beesd exits.
- [Feature Request] Beesd to run a full dedup cycle and then end HOT 6
- How to remove? HOT 10
- cancelling with CTRL+C need to be followed by umount /run/bees/mnt/$UUID HOT 3
- build fails on Fedora 40
- Demystifying needed options with QubesOS pool in btrfs reflink (multiple cow snapshots rotating, beesd dedup and load avg hitting 65+ on 12 cores setup) HOT 4
- btrfs send size HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bees.