Comments (3)
from dmdedup3.19.
For deduplication, there are extra components in the write path when compared to normal writes:
- calculate the checksum for the block
- check whether the checksum exists in the metadata
- Decide to share / write new block. Since your workflow has only 30% deduplication - we are going to end up writing both metadata and data device for the remaining 70 % writes
Provided the fact that you are using the same physical device for storing both meta-data and data, the high latency is expected.
We suggest using different SSDs for storing metadata device(to verify if it significantly affects the performance).
It would also be helpful if you can provide the stats for data with different duplicate %. (For example 90 , 70, 50, 0).
Also please use the new kernel version. You can find the source here
https://github.com/dmdedup/dmdedup4.8
Thanks.
from dmdedup3.19.
tested on dmdedup4.8 as suggested, issue still persists, please see dmdedup/dmdedup4.8#3
from dmdedup3.19.
Related Issues (20)
- Deduplication chunking algorithm with dmdedup HOT 2
- How to configure cache size? HOT 2
- free_pages NULL pointer dereference during mkfs.xfs HOT 18
- Logical Block counter value is not incremented properly HOT 3
- Save statistics in CBT backend HOT 2
- Possible memory leak HOT 6
- Minor issue in hash interface. HOT 16
- how to benchmark dmdedup HOT 8
- Seems like new issue or may not be... HOT 2
- Source code about the DTD backend. HOT 6
- Question about the consistency operations in cowbtree backend HOT 2
- dmdedup cannot be compiled on kernels >4.4 HOT 4
- System crash when change block size to non-4096 number
- Running FIU trace on dmdedup
- hung task timeout issue if flushrq is 1. HOT 2
- dmdedup is not de-duplicating data with ext4fs HOT 1
- How to build this dirver? HOT 2
- Can not insmod driver! HOT 4
- How to replay the FIU trace?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dmdedup3.19.