notional-labs / cosmprund Goto Github PK
View Code? Open in Web Editor NEWprune data for tendermint/cosmos based chains
prune data for tendermint/cosmos based chains
for pebbledb only, no problem for goleveldb
key is encoded differently
https://github.com/sei-protocol/sei-tendermint/blob/4b4be3151c250f4f5ece0f1d4b7962148ed4c8dc/internal/state/indexer/tx/kv/kv.go#L554
https://github.com/tendermint/tendermint/blob/v0.34.24/state/txindex/kv/kv.go#L559
ref:#12
panic: pebble: batch too large: >= 4.0 G
goroutine 733 [running]:
github.com/cockroachdb/pebble.(*Batch).grow(0x0?, 0x418bff?)
/root/go/pkg/mod/github.com/cockroachdb/[email protected]/batch.go:1108 +0x129
github.com/cockroachdb/pebble.(*Batch).prepareDeferredKeyRecord(0xc00311cfc0, 0x32, 0x0)
/root/go/pkg/mod/github.com/cockroachdb/[email protected]/batch.go:498 +0x74
github.com/cockroachdb/pebble.(*Batch).DeleteDeferred(...)
/root/go/pkg/mod/github.com/cockroachdb/[email protected]/batch.go:600
github.com/cockroachdb/pebble.(*Batch).Delete(0xc00311cfc0?, {0xc0b69d6dc0?, 0x32?, 0xc0b69d6dc0?}, 0xc0b69dc750?)
/root/go/pkg/mod/github.com/cockroachdb/[email protected]/batch.go:583 +0x31
github.com/tendermint/tm-db.(*pebbleDBBatch).Delete(0x11?, {0xc0b69d6dc0?, 0xc006a03800?, 0x459bd2?})
/root/go/pkg/mod/github.com/baabeetaa/[email protected]/pebble.go:283 +0x2e
github.com/tendermint/tm-db.prefixDBBatch.Delete({{0xc003696d68, 0x11, 0x18}, {0x1568f40, 0xc005a5e270}}, {0xc0b69e6030, 0x21, 0xc000100400?})
/root/go/pkg/mod/github.com/baabeetaa/[email protected]/prefixdb_batch.go:35 +0x117
github.com/cosmos/iavl.(*nodeDB).DeleteVersionsRange.func1({0xc0b69e4511, 0x31, 0x31}, {0xc0b69d0b00, 0x20, 0x20})
/root/go/pkg/mod/github.com/cosmos/[email protected]/nodedb.go:305 +0x1e4
github.com/cosmos/iavl.(*nodeDB).traversePrefix(0xc000e84b10?, {0xc0b69880d0?, 0x1?, 0x1?}, 0xc006a03c00)
/root/go/pkg/mod/github.com/cosmos/[email protected]/nodedb.go:521 +0x112
github.com/cosmos/iavl.(*nodeDB).traverseOrphansVersion(0xc0022c87e0?, 0xc005fd1b80?, 0x4d2774?)
/root/go/pkg/mod/github.com/cosmos/[email protected]/nodedb.go:487 +0x6e
github.com/cosmos/iavl.(*nodeDB).DeleteVersionsRange(0xc0022c87e0, 0x1cd8a0, 0x205b6c)
/root/go/pkg/mod/github.com/cosmos/[email protected]/nodedb.go:298 +0x457
github.com/cosmos/iavl.(*MutableTree).DeleteVersionsRange(0xc0003e2460, 0x1cd8a0, 0x205b6c)
/root/go/pkg/mod/github.com/cosmos/[email protected]/mutable_tree.go:623 +0x4a
github.com/cosmos/iavl.(*MutableTree).DeleteVersions(0x1559d80?, {0xc01933c000?, 0x382cc, 0x382d6})
/root/go/pkg/mod/github.com/cosmos/[email protected]/mutable_tree.go:611 +0x2ca
github.com/cosmos/cosmos-sdk/store/iavl.(*Store).DeleteVersions(...)
/root/go/pkg/mod/github.com/cosmos/[email protected]/store/iavl/store.go:201
github.com/binaryholdings/cosmos-pruner/internal/rootmulti.(*Store).PruneStores.func1({0x155bae0, 0xc0083953a0?})
/root/cosmprund/internal/rootmulti/store.go:453 +0x159
created by github.com/binaryholdings/cosmos-pruner/internal/rootmulti.(*Store).PruneStores
/root/cosmprund/internal/rootmulti/store.go:450 +0x225
dydx-testnet-4 got apphash after running prunning tool.
Hello, team, hope you are doing well.
I have found one issue while using cosmprund.
This isn't working for goleveldb.
I used --backend=goleveldb but still it was only working for pebbledb.
so I have changed some codes for goleveldb and it is working as well.
Please check this issue and let me know.
Thank you,
compact after prunning so no need to use https://github.com/notional-labs/pebblecompact
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.