Comments (13)
syndtr
commented on June 4, 2024
Do you use custom strict level? If so try add opt.StrictJournalChecksum.
Also, can I have the LOG file?
from goleveldb.
dajohi
commented on June 4, 2024
no read or write options. db is opened with:
myCache := cache.NewEmptyCache()
opts := &opt.Options{
BlockCache: myCache,
MaxOpenFiles: 256,
Compression: opt.NoCompression,
}
Before
...
19:36:44.926197 table@compaction L0·1 -> L1·0 S·3MiB Q·243786
19:36:45.059003 table@build created L1@64 N·9385 S·2MiB "\x00..x,v214820":"6..6,v232168"
19:36:45.120725 table@build created L1@65 N·7443 S·1MiB "6..7,v232171":"u..\x00,v241266"
19:36:45.201134 table@build created L1@66 N·12446 S·582KiB "u..\x00,v209026":"\xff..\x00,v241291"
19:36:45.205425 table@remove removed @63
19:36:45.205903 table@compaction commited F+2 S-137KiB D·4593 T·279.584132ms
19:36:45.258911 table@compaction L1·1 -> L2·2 S·4MiB Q·245733
19:36:45.454177 table@build created L2@67 N·37264 S·2MiB "u..\x00,v8958":"\xbc..x,v145054"
19:36:45.749786 table@build created L2@68 N·35593 S·1MiB "\xbc..\x00,v43050":"\xff..\x00,v123352"
19:36:45.750712 table@compaction commited F-1 S-96KiB D·3779 T·491.638664ms
19:36:45.752133 table@remove removed @66
19:36:45.754116 table@remove removed @53
19:36:45.755883 table@remove removed @49
19:36:45.756681 table@compaction L1·1 -> L2·2 S·5MiB Q·247853
19:36:45.834679 table@build created L2@69 N·5145 S·2MiB "5..8,v193769":"6..5,v239798"
19:36:46.025500 table@build created L2@70 N·18323 S·2MiB "6..6,v239809":"P..\x00,v60230"
19:36:46.112260 table@build created L2@71 N·19060 S·1MiB "P..\x00,v39160":"u..x,v33504"
19:36:46.114825 table@remove removed @65
19:36:46.116941 table@remove removed @51
19:36:46.118945 table@remove removed @52
19:36:46.119063 table@compaction commited F~ S-31KiB D·1379 T·362.273214ms
19:36:46.120702 table@compaction L1·1 -> L2·9 S·20MiB Q·249184
19:36:46.294480 table@build created L2@72 N·28583 S·2MiB "\x00..\x00,v29829":"1..4,v37794"
19:36:46.404459 table@build created L2@73 N·7874 S·2MiB "1..5,v37797":"1..5,v59589"
19:36:46.493073 table@build created L2@74 N·8012 S·2MiB "1..6,v59592":"2..5,v81609"
19:36:46.627392 table@build created L2@75 N·8003 S·2MiB "2..6,v81612":"3..9,v103642"
19:36:46.772611 table@build created L2@76 N·7506 S·2MiB "3258,v10117":"3..7,v126008"
19:36:46.865525 table@build created L2@77 N·6867 S·2MiB "3..8,v126011":"4..7,v150141"
19:36:46.923451 table@build error I·71009 "write /tmp/btcd/mainnet/blocks_leveldb/000078.ldb: no space left on device"
19:36:47.730141 db@close closing
19:36:47.730666 table@build exiting
19:36:47.730842 table@build rollback @72
19:36:47.733039 table@build rollback @73
19:36:47.735067 table@build rollback @74
19:36:47.736986 table@build rollback @75
19:36:47.738725 table@build rollback @76
19:36:47.740732 table@build rollback @77
19:36:47.742760 db@close done T·12.606084ms
After restarting and getting 19:36:52 2014-08-10 [ERR] BTCD: leveldb: bad record in batch
=============== Aug 10, 2014 (EDT) ===============
19:39:09.012267 log@legend F·NumFile S·FileSize N·Entry C·BadEntry B·BadBlock D·DeletedEntry L·Level Q·SeqNum T·TimeElapsed
19:39:09.013378 db@open opening
19:39:09.013711 journal@recovery F·1
19:39:09.014933 journal@recovery recovering @62
from goleveldb.
dajohi
commented on June 4, 2024
So I set Strict to StrictAll and cannot seem to reproduce...
from goleveldb.
dajohi
commented on June 4, 2024
With StrictAll
19:55:06.827228 table@compaction L1·1 -> L2·10 S·23MiB Q·203244
19:55:06.976269 table@build created L2@42 N·25848 S·2MiB "\x00..\x00,v29829":"1..5,v39880"
19:55:07.107509 table@build created L2@43 N·7886 S·2MiB "1..6,v39883":"1..8,v61638"
19:55:07.203274 table@build created L2@44 N·7922 S·2MiB "1..9,v61641":"2..2,v83782"
19:55:07.323501 table@build created L2@45 N·7934 S·2MiB "2..3,v83785":"3..9,v105794"
19:55:07.410912 table@build created L2@46 N·7113 S·2MiB "3324,v10365":"3..7,v128990"
19:55:07.538214 table@build created L2@47 N·7215 S·2MiB "3..8,v129020":"4..2,v152408"
19:55:07.623171 table@build created L2@48 N·6696 S·2MiB "4..3,v152411":"5..3,v176327"
19:55:07.739521 table@build created L2@49 N·3611 S·2MiB "5..4,v176330":"5..0,v194604"
19:55:07.791314 table@build error I·80460 "write /tmp/btcd/mainnet/blocks_leveldb/000050.ldb: no space left on device"
19:55:08.807316 table@build retrying N·1
19:55:09.208943 table@build error I·76162 "write /tmp/btcd/mainnet/blocks_leveldb/000050.ldb: no space left on device"
19:55:09.268996 db@close closing
19:55:09.269122 table@build exiting
19:55:09.269194 table@build rollback @42
19:55:09.271319 table@build rollback @43
19:55:09.273247 table@build rollback @44
19:55:09.275281 table@build rollback @45
19:55:09.277319 table@build rollback @46
19:55:09.279373 table@build rollback @47
19:55:09.281241 table@build rollback @48
19:55:09.283280 table@build rollback @49
19:55:09.285586 db@close done T·16.584526ms
from goleveldb.
calmh
commented on June 4, 2024
I've seen the same, with database opened as db, err := leveldb.OpenFile(filepath.Join(confDir, "index"), nil) (i.e. no options). There's a LOG at https://nym.se/t/LOG and the full db dir in https://nym.se/t/db.tar.gz if you like.
from goleveldb.
syndtr
commented on June 4, 2024
The problem is for some reason the corrupted chunk pass the checksum (it supposed to be dropped).
@dajohi Can I have the DB's journal file (e.g. 000054.log) that generate the error? Also, did the error persistent (DB that generate those error always generate same error when opened)?
@calmh Did you also get leveldb: bad record in batch error?
from goleveldb.
dajohi
commented on June 4, 2024
write /tmp/btcd/mainnet/blocks_leveldb/000040.log: no space left on device
https://www.mindcry.org/000040.log
I have not seen an error other than 10:41:10 2014-08-11 [ERR] BTCD: leveldb: bad record in batch
from goleveldb.
syndtr
commented on June 4, 2024
Thanks! Will look into it.
On Aug 11, 2014 9:41 PM, "David Hill" [email protected] wrote:
@syndtr https://github.com/syndtr
write /tmp/btcd/mainnet/blocks_leveldb/000040.log: no space left on device
https://www.mindcry.org/000040.log
I have not seen an error other than 10:41:10 2014-08-11 [ERR] BTCD:
leveldb: bad record in batch—
Reply to this email directly or view it on GitHub
#61 (comment).
from goleveldb.
calmh
commented on June 4, 2024
@syndtr Yep ("bad record in batch")
from goleveldb.
syndtr
commented on June 4, 2024
@calmh I can't seems to reproduce the error with your db file, but I can with @dajohi journal file.
@dajohi, @calmh Let me know if fcb3916 fix this or not.
from goleveldb.
calmh
commented on June 4, 2024
Hmm, me neither (I must have failed at grabbing the bad db), but I'll update and let you know if I can reproduce it again. If not, assume all good from my side.
from goleveldb.
dajohi
commented on June 4, 2024
Running with it. initial testing looks promising.. will let you know once all data is imported.
from goleveldb.
dajohi
commented on June 4, 2024
Looks good. Thanks for the prompt response!
from goleveldb.
Related Issues (20)
- Abandoned / Maintained? HOT 3
- Browser indexeddb in golang / wasm
- Does the 【Get】 method support * and ? fuzzy matching HOT 1
- long time no update release HOT 3
- Bad result in find by prefix
- Large Manifest file is repeatedly re-created HOT 2
- Locking the DB
- panic in `db.Stats()` during `db.Close()`
- data race
- some node will pass to f() function many times, is this a bug or a feature?
- slice bounds out of range [:18014398510996847] with capacity 4194304
- query efficiency and memory usage
- Opening a transaction will freeze all other db transactions HOT 2
- Is it possible to read the db from to different programs? HOT 3
- leveldb.(*DB).Close() took too much time to finish
- Panic: unexpected fault address
- Mem compression occurred during the creation of the iterator
- is the leveldb write (to disk) rate is 25MB ?
- Can I set file permissions? HOT 2
- How to read assigned start and end pos data?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from goleveldb.