Comments (19)
I've seen this occasionally... what Xapian version are you using?
from mu.
I should have added that in earlier, apologies.
It's 1.4.25
All dependencies are shown below.
/nix/store/gm61h1y42pqyl6178g90x8zm22n6pyy5-libunistring-1.1
/nix/store/ddfzjdykw67s20c35i7a6624by3iz5jv-libidn2-2.3.7
/nix/store/nrwkk6ak3rgkrxbqhsscb01jpzmslf2r-xgcc-13.2.0-libgcc
/nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31
/nix/store/k927dryij6k8ga8slxs39pjdsmsny8c8-gcc-13.2.0-libgcc
/nix/store/p3ffjixpnfgkqh20nsrc13vrj3yfi0nj-gcc-13.2.0-lib
/nix/store/0fn1lbdb83gx7n917akp76djfkkc00x2-cld2-unstable-2015-08-21
/nix/store/117avr6c47nih6gy28wid7yr89xvp94s-libgpg-error-1.48
/nix/store/14b6spahplg5gsk3qna7g2vanqrji3pk-libksba-1.6.6
/nix/store/2bbjyrnwajslpr4kyyf5k0snv3cs5kw7-libgcrypt-1.10.3
/nix/store/nw1bnjadmzk9hnd4nbvxmqlh074vr16j-zlib-1.3.1
/nix/store/2lzvm03ybzp2zbvhj1y283087ghjw0k8-kexec-tools-2.0.28
/nix/store/jrrnf6kak9fbfm910506ppws50d33p9f-ncurses-6.4
/nix/store/2xjpd6ikhjajmdik680cvd8md6cikkp3-readline-8.2p10
/nix/store/h3bhzvz9ipglcybbcvkxvm4vg9lwvqg4-bash-5.2p26
/nix/store/xs3y3icbd26gxr617m1alhjba12pkm92-util-linux-minimal-2.39.3-lib
/nix/store/2zna3wb0sd0rx6xqjmihaqy1wjzczzyb-xapian-1.4.25
/nix/store/7mnvfrpsbivkxyfbmqczdq5pbb1n59j1-zstd-1.5.6
/nix/store/n87dnp3cw99gssql0xk9yycrhrg50gir-xz-5.4.6
/nix/store/33dl0i8mjh76kanigvkgmwyxp04mv436-kmod-31-lib
/nix/store/3iiml9pwff4y6zsis1gp4apbry1h1wfr-xz-5.4.6-bin
/nix/store/wwg6376ymkhpjxy6c3w9gkbhv6l2gmqy-libxcrypt-4.4.36
/nix/store/3q6f0k00yg2ddra93ybism3hy1ha3azk-util-linux-minimal-2.39.3-login
/nix/store/8f10pxi30lvkdjrkzfw5gzz4rzia8163-openssl-3.0.13
/nix/store/q7640gaa4r0jmjsysa2i7ckai7zr9pkr-keyutils-1.6.3-lib
/nix/store/pjmhai9as695ph4ykrx6990pj7xwhpyx-libkrb5-1.21.2
/nix/store/qkpxz2s5aq6lly1lbmaa2ggwmnlslwna-db-5.3.28
/nix/store/4mlz8w1m4420s89sy1rg6d997mbkznsb-cyrus-sasl-2.1.28
/nix/store/q4k8gmw2gpncglwdvz9fmqqwnynsb143-bzip2-1.0.8
/nix/store/5giwsjx504zrgzl7cjzxm0s1423wndif-bzip2-1.0.8-bin
/nix/store/5gzyfqc5vzm4gyijkyxw8bl8i91zfk77-gmp-with-cxx-6.3.0
/nix/store/hffrnc162j2j1rvrdnfiy8jflcxzf9mn-json-c-0.17
/nix/store/xzmgdglbldim3k39pjn16vrgxb3i72as-publicsuffix-list-0-unstable-2024-01-07
/nix/store/g6as2fdsiar8wzkkyiaj62z4knmxaxwl-libpsl-0.21.5
/nix/store/nn2ri4n3klfz71f3aiwycykz9c8xqm9p-libssh2-1.11.0
/nix/store/qa2j4ql5vb7n1zyi8ai5acivsjkg0l8m-brotli-1.1.0-lib
/nix/store/wi6w7jbnskypgc1dd3asv8p6rmw12ih8-nghttp2-1.61.0-lib
/nix/store/his3qhkzxg1s2j34spm27w2f4mvh2y84-curl-8.7.1
/nix/store/5s0glc9fcb902kabb16yl1ak20xnp16w-tpm2-tss-4.1.0
/nix/store/5w1acngnkxqsbidwbfbrrrhrzyjik3d5-libffi-3.4.6
/nix/store/7fmplwm445l3qhp2ck9p8bcbz1aaj5qp-dns-root-data-2023-11-27
/nix/store/v4jrk736s8a8mk3vcf50hx1l0dwa4bdz-libcap-2.69-lib
/nix/store/8f64rn26nd0cyy0brblzyn064fmcmy79-systemd-minimal-libs-255.4
/nix/store/rkj3k7pflfvjf7shraqvb1qd9s2dz1zs-gmp-with-cxx-6.3.0
/nix/store/adalw59bvrk2fg2fl662xnc4y2bqn438-nettle-3.9.1
/nix/store/sqgnfysp5n513gh4qqcs69yx9gxylzmv-libevent-2.1.12
/nix/store/dq7kpdqsxm7z7p26f7ppc8bjyfpwqm03-unbound-1.19.3-lib
/nix/store/nqd74akpcgp1xfaqbdlfsim807ixnfbc-libtasn1-4.19.0
/nix/store/v48dbyr8cmci56xp91m50z6x65mqws9x-p11-kit-0.25.3
/nix/store/a8j5yhyqcqahvrm6p99gkqs1yjmv9wcn-gnutls-3.8.5
/nix/store/sik7a8dic5lxhm4cp9npibk97x9jcxwf-pcre2-10.43
/nix/store/d0amq09n85152751g6zspl60mrjbadvz-libselinux-3.6
/nix/store/bcms6vbac6jyffwyj8y630brpy1il4ks-glib-2.80.0
/nix/store/yb39i0a63bx8kx2bgmx1p4dx1hcwmnzh-attr-2.5.2
/nix/store/y4psv76i8pllhnsd1dg7zbqsj8a2sz64-acl-2.3.2
/nix/store/ddypgcb25zgi0gwvklxlzazjrh9s63dg-coreutils-9.5
/nix/store/wy37jk2hirzqzx0666w1849kjdgzdam6-gnugrep-3.11
/nix/store/g1zh8pzpgkyl7ja4bbi4syjnif18xasp-zstd-1.5.6-bin
/nix/store/hragmx5p3izmwinv4441hj5706sxy4xs-gzip-1.13
/nix/store/bjzji701dz67bfrrwx9smqqyy1pnf2c9-kbd-2.6.4
/nix/store/bwcl7y02qj8mxhwpqflz56wjkaaxmq22-pcsclite-2.1.0-lib
/nix/store/gbx1y99nzj2vj03vn23mbnsfx8dwf9kd-sqlite-3.45.3
/nix/store/hdqqlyrv0gwvxnwpayvl7y8glmx81m44-libsodium-1.0.19
/nix/store/in8kwd36w4fcqd8xizr2r8qy1dx0rbx6-libtool-2.4.7-lib
/nix/store/fdiy11m9nkph9hrl2k0ibl4xmra9icmp-util-linux-minimal-2.39.3-swap
/nix/store/q66l12afk5bly0aprvdxjkjrsz2dbma8-glibc-2.39-31-getent
/nix/store/g11dc1qynwyvbqpf40d81f9205w48m76-getent-glibc-2.39-31
/nix/store/hx4sz22j1gij2gshi9w110x39zl7hpd8-libseccomp-2.5.5-lib
/nix/store/nnvsjd3f3dh9wdl4s9mwg5cfri8kds5j-bash-interactive-5.2p26
/nix/store/wyk82rbsdshy57l4y5zl5bfnyq03q85f-kmod-31
/nix/store/yi18kda2q14sqkimq75qpn2w1kk2g6d3-util-linux-minimal-2.39.3-mount
/nix/store/yhnhwqm09w9302rx24804i67jfp8ar4h-systemd-minimal-255.4
/nix/store/qvgz18lsxl233ph5kk3i1p7lv9wh6p8q-openldap-2.6.7
/nix/store/rs028f1bbm7i0qc3k2gd7j9rk6hywc45-libassuan-2.5.7
/nix/store/vlcly0k4fys9x3ad64faxcyz33a7c00h-libusb-1.0.27
/nix/store/zaf8aj5sl1ml9hcz9b32vyhvjs66wkxv-npth-1.7
/nix/store/fxyrm6pws83nmq89sbxlkaq3ln9kjbl6-gnupg-2.4.5
/nix/store/f59g2b8xbri3gplcai9p90y1p0i0c2yz-gpgme-1.23.2
/nix/store/qsd48ndjzzpy70y0clxjxs2f968lk0hq-gmime-3.2.14
/nix/store/z2ccqp8yipjdk8w2qw9fpgb88swa60h0-mu-1.12.5
from mu.
Just happened again but with a different error:
Database couldn't be opened for reading: DatabaseCorruptError: Expected block 104050 to be level 3, not 0
Continuing check anyway
docdata:
blocksize=8K items=791426 firstunused=77160 revision=236 levels=2 root=6263
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1582852 firstunused=142454 revision=236 levels=2 root=14600
B-tree checked okay
termlist table structure checked OK
postlist:
xapian-check: DatabaseCorruptError: Expected block 104050 to be level 3, not 0
from mu.
And a different error again:
docdata:
blocksize=8K items=790419 firstunused=77826 revision=1773 levels=2 root=6812
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1580838 firstunused=143486 revision=1773 levels=2 root=13888
B-tree checked okay
termlist table structure checked OK
postlist:
blocksize=8K items=3734630 firstunused=203197 revision=1773 levels=3 root=22505
xapian-check: DatabaseError: Block 45602 item 51: key >= right dividing key in level above
from mu.
... working on this.
from mu.
Is there anything I can do or any additional information that would be helpful?
from mu.
I'll let you know when I have something for testing. Hopefully this week.
from mu.
I didn't mean to chase/rush you - I was just thinking if there was anything I could do to help.
There's a new error just now:
docdata:
blocksize=8K items=790760 firstunused=52993 revision=2499 levels=2 root=1389
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1581520 firstunused=147135 revision=2499 levels=2 root=144033
B-tree checked okay
termlist table structure checked OK
postlist:
blocksize=8K items=3751217 firstunused=211301 revision=2499 levels=3 root=55552
xapian-check: DatabaseError: Block 206577 item 65: not in sorted order
from mu.
I think the problem is with multiple writers / transactions to the Xapian database. It's not really documented exactly what Xapian allows, but likely mu broke a few rules.
Anyway, there's wip/djcb/store-worker
which ensures all writes and mu4e commands are handled in a single thread. Works fine for me, with a few (cosmetic) things I want to fix before merging into master
. But, should hopefully address the errors you're seeing.
from mu.
from mu.
from mu.
Thanks for testing; yes, there's some issue the indexing in the indexing state machine, working on that. For now 1.12.5 is probably better for now...
However, did you see database corruption with the branch with xapian-check
? Your two messages seems a bit a contradictory. Thanks!
from mu.
Sequence of events:
- re-indexed with 1.12.5 after the most recent corruption
- changed to your test branch
- after some time, I saw the corruption issue mentioned in #2706 (comment)
- re-indexed with your test branch
- I haven't seen any corruption issues, but still get the hang mentioned in #2706 (comment)
from mu.
I'm wondering if there is anything special about my setup that's triggering this considering nobody else is experiencing it. Anything you can suggest/any workarounds to try out?
from mu.
I don't know... I'm seeing it occasionally.
I only started seeing it recently, perhaps after some system upgrade. I suspect it's some unsafe access to the Xapian database, and some newer versions of Xapian are less forgiving. But.. this is all just speculation.
The "unsafest" time might be trying to use mu while indexing is underway; so not doing that may help. But who knows.
from mu.
Just another data point - I have not seen any corruption issues for quite a while now without any changes to mu and/or xapian.
from mu.
FWIW, I have just experienced the issue again - this time with 1.12.6:
$ xapian-check .cache/mu/xapian
docdata:
blocksize=8K items=797353 firstunused=83370 revision=325 levels=2 root=11949
B-tree checked okay
docdata table structure checked OK
termlist:
blocksize=8K items=1594706 firstunused=154369 revision=325 levels=2 root=151270
B-tree checked okay
termlist table structure checked OK
postlist:
blocksize=8K items=3806077 firstunused=215875 revision=325 levels=3 root=157358
xapian-check: DatabaseError: Block 152998 item 0: leaf key >= right dividing key in level above
from mu.
I have the same issue with 1.1.25, every month or so I have to reinit the database and reindex due to corruption (a different error message each time). Not sure exactly what version I was running before this, but probably 1.0.x. This is perhaps not a very useful comment, but it does at least show that this is not a unique problem to Peter!
Edit for further context: I do tend to jam the update key and sometimes end up spawning multiple update processes at once. I also sometimes send the update process a kill -9 if it gets stuck (e.g. if I shut my laptop while it's updating). Normally the thing that is stuck is mbsyncrc, but it's possible I end up kill-9-ing mu on occasion as collateral damage. This didn't use to be a problem, but maybe I should be sending a kill -15 instead.
Edit to my edit: I guess that makes no sense, mu is running as a server in a separate process, if I kill -9 the *mu4e-update*
buffer I will just kill mbsyncrc. So I don't think this is relevant. I am however pretty certain that I have managed to spawn two such processes at once and presumably when they complete they will kick off two overlapping mu reindexings.
from mu.
For better or worse, homebrew updated things on me when I wasn't paying attention. Now I'm running 1.12.6 and the crash is still happening, but ~/.cache/mu/mu.log
is no longer being updated. Did it move somewhere else?
from mu.
Related Issues (20)
- [mu4e bug] Saved drafts not deleted after send HOT 7
- Encoding issue for some types of emails HOT 8
- [misc] [mu4e] [documentation] Incomplete sentence on section `5.6.2 MIME-part actions` HOT 2
- [mu4e rfe] Adding mailing list commands HOT 1
- Broken `meson setup` command in `README.org`
- [mu4e rfe] Way to jump between fields in compose mode HOT 1
- [misc] [mu4e] [documentation] install instructions for HOT 1
- mu4e-compose-pre-hook not triggering anymore HOT 1
- [mu4e bug] How to silence tip on `user-mail address` HOT 2
- [misc] support a post imapnotify `mu index` while mu4e is running HOT 2
- [mu/mu4e bug] updates made via mu4e not visible to external mu queries anymore HOT 4
- [mu4e rfe] Opening attachments directly in external programs HOT 1
- [mu4e bug] mu loses track of messages marked as (un)read in mu4e HOT 5
- [mu4e bug] Attachments with identical name: mu4e-view-save-attachments can only save one HOT 6
- [RFE] mu4e: option to save sent messages as new HOT 1
- [RFE] mu3e: ability to edit non-draft messages so that the edits synchronize HOT 1
- [mu bug] mu should return the encrypted flag for S/MIME enveloped-data HOT 4
- [mu4e bug] when the captured message's subject is emtpy `mu4e-compose-attach-captured-message` inserts a tag without a value for the description HOT 7
- [mu4e bug] error "Specified window is not displaying the current buffer" HOT 8
- [mu4e bug] "replied" flag is not set after sending a reply HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mu.