compression leads to filesystem hangs about bcachefs HOT 11 CLOSED

from bcachefs.

I'm also getting hangs trying to copy data off of the filesystem or trying to get a git status afterwards - it seems like trying to install nixos makes the entire file system unusable.

Mind that these hangs happen both with the reverted commits and the current head.

from bcachefs.

Which commits were you reverting?

I'm not seeing anything with xfstests, and I wouldn't expect a hang in fdatasync to have anything to do with the format options, so something weird is going on...

Can you try without zstd?

from bcachefs.

Sorry, I should have used the term 'amended', it seems you made this change at some point and I was referring to it happening even with it:

diff --git a/fs/bcachefs/btree_gc.h b/fs/bcachefs/btree_gc.h
index 1905acfa028a..8af5f841a537 100644
--- a/fs/bcachefs/btree_gc.h
+++ b/fs/bcachefs/btree_gc.h
@@ -109,7 +109,7 @@ static inline bool gc_visited(struct bch_fs *c, struct gc_pos pos)
 
        do {
                seq = read_seqcount_begin(&c->gc_pos_lock);
-               ret = gc_pos_cmp(pos, c->gc_pos) < 0;
+               ret = gc_pos_cmp(pos, c->gc_pos) <= 0;
        } while (read_seqcount_retry(&c->gc_pos_lock, seq));
 
        return ret;

I'm still installing but I got past the point where it usually hangs by omitting zstd, looks like that is where the problem lies.

from bcachefs.

The issue seems to also occur with lz4, but I can confirm it does not happen without compression.

from bcachefs.

from bcachefs.

Happens with only --compression, too.

from bcachefs.

from bcachefs.

perf top output does not show whatever is maxing out the core.
backtrace:

[<0>] io_schedule+0x12/0x40
[<0>] wait_on_page_bit_common+0xfe/0x210
[<0>] __filemap_fdatawait_range+0xd5/0x130
[<0>] file_write_and_wait_range+0x68/0x90
[<0>] bch2_fsync+0x28/0xa0 [bcachefs]
[<0>] do_fsync+0x38/0x60
[<0>] __x64_sys_fdatasync+0x13/0x20
[<0>] do_syscall_64+0x4e/0x100
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[<0>] 0xffffffffffffffff

To reproduce, first, we need a NixOS ISO with bcachefs support, which can be created by installing nix, cloning https://github.com/hyperfekt/nix-exp, doing nix-channel --add http://nixos.org/channels/nixos-unstable nixos-unstable and nix-channel --add http://nixos.org/channels/nixos-18.09 nixos, and then doing nix-build '<nixpkgs/nixos>' -A config.system.build.isoImage -I nixos-config=nix-exp/installation-cd-bcachefs-graphical-kde-new-kernel-git.nix. The ISO will be located in a directory named result/iso.

After starting from the NixOS ISO, creating the filesystem, mounting it under /mnt, doing nixos-generate-config --root /mnt, copying this file to /mnt/etc/nixos, add it to the imports list of the /mnt/etc/nixos/configuration.nix file. Then do nix-channel --add http://nixos.org/channels/nixos-unstable nixos-unstable and run nixos-install --root /mnt. The installation should hang with the output of querying info about missing paths. Make sure you've provided networking.

It might be possible to get the nixos-install tool without the ISO and test the whole thing without NixOS, but I'm not able to figure out on the quick how.

from bcachefs.

from bcachefs.

The install works fine now, thanks for the quick fix!

from bcachefs.