maharmstone / ntfs2btrfs Goto Github PK

View Code? Open in Web Editor NEW

639.0 639.0 28.0 478 KB

License: GNU General Public License v2.0

CMake 0.68% C 41.10% Assembly 1.70% C++ 55.84% Roff 0.68%

ntfs2btrfs's People

Contributors

Stargazers

Watchers

ntfs2btrfs's Issues

How to rollback?

According to the guide there is an rollback image created, so do i write it to the partition or is there any way to rollback using ntfs2btrfs?

Segmentation fault during inode creation

The program doesn't start: it notifies me about the algorithms it's gonna use and then crashes instantly. I'm using a gpt partition table on a nvme drive if it can help.
I even tried with another partition on the same disk and it works.
The not converting partition maybe isn't absolutely vanilla ntfs: if I remember well I created it with mkfs.ntfs and not with the windows utility. I'll try later converting the partition using the tool from windows, but sincerely I doubt it will change anything.

After doing some very simple debugging I discovered that probably there's some problem during a memcmp in the function string_view ntfs::find_sd(uint32_t id, ntfs_file& secure) (at this line)

Not working without capital?

Not a big problem, but if I use ntfs2btrfs k:\, it does not work: Could not open k:\ (error 3). I did not do something different, but directly in the next line/prompt I uses ntfs2btrfs K:\ and this does work. Maybe you would want to change it that a capital would not matter in Windows?

Windows 10 NTFS to BTRFS inside Virtualbox freeze

I installed Windows 10 guest in my Fedora 36 host Virtualbox. When I issue ntfs2btrfs /dev/sdaX, virtualbox and the whole computer freezes. So I have to Ctrl+Alt+F4 and go to another tty and force kill virtualbox for my computer to get back working. I was wondering what the issue is? How does anyone try running quibble/ntfs2btrfs in a virtual machine without the need to install on bare metal computer?

installation instructions for debian has a broken link

The link to the installation instructions for debian is broken. Please fix. In the meantime can someone tell me how to install this on debian.

Invalid file signature error?

What causes this error? I attempted to use this to convert a 2TB hdd but it gave me that error.
On arch linux if that matters

Calculating Checksums over 100%

I have been running a convert for about 20 hours now on a 60TB USB raid drive enclosure.

It is an 80TB 5 drive enclosure with 60TB available and about 75% full.

The checksum calculation now shows over 100%

 ntfs2btrfs /dev/sdd2
Processing inode 334451 / 334451 (100.0%)
Updating directory sizes
Calculating checksums 2835087000 / 2684297073 (105.6%)

Should I be worried? Is this gong to fail? Is it going to continue on forever or is there an end in sight?

IF it is going to go on forever is it safe to cancel the process and roll back? Or would there be a "corrupted" image that will result in complete loss?

Can't convert from Fedora (Nobara) 36

Hello,

I've tried to use ntfs2btrfs to convert my data drive (with my games & folder used by Windows for documents, downloads etc) on Fedora 36 (Nobara actually) with Gnome but when I am trying to execute the command I am facing this issue :

Using Zstd compression.
Using CRC32C for checksums.
/usr/include/c++/12/bits/stl_vector.h:1122: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned char; _Alloc = default_init_allocator<unsigned char>; reference = unsigned char&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

I cannot understand where it could come from.

Converting a boot disk

It would be nice if that was made possible. My suggestion is to have it add Quibble's EFI to the list of boot items and set it as default after converting the boot disk.

Cannot convert volume with dirty bit set.

Error is in the title, I tried chkdsk and everything to repair the volume but nothing is working. Any help?

Possible to recover after power outage

Sadly I got very unlucky and just in the moment I was converting a disk using this tool the power in the whole build shut down. Now the disk is not ecognised as NTFS volume, can I resume or recover the block which are left (alredy converted)?

Prevent conversions of file systems with unsupported features unless forced to

There are several pitfalls for the unsuspecting user, without even doing anything tricky:

With a large enough partition, overly large ADS becomes an issue ;
A Windows 10 installation comes with a large number of compressed files out of the box ;

ntfs2btrfs should abort a conversion if it encounters an unsupported NTFS feature unless the user forces it. Assuming the NTFS file system is still pristine just after converting all inodes, it may be a good place to abort so that the user has a complete list of things to fix before proceeding further.

aur package

hey i packaged it https://aur.archlinux.org/packages/ntfs2btrfs-git if you want mention it in readme or something

thanks a lot for your work, you made windows servers bearable

Item too large for tree

Basically title. Sorry for not much context, don't know what else to add.

Can't be built on Ubuntu 20.04 x86_64

cmake .
-- Configuring done
CMake Error at CMakeLists.txt:35 (add_executable):
  Target "ntfs2btrfs" links to target "fmt::fmt-header-only" but the target
  was not found.  Perhaps a find_package() call is missing for an IMPORTED
  target, or an ALIAS target is missing?


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

libfmt-dev is installed.

Can you add compiled x86 too into Version 20210923?

Hi maharmstone,
I just downloaded latest version 20210923 and seems there is compiled only x64 version? Can you please add compiled x86 version too please for users who are unable to compile it?

I have tested current release under XP x64 and ntfs2btrfs.exe must be modded with CFF Explorer to change Major system version from 6 to 5. Then I can run it without problem and no missing depencies.

It would be perfect to have x86 & x64 compiled tool versions in one archive.

Many thanks

George

Cryptic error message: "Error: v > 8"

I get the following when running ntfs2btrfs: "Error: v > 8".

What does this means?

Steam on Linux makes BTRFS drive converted from NTFS read-only

On Windows everything works fine, but on Linux when trying to install a game or update a game, my converted drive turns into read-only

output of command "journalctl -p3 --since "today" | grep BTRFS"

Aug 02 16:03:18 localhost kernel: BTRFS error (device nvme0n1p6): qgroup generation mismatch, marked as inconsistent
Aug 02 14:06:04 localhost.localdomain kernel: BTRFS error (device sda1): unable to find ref byte nr 124385026048 parent 0 root 5 owner 33353 offset 0
Aug 02 14:06:04 localhost.localdomain kernel: BTRFS: error (device sda1: state A) in __btrfs_free_extent:3094: errno=-2 No such entry
Aug 02 14:06:04 localhost.localdomain kernel: BTRFS: error (device sda1: state EA) in btrfs_run_delayed_refs:2151: errno=-2 No such entry
Aug 02 14:08:45 localhost.localdomain kernel: BTRFS error (device sda1): unable to find ref byte nr 670268948480 parent 0 root 5 owner 92017 offset 0
Aug 02 14:08:45 localhost.localdomain kernel: BTRFS: error (device sda1: state A) in __btrfs_free_extent:3094: errno=-2 No such entry
Aug 02 14:08:45 localhost.localdomain kernel: BTRFS: error (device sda1: state EA) in btrfs_run_delayed_refs:2151: errno=-2 No such entry
Aug 02 14:14:03 localhost.localdomain kernel: BTRFS error (device sda1): unable to find ref byte nr 670268948480 parent 0 root 5 owner 92017 offset 0
Aug 02 14:14:03 localhost.localdomain kernel: BTRFS: error (device sda1: state A) in __btrfs_free_extent:3094: errno=-2 No such entry
Aug 02 14:14:03 localhost.localdomain kernel: BTRFS: error (device sda1: state EA) in btrfs_run_delayed_refs:2151: errno=-2 No such entry

drive was converted via Version 20210923

Unable to convert volume: FSCTL_LOCK_VOLUME failed (error 5, access denied)

Windows Specifications:

Windows 10 Pro
22H2
19045.2251
Windows Feature Experience Pack 120.2212.4180.0

Installation Details:
I have tried ntfs2btrfs using scoop install ntfs2btrfs and scoop install ntfs2btrfs -g with Administrator privileges.

The Problem:
When run the command ntfs2btrfs f:\ or ntfs2btrfs F:\ it throws the output:

Using Zstd compression.
Using CRC32C for checksums.
FSCTL_LOCK_VOLUME failed (error 5, access denied.).

Screen Capture:

Note: The terminal is running with administrator privileges

Additional Information

The size of the partition that I'm trying to convert to btrfs is 50 GB
This is a new, freshly created partition that was formatted using MiniTool Partition Wizard v12.7
I have written no data on it

Could not open d: (error 5).

Using administrator. Windows 10. ntfs2btrfs-20210523-win

Option to not make an image of the disk

It would be nice if one was added.

FIXME runtime error

when trying to run it on my NTFS partition I receive a FIXME - other mapping error

Crashes on a use-after-free caused by vector iterator invalidation

When I install the ntfs2btrfs-git AUR package and run, it crashes:

nyanpasu64@ryzen ~> sudo ntfs2btrfs /dev/nvme0n1p6
Using Zstd compression.
Using CRC32C for checksums.
Processing inode 23214 / 23214 (100.0%)
Mapped 13018 inodes directly.
Rewrote 0 inodes.
Inlined 6437 inodes.
Updating directory sizes
Calculating checksums 14862031 / 14862031 (100.0%)
fish: Job 1, 'sudo ntfs2btrfs /dev/nvme0n1p6' terminated by signal SIGSEGV (Address boundary error)

When I rebuild with asan enabled and run again, it prints more information about the crash:

nyanpasu64@ryzen ~/code/ntfs2btrfs/build-GCC_lld_asan-RelWithDebInfo (master)> sudo ./ntfs2btrfs /dev/nvme0n1p6
Using Zstd compression.
Using CRC32C for checksums.
Processing inode 23214 / 23214 (100.0%)
Mapped 13018 inodes directly.
Rewrote 0 inodes.
Inlined 6437 inodes.
Updating directory sizes
Calculating checksums 14862031 / 14862031 (100.0%)
=================================================================
==8895==ERROR: AddressSanitizer: heap-use-after-free on address 0x633000006470 at pc 0x562dbdecd29d bp 0x7ffe88d2a060 sp 0x7ffe88d2a050
READ of size 8 at 0x633000006470 thread T0
    #0 0x562dbdecd29c in std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >::data() const /usr/include/c++/12.1.0/bits/stl_vector.h:1261
    #1 0x562dbdecd29c in root::create_trees(root&, btrfs_csum_type) /home/nyanpasu64/code/ntfs2btrfs/src/ntfs2btrfs.cpp:684
    #2 0x562dbdef752e in convert /home/nyanpasu64/code/ntfs2btrfs/src/ntfs2btrfs.cpp:3595
    #3 0x562dbdf660c9 in main /home/nyanpasu64/code/ntfs2btrfs/src/ntfs2btrfs.cpp:3848
    #4 0x7f7f5003728f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #5 0x7f7f50037349 in __libc_start_main_impl ../csu/libc-start.c:389
    #6 0x562dbde8a5d4 in _start (/home/nyanpasu64/code/ntfs2btrfs/build-GCC_lld_asan-RelWithDebInfo/ntfs2btrfs+0x365d4)

0x633000006470 is located 23664 bytes inside of 98304-byte region [0x633000000800,0x633000018800)
freed by thread T0 here:
    #0 0x7f7f5070378a in operator delete(void*, unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:164
    #1 0x562dbdf138a4 in std::__new_allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > >::deallocate(std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, unsigned long) /usr/include/c++/12.1.0/bits/new_allocator.h:158
    #2 0x562dbdf138a4 in std::allocator_traits<std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::deallocate(std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > >&, std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, unsigned long) /usr/include/c++/12.1.0/bits/alloc_traits.h:496
    #3 0x562dbdf138a4 in std::_Vector_base<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::_M_deallocate(std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, unsigned long) /usr/include/c++/12.1.0/bits/stl_vector.h:387
    #4 0x562dbdf138a4 in std::_Vector_base<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::_M_deallocate(std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, unsigned long) /usr/include/c++/12.1.0/bits/stl_vector.h:383
    #5 0x562dbdf138a4 in void std::vector<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::_M_realloc_insert<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > const&>(__gnu_cxx::__normal_iterator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, std::vector<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > > >, std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > const&) /usr/include/c++/12.1.0/bits/vector.tcc:513

previously allocated by thread T0 here:
    #0 0x7f7f50702672 in operator new(unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:95
    #1 0x562dbdf134c5 in std::__new_allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > >::allocate(unsigned long, void const*) /usr/include/c++/12.1.0/bits/new_allocator.h:137
    #2 0x562dbdf134c5 in std::allocator_traits<std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::allocate(std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > >&, unsigned long) /usr/include/c++/12.1.0/bits/alloc_traits.h:464
    #3 0x562dbdf134c5 in std::_Vector_base<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::_M_allocate(unsigned long) /usr/include/c++/12.1.0/bits/stl_vector.h:378
    #4 0x562dbdf134c5 in void std::vector<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > >::_M_realloc_insert<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > const&>(__gnu_cxx::__normal_iterator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >*, std::vector<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >, std::allocator<std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > > > >, std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > > const&) /usr/include/c++/12.1.0/bits/vector.tcc:453

SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/12.1.0/bits/stl_vector.h:1261 in std::vector<unsigned char, default_init_allocator<unsigned char, std::allocator<unsigned char> > >::data() const
Shadow bytes around the buggy address:
  0x0c667fff8c30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8c40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8c50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8c60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8c70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c667fff8c80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd
  0x0c667fff8c90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8ca0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8cb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8cc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c667fff8cd0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==8895==ABORTING

That was a doozy.

The bug is that t is a dangling reference:

for (const auto& t : trees) {
    trees.push_back(buf);
    auto ln = (leaf_node*)((uint8_t*)t.data() + sizeof(tree_header));

You could replace t with a non-owning std::span constructed from trees[i], but it's still wrong to push into trees while iterating over it (the for loop will still error out).

IMO you should replace the foreach loop with a for (size_t i = 0; i < trees.size(); i++), and either replace all uses of t with trees[i] or apply the std::span transformation as I mentioned above. I'd also comment that trees is being pushed onto, so you can't replace the indexed loop with a foreach.

TBH pushing onto a loop while iterating over it is quite tricky to understand. I'm not sure if it's necessary complexity, or if you can refactor to preserve user-facing functionality without pushing onto a loop while iterating over it; I don't actually know how your program works.

Are there any remaining latent or crashing bugs past this? I don't know! I applied the for-loop fix and wrote trees[i] twice, and the resulting ASAN-enabled binary seems to process my partition without crashing...

I think this is the same bug as #47 and possibly #39.

Is this bad? `Compression signature was 4, expected 3.`

I got:
\Windows\System32\MRT.exe: Compression signature was 4, expected 3. with an exit code of 1. I'm not sure if it finished.
Is that bad?
Ta

Wrong compression signature

Before the update, it was giving me that free space use chkdsk error on every ntfs partition I tried. So I saw you made 2 commits and decided to try again. It is the ntfs partition of a recent install of Windows 10 20H2. The following is what happened:

[root@manjaro westly]# ntfs2btrfs /dev/sda5
Skipping overly large ADS $Extend$RmMetadata$Repair:$Corrupt (2359296 > 16215)
Skipping overly large ADS $Extend$RmMetadata$Repair:$Verify (716800 > 16216)
Skipping overly large ADS $Extend$RmMetadata$TxfLog$Tops:$T (1048576 > 16221)
Skipping overly large ADS \Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.Pb378ec07#\882818c534d0020bdc80236ecc332d62\Microsoft.PowerShell.ConsoleHost.ni.dll:WofCompressedData (347233 > 16206)
Skipping overly large ADS \Windows\assembly\NativeImages_v4.0.30319_64\PresentationCore\66450953713be99c7f54d828732cb56c\PresentationCore.ni.dll:WofCompressedData (7111770 > 16206)
Skipping overly large ADS \Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.Ic1a2041b#\5cf24dafa411c166fcce8415916f973e\Microsoft.Isam.Esent.Interop.ni.dll:WofCompressedData (651969 > 16206)
Skipping overly large ADS \Windows\assembly\NativeImages_v4.0.30319_64\System.Servd1dec626#\15aac93fa4752b025fc8fe4628fac956\System.ServiceModel.Internals.ni.dll:WofCompressedData (468345 > 16206)
Skipping overly large ADS \Windows\assembly\NativeImages_v4.0.30319_64\Microsoft.I0cd65b90#\fa21414c63b8c6a48d359248aa4c37bd\Microsoft.Isam.Esent.Interop.Wsa.ni.dll:WofCompressedData (573566 > 16206)
Compression signature was 4, expected 3.

Bad filesystem state after converting two separate NTFS partitions to BTRFS

I've converted two NTFS partitions now using this tool and both filesystems have had issues that would cause it to drop down into read only mode. Scrub seems to work okay, indicating that the data is at least not corrupted, but attempting to remove image/ntfs.img or doing a defrag will cause the mount to switch into read-only mode.

Below is what I got in my system logs after I finished converting a 2TB Western Digital My Passport external USB disk and then attempted to delete the image subvolume.

Steps to reproduce

ntfs2btrfs /dev/sdf1
mount -o compress-force=zstd:2 /dev/sdf1 "/mnt/My Passport"
btrfs subvolume delete "/mnt/My Passport/image"

Jun 16 16:41:52 username kernel: BTRFS: Transaction aborted (error -75)
Jun 16 16:41:52 username kernel: WARNING: CPU: 6 PID: 45545 at fs/btrfs/file-item.c:908 btrfs_del_csums+0x3b8/0x410 [btrfs]
Jun 16 16:41:52 username kernel: Modules linked in: nvidia_uvm(POE) ses enclosure scsi_transport_sas uinput wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libblake2s blake2s_x86_64 libcurve25519_generic libchacha libblake2s_generic ip6_udp_tunnel udp_tunnel nft_masq snd_seq_dummy snd_hrtimer snd_seq nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast ccm rfcomm nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink uvcvideo hid_logitech_hidpp videobuf2_vmalloc videobuf2_memops nvidia_drm(POE) videobuf2_v4l2 nvidia_modeset(POE) videobuf2_common snd_usb_audio videodev snd_usbmidi_lib uas snd_rawmidi snd_seq_device usb_storage mc nvidia(POE) cmac algif_hash algif_skcipher af_alg bnep joydev iwlmvm snd_hda_codec_realtek intel_rapl_msr mousedev snd_hda_codec_generic hid_logitech_dj intel_rapl_common mac80211 snd_hda_codec_hdmi ledtrig_audio snd_hda_intel vfat libarc4
Jun 16 16:41:52 username kernel:  edac_mce_amd kvm_amd ccp rng_core iwlwifi btusb snd_intel_dspcfg fat btrtl snd_intel_sdw_acpi drm_kms_helper kvm btbcm snd_hda_codec cfg80211 btintel snd_hda_core cec bluetooth snd_hwdep usbhid igb snd_pcm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel wmi_bmof snd_timer mxm_wmi syscopyarea snd sysfillrect crypto_simd i2c_algo_bit ecdh_generic cryptd sp5100_tco ecc sysimgblt rapl pcspkr fb_sys_fops rfkill crc16 soundcore dca k10temp i2c_piix4 wmi pinctrl_amd mac_hid acpi_cpufreq drm fuse crypto_user agpgart bpf_preload ip_tables x_tables btrfs blake2b_generic libcrc32c crc32c_generic xor raid6_pq xhci_pci crc32c_intel xhci_pci_renesas
Jun 16 16:41:52 username kernel: CPU: 6 PID: 45545 Comm: btrfs-transacti Tainted: P           OE     5.12.10-arch1-1 #1
Jun 16 16:41:52 username kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X570 Taichi, BIOS P4.30 04/14/2021
Jun 16 16:41:52 username kernel: RIP: 0010:btrfs_del_csums+0x3b8/0x410 [btrfs]
Jun 16 16:41:52 username kernel: Code: 8d 93 48 0a 00 00 f0 48 0f ba 2a 03 72 22 83 f8 fb 74 45 83 f8 e2 74 40 89 c6 48 c7 c7 c0 a1 55 c0 89 44 24 04 e8 15 dc 99 d2 <0f> 0b 8b 44 24 04 89 c1 ba 8c 03 00 00 4c 89 e7 89 44 24 04 48 c7
Jun 16 16:41:52 username kernel: RSP: 0018:ffffa0bbd4467b30 EFLAGS: 00010286
Jun 16 16:41:52 username kernel: RAX: 0000000000000000 RBX: ffff921a9bad6000 RCX: 0000000000000027
Jun 16 16:41:52 username kernel: RDX: ffff92209eb986e8 RSI: 0000000000000001 RDI: ffff92209eb986e0
Jun 16 16:41:52 username kernel: RBP: ffff921e54830e00 R08: 0000000000000000 R09: ffffa0bbd4467960
Jun 16 16:41:52 username kernel: R10: ffffa0bbd4467958 R11: ffff9220bf2a94a8 R12: ffff921991e300d0
Jun 16 16:41:52 username kernel: R13: ffff921b840f5800 R14: ffff921c4821eb00 R15: 0000000001336000
Jun 16 16:41:52 username kernel: FS:  0000000000000000(0000) GS:ffff92209eb80000(0000) knlGS:0000000000000000
Jun 16 16:41:52 username kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 16 16:41:52 username kernel: CR2: 00000000e706b000 CR3: 0000000252c0e000 CR4: 0000000000350ee0
Jun 16 16:41:52 username kernel: Call Trace:
Jun 16 16:41:52 username kernel:  __btrfs_free_extent+0x4e4/0x940 [btrfs]
Jun 16 16:41:52 username kernel:  __btrfs_run_delayed_refs+0x24f/0xfe0 [btrfs]
Jun 16 16:41:52 username kernel:  ? update_load_avg+0x7e/0x620
Jun 16 16:41:52 username kernel:  ? cpuacct_charge+0x32/0x80
Jun 16 16:41:52 username kernel:  btrfs_run_delayed_refs+0x73/0x1f0 [btrfs]
Jun 16 16:41:52 username kernel:  btrfs_commit_transaction+0xf6/0xb70 [btrfs]
Jun 16 16:41:52 username kernel:  ? start_transaction+0xce/0x5d0 [btrfs]
Jun 16 16:41:52 username kernel:  transaction_kthread+0x12e/0x1a0 [btrfs]
Jun 16 16:41:52 username kernel:  ? btrfs_cleanup_transaction.isra.0+0x570/0x570 [btrfs]
Jun 16 16:41:52 username kernel:  kthread+0x133/0x150
Jun 16 16:41:52 username kernel:  ? kthread_associate_blkcg+0xc0/0xc0
Jun 16 16:41:52 username kernel:  ret_from_fork+0x22/0x30
Jun 16 16:41:52 username kernel: ---[ end trace b2a10b3577fbf3e6 ]---
Jun 16 16:41:52 username kernel: BTRFS: error (device sdf1) in btrfs_del_csums:908: errno=-75 unknown
Jun 16 16:41:52 username kernel: BTRFS info (device sdf1): forced readonly
Jun 16 16:41:52 username kernel: BTRFS: error (device sdf1) in __btrfs_free_extent:3216: errno=-75 unknown
Jun 16 16:41:52 username kernel: BTRFS: error (device sdf1) in btrfs_run_delayed_refs:2163: errno=-75 unknown

I did the conversion on an up-to-date Arch system using a June 14 build of ntfs2btrfs from the AUR.

Kernel 5.12.10-arch1-1
btrfs-progs 5.12.1

Similar issues occured with a Seagate 4TB SHDD.

Non UTF-8 filenames gives vague error

Downloaded version 20210105

When I run ntfs2btrfs.exe D:\, all I get back instantly is just the message wstring_convert::to_bytes and it does nothing
And the drive I want to convert is infact D:\ and it is formatted as NTFS, has no compressed files although I see the newest version added in compressed file support.

ntfs2btrfs on windows doesn't print an explicit error when converting invalid disk drive

Running ntfs2btrfs u: (where u: doesn't exist) there's no user-visible error.
It would be helpful to be explicit in interactive mode, as not everyone can see errorlevel code.

Unable to convert my drive

Did I do this right? It doesn't work for me. Is there a package i need to install first?

Failure to convert Windows 10 20H2 boot disk

I'm following the Quibble installation procedure and I have an error at the file-system conversion step:

Index record magic was not INDX.

The target partition is a straightforward, fresh Windows 10 20H2 French physical UEFI installation on a 480 GiB NVMe drive, with all Windows updates applied, WinBTRFS 1.7.5 installed, fast boot+pagefile disabled. The error happens on both Linux (Debian 10 netinstall media with statically-linked ntfs2btrfs) and Windows (Windows 10 20H2 installation media).

Details about the installation:

PS C:\Windows\system32> systeminfo

Host Name:                 DESKTOP-NJDFLGG
OS Name:                   Microsoft Windows 10 Professionnel
OS Version:                10.0.19042 N/A Build 19042
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          Jean-Baptiste Boric
Registered Organization:
Product ID:                00330-80000-00000-AA022
Original Install Date:     26/12/2020, 15:12:53
System Boot Time:          26/12/2020, 17:16:29
System Manufacturer:       System manufacturer
System Model:              System Product Name
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: AMD64 Family 23 Model 1 Stepping 1 AuthenticAMD ~3600 Mhz
BIOS Version:              American Megatrends Inc. 5603, 28/07/2020
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             fr;French (France)
Input Locale:              fr;French (France)
Time Zone:                 (UTC+01:00) Brussels, Copenhagen, Madrid, Paris
Total Physical Memory:     32 705 MB
Available Physical Memory: 29 165 MB
Virtual Memory: Max Size:  32 705 MB
Virtual Memory: Available: 28 594 MB
Virtual Memory: In Use:    4 111 MB
Page File Location(s):     N/A
Domain:                    WORKGROUP
Logon Server:              \\DESKTOP-NJDFLGG
Hotfix(s):                 7 Hotfix(s) Installed.
                           [01]: KB4586876
                           [02]: KB4562830
                           [03]: KB4570334
                           [04]: KB4580325
                           [05]: KB4586864
                           [06]: KB4593175
                           [07]: KB4592438
Network Card(s):           3 NIC(s) Installed.
                           [01]: Intel(R) I211 Gigabit Network Connection
                                 Connection Name: Ethernet
                                 DHCP Enabled:    Yes
                                 DHCP Server:     172.16.175.2
                                 IP address(es)
                                 [01]: 172.16.175.176
                                 [02]: fe80::4825:ecfa:137d:b60a
                           [02]: Intel(R) Dual Band Wireless-AC 8260
                                 Connection Name: Wi-Fi
                                 Status:          Media disconnected
                           [03]: Bluetooth Device (Personal Area Network)
                                 Connection Name: Connexion réseau Bluetooth
                                 Status:          Media disconnected
Hyper-V Requirements:      VM Monitor Mode Extensions: Yes
                           Virtualization Enabled In Firmware: No
                           Second Level Address Translation: Yes
                           Data Execution Prevention Available: Yes

PS C:\Windows\system32> fsutil fsinfo ntfsinfo c:
NTFS Volume Serial Number :        0x5e38774e38772465
NTFS Version      :                3.1
LFS Version       :                2.0
Total Sectors     :                936 409 120  (446,5 GB)
Total Clusters    :                117 051 140  (446,5 GB)
Free Clusters     :                108 657 441  (414,5 GB)
Total Reserved Clusters :            1 895 112  (  7,2 GB)
Reserved For Storage Reserve :       1 766 319  (  6,7 GB)
Bytes Per Sector  :                512
Bytes Per Physical Sector :        4096
Bytes Per Cluster :                4096
Bytes Per FileRecord Segment    :  1024
Clusters Per FileRecord Segment :  0
Mft Valid Data Length :            206,50 MB
Mft Start Lcn  :                   0x00000000000c0000
Mft2 Start Lcn :                   0x0000000000000002
Mft Zone Start :                   0x00000000000cba40
Mft Zone End   :                   0x00000000000d2540
MFT Zone Size  :                   107,00 MB
Max Device Trim Extent Count :     256
Max Device Trim Byte Count :       0xffffffff
Max Volume Trim Extent Count :     62
Max Volume Trim Byte Count :       0x40000000
Resource Manager Identifier :      E4459AAA-4782-11EB-8D7A-97C30400D972

Question about conversion time, running on Linux.

Hello! I have 16 GB of RAM, a Ryzen 5 5600G (Base Clock 3.9 GHz, Boost Clock up to 4.4 GHz, 6 Cores/12 Thread), and Windows Explorer says I've used 857 GB out of 930 GB total in the boot partition. Do you have any idea how long that would take to convert with my specs? I intend to boot into a fresh Alpine Linux install, then build and run ntfs2btrfs.

Compiling in linux

OS: Ubuntu 20.10

I tried compiling this on a live cd and installed build-essential and cmake but it fails and the error points to fmt::fmt-header-only thing. Any ideas?

Is this a fault?

sudo ntfs2btrfs  /dev/disk/by-id/nvme-P6F3840_SE19C11P7050835-part1 
Using Zstd compression.
Using CRC32C for checksums.
Processing inode 596892 / 596892 (100.0%)
Mapped 220431 inodes directly.
Rewrote 107690 inodes.
Inlined 93062 inodes.
Updating directory sizes
Calculating checksums 9713735 / 9713735 (100.0%)
/usr/include/c++/12.1.0/bits/stl_vector.h:1123: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned char; _Alloc = default_init_allocator<unsigned char>; reference = unsigned char&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
zsh: IOT instruction  sudo ntfs2btrfs /dev/disk/by-id/nvme-P6F3840_SE19C11P7050835-part1

update arch package

In the comments of https://aur.archlinux.org/packages/ntfs2btrfs-git there are some suggestions to fix some problems with the package. I think at least some are pretty easy to fix and shouldn't take too much time, but they would increase the health of that package.
I'm also tagging the maintainer @nicman23

Arch aur: /usr/sbin exists in filesystem (owned by filesystem)

Hello!
I tried to install for arch. But I got the subject error. Do you have any idea? Thanks!

Error: Assertion '__n < this->size()' failed.

There is a bug between commit 7f32753 and
bfb5124 causing ntfs2btrfs failing at approx. 32% inodes processed (both Windows and Linux)

Converted partition is recognized as a NTFS partition on Windows

I was just able to run the NTFS2BTRFS convention on a partition in Windows with no errors and I was happy it completed but Windows is telling me that it is still NTFS even after a reboot and removing the drive letter. any fixes for this? I should mention this is not an empty partition and I'm trying to convert a partition with windows on it. (Not the one I'm currently running off of tho.)

FTBFS on Fedora with non-x86_64 architectures

I'm packaging ntfs2btrfs for Fedora, and I was caught by surprise by it failing to build on all architectures other than x86_64.

I've restricted it to x86_64 only for now, but it'd be nice if it could work on any architecture.

Issues with volume on linux

All of the following was done on linux, but the drive was converted from windows as the conversion failed on linux, with the error mentioner in #15.

I also experienced some issues with reflinking across subvolumes; all of my reflinked files would disappear within a minute, and the filesystem would get remounted read-only. Sometimes the filesystem would get remounted without any action from me. Upon further inspection, it seems that nothing commits to disk; no files/directories I create survive an unmount+remount cycle.

Running btrfsck gives the following output:

Opening filesystem to check...
Checking filesystem on /dev/sda3
UUID: fa5e11e7-8c6a-ec3b-e718-746320be1f1b
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
block group 1048576 has wrong amount of free space, free space cache has 103882752 block group has 103813120
ERROR: free space cache has more free space than block group item, this could leads to serious corruption, please contact btrfs developers
failed to load free space cache for block group 1048576
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 102513819648 bytes used, error(s) found
total csum bytes: 99907984
total tree bytes: 207847424
total fs tree bytes: 91897856
total extent tree bytes: 9797632
btree space waste bytes: 4445536
file data blocks allocated: 194846035968
 referenced 194846035968

gentoo package

The ebuild already exists, and sys-fs/ntfs2btrfs hasn't been taken yet...

Error when compiling.

I'm on linux trying to build from source. Followed the instructions on the readme, including installing the libfmt package.
When running the make command I got the following error:

ntfs2btrfs/src/compress.cpp:25:10: fatal error: lzo/lzo1x.h: No such file or directory
#include <lzo/lzo1x.h>

Am I missing a package? I have the liblzo2-2 package.

Any help would be appreciated.

SD headers do not match

hey, i cant convert my ntfs windows drive into btrfs.
I started Windows Setup with command line (which uses Windows PE i guess) and launched ntfs2btrfs.

Pagefile is disabled for that drive.
partition table is GPT.

C: drive i want to convert
D: drive containing ntfs2btrfs.exe

executing D:\>ntf2btrfs.exe C:\ gave me error SD headers do not match

Files with very long name will become not accessible after converting

I have a file with a very long name which contains non-ascii characters (Japanese and its marks). After converting the partition from NTFS to btrfs under Windows, the file become inaccessible with Winbtrfs. The file cannot be read or deleted with explorer.exe. The partition cannot be mounted under Linux with following error:

[  773.543848] BTRFS info (device sdd1): disk space caching is enabled
[  773.543851] BTRFS info (device sdd1): has skinny extents
[  774.873924] BTRFS critical (device sdd1): corrupt leaf: root=2 block=1489133600768 slot=176 bg_start=4000763084800 bg_len=23003136, invalid block group used, have 23064576 expect [0, 23003136)
[  774.873927] BTRFS error (device sdd1): block=1489133600768 read time tree block corruption detected
[  774.873934] BTRFS error (device sdd1): failed to read block groups: -5
[  774.890804] BTRFS error (device sdd1): open_ctree failed

In NTFS, file names can be at most 255 unicode characters. While in btrfs, the limit is 255 Bytes. Maybe we need truncate names exceeded the length limit?

create into subvolume

wouldnt it make sense to create the windows/ntfs root into a subvolume if possible?
Think about making a snapshot with the initial state, you tinker around and want to restore the snapshot. The beautiful way is to have the root filesystem as a manager for snapshots, holding nothing but the snapshot folders (including the active subvolume).
If we use the snapshots as is at the moment, you would set-default the initial snapshot or a copy of it. That leaves behind the root subvolume 5 filled with windows files, that are used.

current solution:

root (subvolume 5/set-default):
- snapshots - convertedWindows (readonly)
            - snapshot 20210101 (readonly)
            - snapshot 20210201 (readonly)
            - ...
- Users
- Windows
- ...

prefered way:

root (subvolume 5):
- current (set-default)
- convertedWindows (readonly)
- 20210101 (readonly)
- 20210201 (readonly)
- ...

sbin install path should be relocatable (cmake)

Currently ntfs2btrfs gets installed to sbin by default, however some distributions (e.g. Void Linux) do not allow this as they symlink sbin to bin. A cmake flag to set this would be useful in order to get it packaged.

Metadata with Large ADS

Hi, I'm trying to convert my data drive (mainly contains games) to btrfs, but am getting the following error:

Skipping overly large ADS $Extend$RmMetadata$Repair:$Corrupt (9216000 > 16215)
Skipping overly large ADS $Extend$RmMetadata$Repair:$Verify (4096000 > 16216)
Skipping overly large ADS $Extend$RmMetadata$TxfLog$Tops:$T (1048576 > 16221)

I can't seem to find any way to resolve this, do you know of a way? I've tried a couple fsutil commands I found on the web with no luck.

Thanks!

Converted partition is broken and recognized as a NTFS partition on linux

Hello, thanks for your promising work! I have converted a partition (the one in the previous issue) into Btrfs using this tool. The converted partition was recognized as Btrfs in Windows with the Winbtrfs driver. However, Linux says that it is a NTFS partition and some errors were found by btrfsck:

keuin@keuin-galaxy-ng:~$ sudo btrfsck /dev/sdd1
Opening filesystem to check...
Checking filesystem on /dev/sdd1
UUID: 7b7c8a67-b0d3-a95c-6512-3a7e44f24caf
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
cache and super generation don't match, space cache will be invalidated
[4/7] checking fs roots
root 5 inode 81055 errors 1, no inode item
root 5 inode 81110 errors 1, no inode item
root 5 inode 81158 errors 1, no inode item
root 5 inode 81213 errors 1, no inode item
root 5 inode 81266 errors 1, no inode item
root 5 inode 81321 errors 1, no inode item
root 5 inode 81373 errors 1, no inode item
root 5 inode 81430 errors 1, no inode item
root 5 inode 81482 errors 1, no inode item
root 5 inode 81532 errors 1, no inode item
root 5 inode 81584 errors 1, no inode item
root 5 inode 81639 errors 1, no inode item
root 5 inode 81695 errors 1, no inode item
root 5 inode 81751 errors 1, no inode item
root 5 inode 81804 errors 1, no inode item
root 5 inode 81863 errors 1, no inode item
root 5 inode 81918 errors 1, no inode item
root 5 inode 81973 errors 1, no inode item
root 5 inode 82028 errors 1, no inode item
root 5 inode 82083 errors 1, no inode item
root 5 inode 82139 errors 1, no inode item
root 5 inode 82190 errors 1, no inode item
root 5 inode 82242 errors 1, no inode item
root 5 inode 82293 errors 1, no inode item
root 5 inode 82347 errors 1, no inode item
root 5 inode 82399 errors 1, no inode item
root 5 inode 82451 errors 1, no inode item
root 5 inode 82505 errors 1, no inode item
root 5 inode 82560 errors 1, no inode item
root 5 inode 82617 errors 1, no inode item
root 5 inode 82672 errors 1, no inode item
root 5 inode 82723 errors 1, no inode item
root 5 inode 82778 errors 1, no inode item
root 5 inode 82831 errors 1, no inode item
root 5 inode 82884 errors 1, no inode item
root 5 inode 82937 errors 1, no inode item
ERROR: errors found in fs roots
found 3096923078656 bytes used, error(s) found
total csum bytes: 3020416752
total tree bytes: 4016324608
total fs tree bytes: 388022272
total extent tree bytes: 189415424
btree space waste bytes: 366564742
file data blocks allocated: 6154863652864
 referenced 6154863652864

fdisk -l says:

Disk /dev/sdd: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: WDC WD40EJRX-89A
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B59B6CA3-DFC4-4811-ABEF-86C4183103A8

Device     Start        End    Sectors  Size Type
/dev/sdd1   2048 7814035455 7814033408  3.6T Microsoft basic data

Windows reported that this partition was not a NTFS one, but Linux did. This is weird.

Before converting, I have ran CHKDSK /F to ensure the integrity and it reported no errors. Since btrfsck --repair is highly discouraged by many instructions available on the Internet, I did not run that. I have no idea if this is a bug of ntfs2btrfs.

By the way, this partition was previously Bitlocker encrypted and had been decrypted before converting. Is this a potential issue? I can help with further inspection.

Convert BTRFS back to NTFS?

Is there a way to convert BTRFS back to NTFS?

I think this would be good to have as a revert mechanisk if there is an issue with the whole installation and booting of Windows on BTRFS+Quibble.

Crash while processing inodes

C:\Users\Florens\Downloads\ntfs2btrfs-20220812>.\ntfs2btrfs.exe D:\
Using Zstd compression.
Using CRC32C for checksums.
Processing inode 43 / 197774 (0.0%)

I have tried mutiple times and it always crashes at 43. Also I have very high memory usage in this step, it's using all 16GB of RAM and a lot in the pagefile as well.

Error assigning space to file despite running chkdsk

Whenever I try to convert a specific NTFS partition to BTRFS, I'll get
sudo ntfs2btrfs /dev/sda4
Error assigning space to file. This can occur if the space bitmap has become corrupted. Run chkdsk and try again.
Even after trying chkdsk /f and running btrfs2ntfs on Windows, it will still output the same error...

maharmstone / ntfs2btrfs Goto Github PK

ntfs2btrfs's People

Contributors

Stargazers

Watchers

Forkers

ntfs2btrfs's Issues

Recommend Projects

Recommend Topics

Recommend Org