Incorrect size when doing preallocation about btrfs HOT 5 OPEN

Python seems to use GetFileInformationByHandle to get a BY_HANDLE_FILE_INFORMATION and uses its nFileSizeLow and nFileSizeHigh.

A simple dir command will also show the incorrect size:

(.venv) H:\dev\openzfs>dir d:
 Volume in drive D has no label.
 Volume Serial Number is 609A-6821

 Directory of D:\

08/10/2023  06:57               512 testfile.bin  
               1 File(s)            512 bytes     
               0 Dir(s)  10,717,904,896 bytes free

It should show 117 here with the test program. Rclone is a good tool to test this too, it always preallocates files with multiple of the volumes cluster size:

https://github.com/rclone/rclone/blob/bcb3289dad58efad617eacd2ee1103b51a6ee3cf/lib/file/preallocate_windows.go#L74

But when it writes less than that it errors later because it expects the created file to be the size of actually written data after close.

from btrfs.

Is there a userspace API to get all 3 sizes? I'd like to get all 3 in my test and compare them to NTFS & ReFS.

Edit: seems like there is no userspace way to get the valid data length, see https://stackoverflow.com/questions/35572871/how-to-get-valid-data-length-of-a-file

from btrfs.

Python's not very useful here, as you have to dig deep to know what it's actually doing. It'd be better for there to be a C or C++ program using ntdll calls (not kernel32).

the file always shows the preallocated size

Do we know which size os.path.getsize returns? There's three on Windows: allocation size, end of file, and valid data length.

from btrfs.

As a side comment, just wanted to mention it, the set_validdatalength() function:

https://github.com/maharmstone/btrfs/blob/master/src/fileinfo.c#L3754
The "<=" in
if (fvdli->ValidDataLength.QuadPart <= fcb->Header.ValidDataLength.QuadPart ...
and I could never get the call to succeed, even with your test.exe and ifstest.exe. Always fails.

Then I noticed:
https://github.com/microsoft/Windows-driver-samples/blob/main/filesys/fastfat/fileinfo.c#L4847
has a simple "<" instead. With that, the code will occasionally trigger the valid path with test.exe and ifstest.exe.

from btrfs.

FYI the test I added in openzfsonwindows/openzfs#284 also works on other filesystems, I tested on NTFS, ReFS, FAT32, ZFS and SMB. It doesn't have dependencies other than Python. You can just call it with --path <path to btrfs dir> --no_pool so it doesn't try to create a zpool and uses the path as a test directory instead.

from btrfs.