openzfsonwindows / zfsin Goto Github PK

The new CACHE code is not quite right, and will generate the above unhelpful error message. The CACHE code will be by-passed until it can be debugged properly.

I would like to try ZFS in VMware but I failed to find any instructions about installing released binary. I don't want to compile source code. I just want to try if latest build is working.

As recommended at https://news.ycombinator.com/item?id=15549190#15549410 a possibly better way to handle that IO completion callback coming back at a different IRQ level (calling ZIO continue will die at first mutex call), is to use Deferred Procedure Calls (DPC) to call ZIO continue.

Right now, I spawn a thread which goes into sleep on Event, and IO completion routine just triggers the Event. thread resumes and calls ZIO continue.

https://github.com/lundman/ZFSin/blob/master/ZFSin/zfs/module/zfs/vdev_disk.c#L523
https://github.com/lundman/ZFSin/blob/master/ZFSin/zfs/module/zfs/vdev_disk.c#L470
https://github.com/lundman/ZFSin/blob/master/ZFSin/zfs/module/zfs/vdev_disk.c#L728

Wrote 8 or so GB to the drive with repeated dd runs and then ran CrystalDiskMark default test, resulting in the panic. See attached debug information.
20180918c.txt

Reproduce:

• Open file with notepad.exe (or right click edit).
• Edit it, save.
• Reopen it: It shows old content.
• However 'cat file' (powershell) shows correct content.

Given that 'cat' and notepad give different content: Looks like it's served from different cache?
Looks like one is served via ZFS, one from Windows cache.

More investigations will follow.

The test command git clone https://github.com/lundman/ZFSin.git will eventually get the following request

  FileName: \ZFSin\.git\objects\pack\pack-18ffe57b902f601b8f93f745d0eaf79d55425f29.pack offset 0x20000 len 0x8000 mdl FFFF8087E4086B60 System 0000000000000000

and we start in fs_read() for it;

	uio = uio_create(1, byteOffset.QuadPart, UIO_SYSSPACE, UIO_READ);
	ASSERT(address != NULL);
	uio_addiov(uio, address, bufferLength);
	error = zfs_read(vp, uio, 0, NULL, NULL);

In this case, the allocations are (lower 32bits)

uio = 0xebb4bb80 (size 0x40)
uio->iov = 0xeb44ddd0 (size 0x10)
uio->iov->iov_base[0] = 0x15a18000

Nothing special about that, we will be keeping an eye on uio->iov.

So, we enter zfs_read() and the stack looks like

	ZFSin!dmu_buf_hold_array_by_dnode+0x1d9 [c:\src\zfsin\zfsin\zfs\module\zfs\dmu.c
 	ZFSin!dmu_read_uio_dnode+0x78 [c:\src\zfsin\zfsin\zfs\module\zfs\dmu.c @ 1194]
 	ZFSin!dmu_read_uio_dbuf+0x60 [c:\src\zfsin\zfsin\zfs\module\zfs\dmu.c @ 1264]
 	ZFSin!zfs_read+0x43b [c:\src\zfsin\zfsin\zfs\module\zfs\zfs_vnops.c @ 725]
 	ZFSin!fs_read+0x343 [c:\src\zfsin\zfsin\zfs\module\zfs\zfs_vnops_windows.c @ 2118]

which is pretty much a straight shot down. (up).

dmu_buf_hold_array_by_dnode

	dbp = kmem_zalloc(sizeof (dmu_buf_t *) * nblks, KM_SLEEP);

Here nblks = 2 so the size allocated is 0x10. The inside of kmem looks like

void *
zfs_kmem_zalloc(uint32_t size, int kmflag)
{
	uint32_t index;
	void *buf;

	if ((index = ((size - 1) >> KMEM_ALIGN_SHIFT)) < KMEM_ALLOC_TABLE_MAX) {
		kmem_cache_t *cp = kmem_alloc_table[index];
		buf = kmem_cache_alloc(cp, kmflag);

Where cp points to kmem_alloc_16, so that is all good. Inside that

	for (;;) {
		/*
		 * If there's an object available in the current CPU's
		 * loaded magazine, just take it and return.
		 */
		if (ccp->cc_rounds > 0) {
			buf = ccp->cc_loaded->mag_round[--ccp->cc_rounds];

Here, cc_rounds == 2 before the decrement, so it assigns [1] and the value of buf = 0xeb44ddd0, which is the same address as uio->iov. :(

The following bzero() (we are in zalloc) blows the data away, and we are going down a dark road.

So both allocs come from kmem_alloc_16 with correct cp and size is 0x10. Nothing has been freed. It is somewhat undesirable to return the same value twice. The cpu cache (controlled by cpu_number()) has been tested to always return 0, which makes no difference.

Import, generate data.
Click on Windows Explorer, BOOM
Not specifying the drive, though it is mapped to D:

See attached debug input
dbg-2018-09-16-9.txt

LONG because of so much success.

Touching 10,000 files is reliable.

As displayed in the Are you sure? dialog box

Attempt to find out where this size comes from and why it is wrong.

Copying or moving data from a zpool fails with error "More data is available".
The same error occurs when trying to read a directory with the command shell.

While testing large file copies on windows, I recognized that at some point the whole memory in my zpool was occupied, even though I deleted the files after copying.
To rule out that it is not a problem with copying on windows, I created a 1GB file with FreeBSD and deleted it on windows.
This is the free space before the deletion:

And this after deletion:

As you can see, the free space got even less (~50kB), really strange ... 😳
I placed the file in the root directory, don't know if that has any impact ..?

It seems that on re-import windows gets the free space right again:

Currently there is no connection between the uid/gid on ZFS disk, and the Windows users. Research other software solutions, like smb, ext3, btrfs etc, and attempt to find a good workable solution. Keep in mind we can probably use ZFS ZAP, or SA, to store any ID hashing needed.

Primary disk is a spinning disk with a standard FreeBSD bsdinstall(8) GPT/UEFI/MBR layout.

zpool.exe will not detect the zpool for importation but FreeBSD will recognize the Windows zpool on the second physical disk.

Steps to reproduce:
Import pool
Change directory to pool's "implicit dataset" (top directory)
Edit a file in VIM with a leftover from the previous session (i.e. .touch-files.sh.swp for file touch-files.sh)$ cat .touch-files.sh.swp
U3210#"! U¦[¦C¦¦@!dexterT430/cygdrive/d/touch-files.shutf-8
Using Cygwin, not sure if that changes anything.
See attached debug output.
dbgview2018-09-15-4.txt

While lz4 compression has been working, it is not enabled in this snapshot, either at pool creation time or after.

Confirmed with: openzfsonwindows

Thanks a lot to the author of this awesome project, being able to natively access ZFS pools from Windows would be awesome. Do you also plan to add support for native encryption to this driver or it's something not really feasible do accomplish?

Although the quick read of xattrs on Windows looked to be mostly just writing the IRP handlers for it, and convert to ZFS.

Steps to reproduce:
Import pool
Change directory to pool's "implicit dataset" (top directory)
Edit file in VIM
PANIC
dbgview2018-09-15-6.txt

Using Cygwin, not sure if that changes anything.
See attached debug output.

I used devcon.exe install ZFSin.inf ROOT/ZFSin to install the driver.

After that I tried to uninstall it through the device manager. Right-clicked "ZFSin Controller" -> Uninstall device, ticked "Delete driver software for this device." -> Uninstall -> Reboot.

When I execute zpool status after that, I get no pools available, so the driver is still there...
Probably something wrong with the .inf file?

On the "implicit" (top of the pool) dataset:

dexter@T430 /cygdrive/d
$ touch foo
touch: setting times of 'foo': Invalid request code

dexter@T430 /cygdrive/d
$ touch ~/cyghome/foo

Attaching debug information.
dbg-2018-09-17-1.txt

The logged commands below should be:
zpool.exe import -N tank
zpool export tank

but, obviously, something goes wrong

2018-07-09.16:34:32 zpool.exe import -N tank
2018-07-09.16:39:51 þÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺþÊݺ

https://github.com/lundman/ZFSin/blob/master/ZFSin/zfs/lib/libspl/posix.c#L831

getline is currently only used for password input, but leaves ECHO on so the input is displayed as the user types it.

getch() will not work here, as we need input to come from stream which can be the console, or pipe, or file. It would be nice if the solution found works in regular CMD as well as bash and other fancy shells. See the win_isatty() function higher up.

Hello,
I try to install zfs on win10entreprise on my laptop (no vm) after reading the install procedure.

What can i do?

Is it possible to make it compatible with Windows Server 2003 SP2?

This just got posted to Hacker News (https://news.ycombinator.com/item?id=15549190), where a ton of people are going to learn about it. It'll probably end up on Reddit soon too, if it hasn't already.

I don't see a license file anywhere. Now would be a good time to clarify the license.

AFAIK, the difference between BSD and MIT is that (IIRC!!!) BSD requires you to cite the original author(s), while MIT does not.

When mentioning licenses I usually link to the https://en.wikipedia.org/wiki/Sybase_Open_Watcom_Public_License as well, primarily because it makes for good food for thought. (In certain ways it's stricter than the GPL, but these differences also make it GPL incompatible).

Now that most tests can run and pass, we can look at the smaller issues as well. Although the relative-path tests BSOD, so I'll look at that first.

There are a few failures when running ifstest.exe regarding the correct behaviour of flags to IRP_MJ_CREATE.

-------------------------------
Group: OpenCreateGeneral
NonDirectoryFileOpenTestTest         :NonDirectoryFileOpenTest
Group        :OpenCreateGeneral
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :C0000034 STATUS_OBJECT_NAME_NOT_FOUND
ExpectedNtStatus :C00000BA STATUS_FILE_IS_A_DIRECTORY
Description  :{Msg# OpCreatG!nondirf!10} Attempt to open a directory
              with NON DIRECTORY FILE specified. This operation
              did not return the expected status.

Looks like wrong errorcode returned, easy.

-------------------------------
Group: OpenCreateGeneral
OpenVolumeTestTest         :OpenVolumeTest
Group        :OpenCreateGeneral
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :00000000 STATUS_SUCCESS
ExpectedNtStatus :C0000043 STATUS_SHARING_VIOLATION
Description  :{Msg# OpCreatG!openvol!11} Attempting to open the
              volume with ShareAccess 0. This can only work if
              there are NO open handles. We were expecting a failure
              since we have 1 open handle.

Guess volumes IRP_MJ_CREATE needs to have a open counter, and return error in the non-zero case here.

-------------------------------
Group: OpenCreateGeneral
FileNameLengthTestTest         :FileNameLengthTest
Group        :OpenCreateGeneral
Status       :C0000021 (IFSTEST_TEST_FILENAME_FAILURE)
LastNtStatus     :C0000161 STATUS_ILLEGAL_CHARACTER
ExpectedNtStatus :00000000 STATUS_SUCCESS
LastGeneralValue :00000200
TestVariant  :00010000 TEST_VARIANT_FILE
Description  :{Msg# OpCreatG!fnlen!10} The file name length test
              failed. This was at length 512 with the boundary is
              1024.

Not sure this is fixable

-------------------------------
Group: OpenCreateGeneral
UnicodeOnDiskTestTest         :UnicodeOnDiskTest
Group        :OpenCreateGeneral
Status       :C000002E (IFSTEST_TEST_UNICODE_NAME_PRESERVED)
LastNtStatus     :6CA52CE8 ?
ExpectedNtStatus :00000000 STATUS_SUCCESS
Description  :{Msg# OpCreatG!unicode!12} Failure quering directory
              for a file/directory with unicode name.

Should work, but clearly doesn't. Debugging needed

-------------------------------
Group: OpenCreateGeneral
CaseSensitiveTestTest         :CaseSensitiveTest
Group        :OpenCreateGeneral
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :C0000002 STATUS_NOT_IMPLEMENTED
ExpectedNtStatus :00000000 STATUS_SUCCESS
Description  :{Msg# OpCreatG!casesen!12} Failure while querying
              internal information of test file \casesen\nesesac.dat.
              We need this unique number to verify case sensitivity.

Missing FileID support

-------------------------------
Group: OpenCreateParameters
CreateFileTestTest         :CreateFileTest
Group        :OpenCreateParameters
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :00000000 STATUS_SUCCESS
ExpectedNtStatus :C0000035 STATUS_OBJECT_NAME_COLLISION
Description  :{Msg# OpCreatP!crfile!14} An attempt was made to
              create an existing file \crfile.dat in the directory
              \??\E:\run1\opcreatp.  The status returned should
              have been: object name collision.  Check the last NT
              status returned.

Looks like we overwrite when we shouldn't. Connect Windows flag to ZFS flag. Easy

-------------------------------
Group: OpenCreateParameters
CreateFileDirTestTest         :CreateFileDirTest
Group        :OpenCreateParameters
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :00000000 STATUS_SUCCESS
ExpectedNtStatus :C0000035 STATUS_OBJECT_NAME_COLLISION
Description  :{Msg# OpCreatP!crfiled!14} An attempt was made to
              create an existing file \crfiled in the directory
              \??\E:\run1\opcreatp.  The status returned should
              have been: object name collision.  Check the last NT
              status returned.

Same

-------------------------------
Group: OpenCreateParameters
OpenAlwaysFileTestTest         :OpenAlwaysFileTest
Group        :OpenCreateParameters
Status       :C000001F (IFSTEST_TEST_IOSTATUSBLOCK_FAILURE)
LastGeneralValue :00000002
ExpectedGeneralValue :00000001
Description  :{Msg# OpCreatP!opalfile!15} The Io Status Block did
              not say the file \opalfile.dat in the directory
              \??\E:\run1\opcreatp was opened.  The Io Status Block
              contains results of Nt function calls.

Looks like we should return something to say if file was created or not. Easy

-------------------------------
Group: OpenCreateParameters
OverwriteFileTestTest         :OverwriteFileTest
Group        :OpenCreateParameters
Status       :C000001F (IFSTEST_TEST_IOSTATUSBLOCK_FAILURE)
LastGeneralValue :00000001
ExpectedGeneralValue :00000003
Description  :{Msg# OpCreatP!ovwrfile!14} The Io Status Block did
              not say the file \ovwrfile.dat in the directory
              \??\E:\run1\opcreatp was overwritten.  The create
              function returned back a success status on the actual
              overwrite but the IoStatusBlock did not contain the
              status.

same

-------------------------------
Group: OpenCreateParameters
SupersedeFileTestTest         :SupersedeFileTest
Group        :OpenCreateParameters
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :C0000034 STATUS_OBJECT_NAME_NOT_FOUND
ExpectedNtStatus :00000000 STATUS_SUCCESS
Description  :{Msg# OpCreatP!suprfile!11} A failure was encountered
              when trying to create the file \elifrpus.dat in the
              directory \??\E:\run1\opcreatp.  This create was
              called with a disposition of file supersede.  Check
              the last NT status returned.

Map Windows flag to ZFS flag again.

-------------------------------
Group: OpenCreateParameters
ReadOnlyAttributeTestTest         :ReadOnlyAttributeTest
Group        :OpenCreateParameters
Status       :C000001E (IFSTEST_TEST_NTAPI_FAILURE_CODE)
LastNtStatus     :00000000 STATUS_SUCCESS
ExpectedNtStatus :C0000022 STATUS_ACCESS_DENIED
LastGeneralValue :00000001
Description  :{Msg# OpCreatP!roattr!10} An attempt was made to open
              the read only file \roattr\roattr.dat in the directory
              \??\E:\run1\opcreatp with the write data access set.
              This should have failed with a access denied error.
              Check the last NT status.

Neat, you can create files read-only, guess we need to map this flag to Unix file permissions.

-------------------------------
Group: OpenCreateParameters
NormalAttributeTestTest         :NormalAttributeTest
Group        :OpenCreateParameters
Status       :C000001A (IFSTEST_TEST_NORMAL_ATTRIBUTE_ERROR)
LastGeneralValue :00000080
ExpectedGeneralValue :00000020
Description  :{Msg# OpCreatP!normattr!12} The ARCHIVE attribute
              was not set on the file normattr.dat in the directory
              \??\E:\run1\opcreatp.  This attribute was set at file
              creation time.

Map Windows attributes to ZFS attributes. We already have code for this to map OSX "bsd" to ZFS, in zfs_vnop_windows_lib.c so make it be called, and update flags for windows.

-------------------------------
Group: OpenCreateParameters
DirectoryAttributeTestTest         :DirectoryAttributeTest
Group        :OpenCreateParameters
Status       :C000001A (IFSTEST_TEST_NORMAL_ATTRIBUTE_ERROR)
LastGeneralValue :00000080
ExpectedGeneralValue :00000020

same

Nothing happens when properties is selected, many queries are issued though

  fsDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL fsDeviceObject
IOCTL_MOUNTDEV_QUERY_DEVICE_NAME
replying with '\'
dispatcher: exit: 0x80000005 Overflow Information 0x4
dispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL
  fsDispatcher: enter: major 14: minor 0: IRP_MJ_DEVICE_CONTROL fsDeviceObject
IOCTL_MOUNTDEV_QUERY_DEVICE_NAME
replying with '\Device\ZFS{0b1bb601-af0b-32e8-a1d2-54c167af6277}'
dispatcher: exit: 0x0 OK Information 0x66
dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE
  fsDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE fsDeviceObject
dispatcher: exit: 0x0 OK Information 0x0
dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE
  fsDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE fsDeviceObject
IRP_MJ_CREATE: FileObject FFFFC68633F76320 related 0000000000000000 name '\Pippin3.jpg\' flags 0x0
zfs_vnop_lookup: enter
+zget 4
+zget 9
dispatcher: exit: 0x0 OK Information 0x1
dispatcher: enter: major 13: minor 0: IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_USER_FS_REQUEST)
  fsDispatcher: enter: major 13: minor 0: IRP_MJ_FILE_SYSTEM_CONTROL(IRP_MN_USER_FS_REQUEST) fsDeviceObject
    FSCTL_GET_REPARSE_POINT
dispatcher: exit: 0xc0000275 <*****> Information 0x0
dispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP
  fsDispatcher: enter: major 18: minor 0: IRP_MJ_CLEANUP fsDeviceObject
IRP_MJ_CLEANUP: iocount 0 usecount 2
dispatcher: exit: 0x0 OK Information 0x0
dispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE
  fsDispatcher: enter: major 2: minor 0: IRP_MJ_CLOSE fsDeviceObject
IRP_MJ_CLOSE: iocount 0 usecount 1
dispatcher: exit: 0x0 OK Information 0x0
dispatcher: enter: major 0: minor 0: IRP_MJ_CREATE
  fsDispatcher: enter: major 0: minor 0: IRP_MJ_CREATE fsDeviceObject
IRP_MJ_CREATE: FileObject FFFFC68633F76320 related 0000000000000000 name '\Pippin3.jpg' flags 0x0
zfs_vnop_lookup: enter
+zget 4
+zget 9
dispatcher: exit: 0x0 OK Information 0x1
dispatcher: enter: major 5: minor 0: IRP_MJ_QUERY_INFORMATION
  fsDispatcher: enter: major 5: minor 0: IRP_MJ_QUERY_INFORMATION fsDeviceObject
* query_information: FileStreamInformation
file_stream_information: 
dispatcher: exit: 0x0 OK Information 0xffffc6a6

Version in title. Output:

PS C:\Users\dexter\Desktop> .\zpool.exe export tank
(ioctl 0x805 status 0 bytes 4548)
(ioctl 0x804 status 0 bytes 4548)
(ioctl 0x812 status 0 bytes 4548)
(ioctl 0x805 status 0 bytes 4548)
(ioctl 0x814 status 3 bytes 4548)
(ioctl 0x815 status 0 bytes 4548)
(ioctl 0x815 status 3 bytes 4548)
(ioctl 0x803 status 16 bytes 4548)
cannot export 'tank': pool is busy

Same output for:

PS C:\Users\dexter\Desktop> .\zpool.exe destroy tank

Keep up the good work!

Creating a new folder, and leaving the name New Folder works, and renaming it to another name as a separate process also works. But creating a new folder, and changing the name as part of the same task will corrupt stack.

if (vnode_isdir(ZTOV(szp))) {
		if ((error = zfs_rename_lock(szp, tdzp, sdzp, &zl)))

sets up the zl structure, which is from kmem_alloc(). Inside is mostly NULLs.

Then the return of this call

	error = dmu_tx_assign(tx, waited ? TXG_WAITED : TXG_NOWAIT);

corrupts the inside of zl. Cause unknown.

Hi
at first : awsome job to port ZFS for windows. I really needed this. Thanks alot !!

Now I am trying to install the driver to Windows Server 2012 R2 running in an VMWare ESXi virtual Machine. I try to install the latest binary (OpenZFSonWindows-20180202).

I set bcdedit.exe -set testsigning on
and also I set
bcdedit.exe /set nointegritychecks on
bcdedit.exe /set LoadOptions DDISABLE_INTEGRITY_CHECKS

Also I booted with the "disable driver signing", but I still get
Unable to open \.\ZFS: No error.
Failed to initialize the libzfs library.
when running zpool status.

What am I dooing wrong ?

Error:

• In Windows Explorer, on ZFS Volume. Right-Click->New Text File.
• It shows Error dialog: 0x8000FFFF Catastrophic failure.
• After confirming, the file is actually created just fine.

It fails on the missing `FileAllocationInformation'. If that is implemented very crudely, it works.
The very crude implementation to confirm that its the missing FileApplicationInfo call: #23

I tried to copy a few dummy .txt-files to the zpool one by one. After some files vnode_put is called with the callstack below and the vnode->iocount is already 0.
I would suggest this is a timing problem. I have the feeling, that the problem occurs earlier, when I copy the files faster one behind the other.

>	ZFSin!vnode_put+0xa7 [c:\users\julianh\source\repos\zfsin\zfsin\spl\module\spl\spl-vnode.c @ 767]	C/C++/ASM
 	ZFSin!spl_rele_async+0x25 [c:\users\julianh\source\repos\zfsin\zfsin\spl\module\spl\spl-vnode.c @ 586]	C/C++/ASM
 	ZFSin!taskq_thread+0x494 [c:\users\julianh\source\repos\zfsin\zfsin\spl\module\spl\spl-taskq.c @ 1610]	C/C++/ASM
 	nt!PspSystemThreadStartup+0x47	C/C++/ASM
 	nt!KiStartSystemThread+0x16	C/C++/ASM

In commit ad3380c some hackery was done with ZIL, which leads to the problem, that file transfers take a long time to finish (if they even come to an end). When copying or moving files into the zpool, the explorer shows the progress bar, which then stops at 99%. Sometimes the process finishes by itself after 5-10 minutes. In other cases I did not have had enough patience to wait for it...
When setting the sync property of the mounted dataset to "disabled", single small files can be copied instantly. (In my understanding, enforcing asynchronous operation should get ZFS to not use ZIL, as it is intended to accelerate synchronous operation)
Nevertheless, large files or a large number of files still cause the file transfer to hang. But that is another problem I guess...

When trying to access second level folders (and probably also deeper folders and maybe also files?) I get an access denied error. Editing files and folders on the top level is also not allowed.
File/folder creation on the top level works and I am even able to modify files that I have created in that session. But after a remount of the dataset I get an access error on that files, too.
I suspect the causal change was made somewhere between prerelease 0.03 and 0.04?

32bit version of ZFSin... is it possible?

When verifying the zpool history on a ZFSin-created pool under FreeBSD, the zpool history appears to be inaccurate or incomplete:

zpool history tank
History for 'tank':
-1704805-09-07.09:48:22
2018-05-28.19:48:01 zpool import -f -R /media tank
2018-05-28.19:50:05 zfs create tank/dataset1
2018-05-28.19:50:07 zfs create tank/dataset2
1969-12-31.16:00:00 zfs.exe snap tank/dataset1@send
2018-06-01.22:27:29 zpool import -f tank
2018-06-01.22:30:06 zpool scrub tank
2018-06-01.22:31:54 zpool export tank
2018-06-01.23:11:42 zpool import -f tank

The original creation command is absent and one data is 1969-12-31.16:00:00

Currently we claim to support NAMED_STREAMS, and they should be mapped as XATTR, but currently if they are used they are exposed:

-rw-r--r-- 1 WDKRemoteUser 197121      27 May 30 14:11 ZFSuninstall.bat
-????????? ? ?             ?            ?            ? ZFSuninstall.bat:Zone.Identifier
-rwxr-xr-x 1 WDKRemoteUser 197121 1698304 May 30 14:11 zpool.exe*
-????????? ? ?             ?            ?            ? zpool.exe:Zone.Identifier
-rwxr-xr-x 1 WDKRemoteUser 197121  163328 May 30 14:11 zstreamdump.exe*
-????????? ? ?             ?            ?            ? zstreamdump.exe:Zone.Identifier

Easily tested by drag'n'drop files from a .ZIP archive to ZFS.

After importing and exporting the "zroot"-zpool in Windows, the FreeBSD-system can not be booted anymore.
Mounting from zfs:zroot/ROOT/default failed with error 22

Reproduce:
- Create a bunch of files
- Delete them
- Files are gone, but space isn't freed up.

More investigation will follow.

Binary install says:

ERROR
The DEMO version of QSetup may only be used for evaluation
Setup Aborted!

Even though the virtual volumes we create to simulate a mount work, they are not quite right. No calls to mountmanager to set drive letter or path will work, so only the initial drive letter set will work.

This turns up again for ZVOLs when we create volumes to use, they are not quite recognised, so you can not do anything useful with them.

This needs someone who knows more about volumes to take a peek and see if they can't be made more correctly.

Looks like its just a matter of defining it correct during init, and some special conditionals in IRP handling.

Put it on the TODO.

To handle EFI partitions, a possible good solution would be to take the sources for disk.sys kindly opensourced by Microsoft.

Rename it efidisk.sys or similar, and remove the regular partition support (So that MS is still controlling that part). Then link with libefi, and create the Volumes as needed but for EFI partitions instead.

We could even add code to SET_PARTITION_INFORMATION as well, and properly handle creating of pools on Windows to be the same partition layout as upstream.

In addition to this, if the functions for creating volumes (and removing) can be called directly, the ZFS current sources can call them for the MOUNT, and ZVOL creation. Since now, that Volume code is not quite right, and hopefully disk.sys's Volume code will be perfection.

Should be about a week's worth of work.

When NTFS is case insensitive, and ZFS (case-sensitive) is mounted inside it (C:\Pool) the name lookups with be in UPPERCASE and fail.

One solution is to require ZFS to be created case-insensitive if mounted inside NTFS.

Following the SYSTEM SERVICE EXCEPTION panic, two attempts to import the pool result in this error.

See attached debug information.
20180918d.txt

Once in a while a vnode is recycled via fsdispatcher()->IRP_MJ_CLEANUP->VN_RELE(vp)->...
After that the cache file (if it exists) shall be purged with

SECTION_OBJECT_POINTERS *section;
section = vnode_sectionpointer(vp);
if (/*(IrpSp->FileObject->Flags & FO_CACHE_SUPPORTED) &&*/ section && section->DataSectionObject) {
	CcPurgeCacheSection(section, NULL, 0, FALSE);
}

This fails with BSOD because every parameter of the vnode is something like 0xdeadbeef after recycling, which is obviously an invalid value.

dprintf output:

IRP_MJ_CLEANUP: '\usr\share\man\man3' iocount 1 usecount 0
  zfs_vnop_recycle: releasing zp FFFF9988462507B8 and vp FFFF998846238080: '\usr\share\man\man3'
Removing all notifications for directory: FFFF9988462507B8
Removing notifications for file
KDTARGET: Refreshing KD connection

*** Fatal System Error: 0x0000000a
                       (0x0000000000000008,0x0000000000000002,0x0000000000000000,0xFFFFF8036487F82A)

Break instruction exception - code 80000003 (first chance)
Connected to Windows 10 16299 x64 target at (Mon Jul 30 16:27:08.445 2018 (UTC + 2:00)), ptr64 TRUE

In other cases the vnode won't be recycled, therefore the section object pointer is (probably) NULL, which is caught by the if query...
For me the problem occurs when searching for "*" on my FreeBSD card...

We probably could just check for 0xdeadbeef, but I am actually looking for a cleaner solution...

hmm..looks like it is not that easy, just crashed the Filesystem Filter Manager 😒

When copying large files (last tested 367MB) to the zpool, the process suddenly stops in the middle and after that nothing happens anymore.

...
zfs_write: resid/n 1048576 : offset 134217728 (rl_len 1048576) blksz 18446670930792613376
zfs_write done remainder 0
dispatcher: enter: major 4: minor 0: IRP_MJ_WRITE: type 0x6
  fsDispatcher: enter: major 4: minor 0: IRP_MJ_WRITE fsDeviceObject
zfs_write: resid/n 1048576 : offset 135266304 (rl_len 1048576) blksz 18446735286582576640
zfs_write done remainder 0
dispatcher: enter: major 4: minor 0: IRP_MJ_WRITE: type 0x6
  fsDispatcher: enter: major 4: minor 0: IRP_MJ_WRITE fsDeviceObject
zfs_write: resid/n 1048576 : offset 136314880 (rl_len 1048576) blksz 18446735286582576640

(that's it)

as I wasn't able to abort the copy-process, I tried to restart the PC, but it seems to hang in flush_buffers:

dispatcher: enter: major 9: minor 0: IRP_MJ_FLUSH_BUFFERS: type 0x6
  fsDispatcher: enter: major 9: minor 0: IRP_MJ_FLUSH_BUFFERS fsDeviceObject
flush_buffers: 

will add some printfs to get some more detailed info...

maybe that is something?
https://github.com/openzfsonwindows/ZFSin/blob/master/ZFSin/zfs/module/zfs/dmu.c#L1559

                 /*
		 * XXX uiomove could block forever (eg.nfs-backed
		 * pages).  There needs to be a uiolockdown() function
		 * to lock the pages in memory, so that uiomove won't
		 * block.
		 */
		err = uiomove((char *)db->db_data + bufoff, tocpy,
		    UIO_WRITE, uio);

here is the same code, but it has been commented out ...
https://github.com/openzfsonwindows/ZFSin/blob/master/ZFSin/zfs/module/zfs/dmu.c#L1788