Once we've implemented cryptomator/cryptofs#38, we are theoretically able to support symlinks in Cryptomator. Of course we also need a file system frontend that can leverage this possibility.

Thats why we should implement readlink and symlink from FuseOperations. :-)

While profiling for issue #33 I noticed, that getattr of directories (which happens frequently, e.g. when the caller wants to know if a file exists or is a directory) takes a lot of time due to nlinks depending on the number of subdirectories:

The run time of countSubDirs vastly depends on the number of items in a directory, which makes getattr of the directory dependend on its size, which can lead to significant slowdowns:

Other FUSE file systems don't support link count either, such as sshfs, which simply hard-coded st_nlink=1.

Citing Han-Wen Nienhuys:

The st_nlink for dirs is actualy not well defined. But the convention under unix is that it is the number of subdirectories including "." and "..".

[...]

But I think st_nlink == 1 is save in that regard and find will see that st_nlink doesn't follow the unix convention. Any other st_nlink will need "find -noleaf".

While we should try to set st_nlink to the number of subdirectories + 2, it should be ok to hard-code a fixed value in case we can't find an efficient solution to get the number of subdirectories.

I came here after this message but I don't see any examples or how to use this repo

Due to cryptomator/cryptofs#77, we need to pass through "file name too long" errors thrown by the underlying file system.

Example thrown by underlying ecryptfs on Linux:

java.nio.file.FileSystemException: path/of/long/file.txt: File name too long
    at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
    at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)

Besides operations creating nodes (such as mkdir or symlink) we noticed that error code ENAMETOOLONG is also expected to be returned by getattr.

Calling umount on an already unmounted volume fails and the error cascades up and breaks vault locking, even though an already unmounted volume is an acceptable error. See cryptomator/cryptomator#1192 for full details in context–I'm filing this issue with just the context that is relevant to fuse-nio-adapter.

I had some energy so I hunted down this bug. I'm not set up for Java dev right now so I'm limited in my ability to submit a PR, and debugging java codebases with github's web interface is a huge pain, but the issue seems to ultimately come from how you are incorrectly checking whether the volume is already mounted or not.

AbstractMount does try to check first whether or not the Fuse volume is mounted, calling the isMounted interface of FuseNioAdapter:

Which just gets an AtomicBoolean that gets managed internally:

That means that when a volume is unmounted outside of these java methods (which can easily happen for many many different and entirely valid reasons), the mounted flag doesn't have its state updated to match, and isMounted() doesn't check the actual reality of whether the volume is mounted or not. When Cryptomator is trying to unmount a volume, it does correctly try to check to avoid calling umount on an already unmounted volume, but that check as implemented can return a value inconsistent with the actual mounted state of the volume. In this case, the check will return true, and cryptomator will proceed to try to umount a volume that isn't mounted, resulting in an error.

It seems like one way to frame and approach this is as not handling the umount error specifically enough (because "not currently mounted" actually implies success for this particular context, and the rest of vault locking should be able to proceed as normal). The unmount syscall is meant to return EINVAL for this failure, unfortunately on OS X neither umount nor diskutil unmount seem to return useful error codes, so you'd be left parsing the error message:

% umount -f /Volume/fake; echo $?
umount: /Volume/fake: not currently mounted
1
% umount /Volume/fake; echo $?
umount: /Volume/fake: not currently mounted
1

Another way to frame it is that you need to check the actual mounted status before trying to call umount. On OS X I don't know of a better way than parsing the output of mount. You could try something as simple as mount | /usr/bin/awk '$3 == "/Volumes/path/to/volume" {print $3}' (or do the parsing in Java, w/e). Either way is going to be platform specific so it seems like this checking behavior probably belongs in the Mount interface.

Minimum version of FUSE for macOS should be checked, see cryptomator/cryptomator#657.

High level issue is described in cryptomator/cryptomator#673
For the low level causing see cryptomator/cryptofs#29.

We should handle IllegalArgumentExceptions in the methods collecting file meta data, such that valid cipher text files are shown and a message about the not shown/invalid cipher text files is logged.

Basic Info

OS: Windows 10
fuse-nio-adapter: commit 426439b

Description

Mounting a directory after another mount operation was executed fails. (No matter if the latter directory is still mounted or already unmounted.)
On linux with the native fuse-layer it works.
Also noteworthy is the fact, that with two independent processes it works but with two threads it fails.

Attachments

Enter path to the directory you want to mirror:
Y:\test
Enter drive letter to use:
J
The service java has been started.
Mounted successfully. Enter anything to stop the server...
quit
Unmounted successfully. Exiting...
The service java has been stopped.
Mount again?(y/n)
y
Enter path to the directory you want to mirror:
Y:\tresor
Enter drive letter to use:
K
The service java has failed to run (Status=c000010e).
ru.serce.jnrfuse.FuseException: Unable to mount FS
	at ru.serce.jnrfuse.AbstractFuseFS.mount(AbstractFuseFS.java:286)
	at org.cryptomator.frontend.fuse.WindowsMirroringTest.main(WindowsMirroringTest.java:33)
Caused by: ru.serce.jnrfuse.FuseException: Unable to mount FS, return code = 1
	at ru.serce.jnrfuse.AbstractFuseFS.mount(AbstractFuseFS.java:282)
	... 1 more

In order for Apple's notarization to work, we need to update to jffi 1.2.23 (due to jnr/jffi#74), which is a transitive dependency of jnr-fuse. jnr-fuse 0.5.4 updates this dependency (see SerCeMan/jnr-fuse#94).

Not really an issue, but I don't seem to find any clue. How do I mount a custom filesystem?
I'm trying this:

public static void main(String[] args) throws CommandFailedException {
        Mounter mounter = FuseMountFactory.getMounter();
        mounter.mount(Paths.get(URI.create("pdinfs:mem:/")), EnvironmentVariables.create().withMountPoint(Paths.get("/tmp/mnt")).build());
    }

and I'm getting:

Exception in thread "main" org.cryptomator.frontend.fuse.mount.CommandFailedException: java.lang.NullPointerException
	at org.cryptomator.frontend.fuse.mount.MacMounter.mount(MacMounter.java:42)
	at pt.ipb.pdinfs.Main.main(Main.java:14)
Caused by: java.lang.NullPointerException
	at ru.serce.jnrfuse.AbstractFuseFS.mount(AbstractFuseFS.java:252)
	at org.cryptomator.frontend.fuse.mount.MacMounter.mount(MacMounter.java:40)
	... 1 more

What am I doing wrong?

As an attempt to fix cryptomator/cryptomator#770, we should set f_namemax in the struct returned by statfs.

Whether this fix works can be verified using a mirrored directory by invoking getconf _POSIX_NAME_MAX /some/path on the shell.

Up until version 1.1.3 we had the option to append additional fuse flags via the additionalMountParams parameter in the mount method:

However, there is no option to remove or redefine those flags that have been predefined in places like this:

Therefore we should redefine the API to include the complete set of flags in the EnvironmentVariables passed to the mount function.

The default flags may server as an example or may be exposed via a public method, but should no longer be used internally.

This will be a breaking API change for the mounter.

Basically the same as #27 but we need a different solution. The issue is that the mount option -omodules=iconv,from_code=UTF-8,to_code=UTF-8-MAC causes issues with Emojis: cryptomator/cryptomator#961

Referencing another issue that boils the issue down to a buggy(?) iconv: osxfuse/osxfuse#706 (comment)

Proposed solution:

We resolved this issue by implementing the NFD to NFC conversion in our user-space code instead of using the built-in iconv module.

Implementing the NFC/NFD conversion in Java instead of instructing it via a mount option.

Since we're relying on a fix (SerCeMan/jnr-fuse#73).

Basic Info

OS: independent
fuse-nio-adapter: commit 943c430

Description

If you want to rename a file into an already existing one, it throws an error (File already exist)

Suggestion

Change this behaviour to just overwriting the existing file.

Background info

Some programs (e.g. gedit) need this to save an edited file. At first they create a temporary file, read the content of the original file into the tmp-file and in the save process they just rename the tmp-file into the original one.

Due to a hotfix (366af19), we have deactivated the version check. The version check was implemented because of #18 in the first place.

On MacOS we are using the fuse api provided by the https://github.com/osxfuse/osxfuse project.

A mount option of this library is negative_vncache, which lets the kernel save not existing files upon their first LOOKUP call and does not bother the fuse-filesystem with calls to these files.

Since we are using the mount-class specifically for https://github.com/cryptomator/cryptomator and due to its architecture we could use this option to increase performance, assuming that files can only be added when a vault is unlocked

Description

Two unit tests for resource locking contain a race condition. Both reside inside LockManagerTest. The bug was encoutered during CI runs ( see https://travis-ci.org/github/cryptomator/fuse-nio-adapter/builds/735702459).

One of the faulty tests is

It is unclear what the cause for the bug is, but it needs to be a race condition and not a timeout:

[ERROR] Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.482 s <<< FAILURE! - in org.cryptomator.frontend.fuse.locks.LockManagerTest
[ERROR] testMultipleReadLocks  Time elapsed: 0.067 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <8> but was: <7>
	at org.cryptomator.frontend.fuse.locks.LockManagerTest$DataLockTests.testMultipleReadLocks(LockManagerTest.java:233)

We call force() when a flush() is requested, because the Java NIO API doesn't have any equivalent to flush():

As clearly stated in the docs, flush IS NOT force. As there is nothing like flush() in the NIO API, we should drop it and implement fsync() instead.

This issue documents the decision made in project https://github.com/orgs/cryptomator/projects/1 in the fuse-nio-adapter repository.

The to be implemented getMountPoint() method should return the mount point of the mounted filesystem as a java.nio.Path object.

As macOS (by default) uses NFD for filenames, there are a few minor problems in Finder regarding filenames encoded differently. Therefore we should make sure to convert filenames to NFD during directory listings and back to NFC when accessing files.

See details in the macfuse wiki.

The class DaggerFuseNioAdapterComponent can not be found at the class AdapterFactory and FuseMountFactory.

We should internally use a ReadWriteLock to sync access per path (potentially including child paths). Until then we will use the -s mount option.

On Macs we can easily create such race conditions by copying a (nonempty) directory onto the mounted volume when not using -onoappledouble.

(not to be confused with #13)

Steps to Reproduce

1. Remove -onoappledouble mount option in MacMounter.
2. Mount with MirroringFuseMountTest.
3. Copy folder that includes a file to FUSE volume.

Expected Results

A successful copy.

Actual Results

Finder shows a warning with The operation can’t be completed because you don’t have permission to access “foo.jpg”. during the copy. The folder is created but the file is missing.

Notes

This is not 100% reproducible. However, the issue occurs quite often.

It actually appears less often when -onoappledouble mount option is set. That's why it's being removed to increase the probability.

We already did a lot of experiments. Adding/removing mount options, changing/removing methods in ReadOnlyAdapter and ReadWriteAdapter etc. and some of them were particularly interesting:

• When adding the debug option -d, I was unable to reproduce the issue. I thought it was because of the foreground mode, however the -f option is actually even worse.
• When removing the utimens method in ReadWriteAdapter, I was unable to reproduce the issue.

In my opinion, these findings are just indications and are not necessarily the cause of the issue.

Console

Even though there is no stack trace, the console log of Finder may or may not indicate the issue.

Sandbox: Finder(18823) System Policy: deny(1) file-read-metadata /Users/<redacted>/foo
Violation:       System Policy: deny(1) file-read-metadata /Users/<redacted>/foo 
Process:         Finder [18823]
Path:            /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
Load Address:    0x10fa47000
Identifier:      com.apple.finder
Version:         10.13.5 (10.13.5)
Build Info:      20-Finder_FE~1054005004000000
Code Type:       x86_64 (Native)
Parent Process:  launchd [1]
Responsible:     /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder [18823]
User ID:         501

Date/Time:       2018-09-13 16:30:53.746 GMT+2
OS Version:      Mac OS X 10.13.6 (17G65)
Report Version:  8


MetaData: {"hardware":"Mac","path":"\/Users\/<redacted>\/foo","platform-binary":true,"profile":"platform","flags":21,"errno":1,"summary":"deny(1) file-read-metadata \/Users\/<redacted>\/foo","primary-filter":"path","uid":501,"pid":18823,"profile-flags":0,"vnode-type":"DIRECTORY","process":"Finder","platform_binary":"yes","normalized_target":["Users","<redacted>","foo"],"primary-filter-value":"\/Users\/<redacted>\/foo","target":"\/Users\/<redacted>\/foo","action":"deny","process-path":"\/System\/Library\/CoreServices\/Finder.app\/Contents\/MacOS\/Finder","rdev":10,"signing-id":"com.apple.finder","platform-policy":true,"build":"Mac OS X 10.13.6 (17G65)","rootless-flags":128,"operation":"file-read-metadata"}

Thread 0 (id: 21982624):
0   libsystem_kernel.dylib        	0x00007fff7dd1b20a mach_msg_trap + 10
1   CoreFoundation                	0x00007fff55d94785 __CFRunLoopServiceMachPort + 341
2   CoreFoundation                	0x00007fff55d93ad7 __CFRunLoopRun + 1783
3   CoreFoundation                	0x00007fff55d93153 CFRunLoopRunSpecific + 483
4   HIToolbox                     	0x00007fff5507dd96 RunCurrentEventLoopInMode + 286
5   HIToolbox                     	0x00007fff5507db06 ReceiveNextEventCommon + 613
6   HIToolbox                     	0x00007fff5507d884 _BlockUntilNextEventMatchingListInModeWithFilter + 64
7   AppKit                        	0x00007fff5332ea73 _DPSNextEvent + 2085
8   AppKit                        	0x00007fff53ac4e34 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 3044
9   AppKit                        	0x00007fff53323885 -[NSApplication run] + 764
10  AppKit                        	0x00007fff532f2a72 NSApplicationMain + 804
11  Finder                        	0x000000010fa4ca8e
12  libdyld.dylib                 	0x00007fff7dbd4015 start + 1
13  Finder                        	0x0000000000000001

Thread 1 (id: 21983080, com.apple.NSEventThread):
0   libsystem_kernel.dylib        	0x00007fff7dd1b20a mach_msg_trap + 10
1   CoreFoundation                	0x00007fff55d94785 __CFRunLoopServiceMachPort + 341
2   CoreFoundation                	0x00007fff55d93ad7 __CFRunLoopRun + 1783
3   CoreFoundation                	0x00007fff55d93153 CFRunLoopRunSpecific + 483
4   AppKit                        	0x00007fff5346bfc4 _NSEventThread + 184
5   libsystem_pthread.dylib       	0x00007fff7deec661 _pthread_body + 340
6   libsystem_pthread.dylib       	0x00007fff7deec50d _pthread_body + 0
7   libsystem_pthread.dylib       	0x00007fff7deebbf9 thread_start + 13

Thread 2 (id: 21983144):
0   libsystem_kernel.dylib        	0x00007fff7dd1b20a mach_msg_trap + 10
1   CoreFoundation                	0x00007fff55d94785 __CFRunLoopServiceMachPort + 341
2   CoreFoundation                	0x00007fff55d93ad7 __CFRunLoopRun + 1783
3   CoreFoundation                	0x00007fff55d93153 CFRunLoopRunSpecific + 483
4   CoreFoundation                	0x00007fff55dd1be3 CFRunLoopRun + 99
5   DiscRecording                 	0x00007fff57bcb723 DRWorkLoop::WorkLoop() + 237
6   DiscRecording                 	0x00007fff57bcb623 DRWorkLoop::WorkLoopEntry(DRWorkLoop*) + 9
7   DiscRecording                 	0x00007fff57bcb30b DRThreadObject::StartRoutine(DRThreadObject*) + 111
8   DiscRecording                 	0x00007fff57bcb1a5 DRThreadObject::SymbolRoutine(DRThreadObject*) + 9
9   libsystem_pthread.dylib       	0x00007fff7deec661 _pthread_body + 340
10  libsystem_pthread.dylib       	0x00007fff7deec50d _pthread_body + 0
11  libsystem_pthread.dylib       	0x00007fff7deebbf9 thread_start + 13

Thread 3 (id: 22055853):
0   libsystem_kernel.dylib        	0x00007fff7dd24a16 __psynch_cvwait + 10
1   libc++.1.dylib                	0x00007fff7bb28cb0 std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) + 18
2   JavaScriptCore                	0x00007fff59a098a6 void std::__1::condition_variable_any::wait<std::__1::unique_lock<bmalloc::Mutex> >(std::__1::unique_lock<bmalloc::Mutex>&) + 86
3   JavaScriptCore                	0x00007fff59a0955b bmalloc::Scavenger::threadRunLoop() + 251
4   JavaScriptCore                	0x00007fff59a09359 bmalloc::Scavenger::threadEntryPoint(bmalloc::Scavenger*) + 9
5   JavaScriptCore                	0x00007fff59a09698 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (*)(bmalloc::Scavenger*), bmalloc::Scavenger*> >(void*) + 40
6   libsystem_pthread.dylib       	0x00007fff7deec661 _pthread_body + 340
7   libsystem_pthread.dylib       	0x00007fff7deec50d _pthread_body + 0
8   libsystem_pthread.dylib       	0x00007fff7deebbf9 thread_start + 13

Thread 4 (id: 22055859, com.apple.coreanimation.render-server):
0   libsystem_kernel.dylib        	0x00007fff7dd1b20a mach_msg_trap + 10

Thread 5 (id: 22055908, com.apple.CFNetwork.CustomProtocols):
0   libsystem_kernel.dylib        	0x00007fff7dd1b20a mach_msg_trap + 10
1   CoreFoundation                	0x00007fff55d94785 __CFRunLoopServiceMachPort + 341
2   CoreFoundation                	0x00007fff55d93ad7 __CFRunLoopRun + 1783
3   CoreFoundation                	0x00007fff55d93153 CFRunLoopRunSpecific + 483
4   CFNetwork                     	0x00007fff54edde84 -[__CoreSchedulingSetRunnable runForever] + 722
5   Foundation                    	0x00007fff57e9e1f8 __NSThread__start__ + 1197
6   libsystem_pthread.dylib       	0x00007fff7deec661 _pthread_body + 340
7   libsystem_pthread.dylib       	0x00007fff7deec50d _pthread_body + 0
8   libsystem_pthread.dylib       	0x00007fff7deebbf9 thread_start + 13

Thread 6 (id: 22646489):
0   libsystem_kernel.dylib        	0x00007fff7dd1b246 semaphore_wait_trap + 10
1   libdispatch.dylib             	0x00007fff7dba2979 _dispatch_semaphore_wait_slow + 101
2   DesktopServicesPriv           	0x00007fff6a25775e TNodeEvent::Notify(TRef<TNodeEvent*, TInternalRefCountPolicy<TNodeEvent> > const&, bool, bool) + 334
3   DesktopServicesPriv           	0x00007fff6a23ea07 TOperation::ReportErrorFromUnprotectedContext(TRef<TOperationErrorRecord*, TInternalRefCountPolicy<TOperationErrorRecord> > const&) + 245
4   DesktopServicesPriv           	0x00007fff6a2c421e TDSHelperOperation::HandleMsgFromHelper(char const*, NSObject<OS_xpc_object>*) + 1586
5   DesktopServicesPriv           	0x00007fff6a2c58d5 -[TDSHelperConnectionHandler handleHelperEvent:] + 58
6   libxpc.dylib                  	0x00007fff7df29773 _xpc_connection_call_event_handler + 35
7   libxpc.dylib                  	0x00007fff7df27f31 _xpc_connection_mach_event + 1092
8   libdispatch.dylib             	0x00007fff7db9fda9 _dispatch_client_callout4 + 9
9   libdispatch.dylib             	0x00007fff7dba0062 _dispatch_mach_msg_invoke + 488
10  libdispatch.dylib             	0x00007fff7dbaf07a _dispatch_queue_serial_drain + 222
11  libdispatch.dylib             	0x00007fff7db9e972 _dispatch_mach_invoke + 534
12  libdispatch.dylib             	0x00007fff7dbaff0d _dispatch_root_queue_drain_deferred_wlh + 332
13  libdispatch.dylib             	0x00007fff7dbb3d21 _dispatch_workloop_worker_thread + 880
14  libsystem_pthread.dylib       	0x00007fff7deebfd2 _pthread_wqthread + 980
15  libsystem_pthread.dylib       	0x00007fff7deebbe9 start_wqthread + 13
16  Finder                        	0x0000000000000a00

Thread 7 (id: 22648959):
0   libsystem_kernel.dylib        	0x00007fff7dd2528a __workq_kernreturn + 10
1   libsystem_pthread.dylib       	0x00007fff7deebbe9 start_wqthread + 13
2                                 	0x0000000300010000

Thread 8 (id: 22649720):
0   libsystem_kernel.dylib        	0x00007fff7dd2528a __workq_kernreturn + 10
1   libsystem_pthread.dylib       	0x00007fff7deebbe9 start_wqthread + 13
2                                 	0x0010f97c00004000

Thread 9 (id: 22649821):
0   libsystem_kernel.dylib        	0x00007fff7dd258ea getattrlistbulk + 10
1   CoreServicesInternal          	0x00007fff698194c5 _GetDirectoryURLs(_CFURLEnumerator*) + 1508
2   CoreServicesInternal          	0x00007fff69816e50 _URLEnumeratorGetNextURL + 171
3   DesktopServicesPriv           	0x00007fff6a2b4543 TFSIterator::NextRaw(TRef<TFSInfo*, TInternalRefCountPolicy<TFSInfo> >&) + 41
4   DesktopServicesPriv           	0x00007fff6a2a8e8b TFSIterator::Next(TRef<TFSInfo*, TInternalRefCountPolicy<TFSInfo> >&) + 31
5   DesktopServicesPriv           	0x00007fff6a25f99b TFSInfoSynchronizer::FetchChildren(bool, bool, LSProperties) + 4413
6   DesktopServicesPriv           	0x00007fff6a22a267 TNode::SynchronizeChildren(NodeRequestOptions, TNodeEventPtrs&) + 747
7   DesktopServicesPriv           	0x00007fff6a225041 TNode::HandleSync(NodeRequestOptions, TNodeEventPtrs&) + 629
8   DesktopServicesPriv           	0x00007fff6a218e34 TNode::HandleSync(NodeRequestOptions) + 32
9   DesktopServicesPriv           	0x00007fff6a235ac7 TNode::HandleSync(TRef<TNodeTask*, TInternalRefCountPolicy<TNodeTask> > const&, TNodePtr const&) + 317
10  DesktopServicesPriv           	0x00007fff6a234489 TNode::HandleNodeRequest(TRef<TNodeTask*, TInternalRefCountPolicy<TNodeTask> > const&) + 209
11  DesktopServicesPriv           	0x00007fff6a1e4ce6 invocation function for block in TVolumeSyncThread::PostNodeTaskRequest(TNodeTask*) + 67
12  DesktopServicesPriv           	0x00007fff6a1e4c4c ExceptionSafeBlock(void () block_pointer) + 9
13  DesktopServicesPriv           	0x00007fff6a1e4c3d invocation function for block in TVolumeSyncThread::PostNodeTaskRequest(TNodeTask*) + 88
14  libdispatch.dylib             	0x00007fff7dba25fa _dispatch_call_block_and_release + 12
15  libdispatch.dylib             	0x00007fff7db9adb8 _dispatch_client_callout + 8
16  libdispatch.dylib             	0x00007fff7dbaf217 _dispatch_queue_serial_drain + 635
17  libdispatch.dylib             	0x00007fff7dba2166 _dispatch_queue_invoke + 373
18  libdispatch.dylib             	0x00007fff7dbaff0d _dispatch_root_queue_drain_deferred_wlh + 332
19  libdispatch.dylib             	0x00007fff7dbb3d21 _dispatch_workloop_worker_thread + 880
20  libsystem_pthread.dylib       	0x00007fff7deebfd2 _pthread_wqthread + 980
21  libsystem_pthread.dylib       	0x00007fff7deebbe9 start_wqthread + 13
22                                	0x4058504320000000

Thread 10 (id: 22650462):
0   libsystem_kernel.dylib        	0x00007fff7dd2528a __workq_kernreturn + 10
1   libsystem_pthread.dylib       	0x00007fff7deebbe9 start_wqthread + 13
2                                 	0x0000120000000030

Thread 11 (id: 22650663):
0   libsystem_kernel.dylib        	0x00007fff7dd24a16 __psynch_cvwait + 10
1   DesktopServicesPriv           	0x00007fff6a2c5237 TDSHelperContext::SendMessageAndWait(NSObject<OS_xpc_object>*) + 73
2   DesktopServicesPriv           	0x00007fff6a2490a9 invocation function for block in TDesktopServicesHelperCopyMoveOperation::PerformCopyOrForceMove(TRef<TOperationRecord*, TInternalRefCountPolicy<TOperationRecord> > const&, bool) + 36
3   Foundation                    	0x00007fff57f336e5 -[NSFileCoordinator _invokeAccessor:thenCompletionHandler:] + 130
4   Foundation                    	0x00007fff57f76c47 __73-[NSFileCoordinator coordinateWritingItemAtURL:options:error:byAccessor:]_block_invoke + 96
5   Foundation                    	0x00007fff580f371a __85-[NSFileCoordinator(NSPrivate) _coordinateWritingItemAtURL:options:error:byAccessor:]_block_invoke.422 + 224
6   Foundation                    	0x00007fff580f3167 -[NSFileCoordinator(NSPrivate) _withAccessArbiter:invokeAccessor:orDont:andRelinquishAccessClaim:] + 466
7   Foundation                    	0x00007fff57f76a00 -[NSFileCoordinator(NSPrivate) _coordinateWritingItemAtURL:options:error:byAccessor:] + 810
8   Foundation                    	0x00007fff57f766d0 -[NSFileCoordinator coordinateWritingItemAtURL:options:error:byAccessor:] + 82
9   DesktopServicesPriv           	0x00007fff6a24828e TDesktopServicesHelperCopyMoveOperation::PerformCopyOrForceMove(TRef<TOperationRecord*, TInternalRefCountPolicy<TOperationRecord> > const&, bool) + 1688
10  DesktopServicesPriv           	0x00007fff6a2471df TDesktopServicesHelperCopyMoveOperation::PerformOperation(TRef<TOperationRecord*, TInternalRefCountPolicy<TOperationRecord> > const&) + 861
11  DesktopServicesPriv           	0x00007fff6a23e1cc TOperation::ProcessSelection() + 186
12  DesktopServicesPriv           	0x00007fff6a23d9ab TOperation::Perform() + 169
13  DesktopServicesPriv           	0x00007fff6a25c7bd TNodeOperationTask::HandleOperation(TRef<TOperation*, TInternalRefCountPolicy<TOperation> > const&) + 573
14  DesktopServicesPriv           	0x00007fff6a25c95c TNodeOperationTask::OperationTaskProcedure() + 68
15  DesktopServicesPriv           	0x00007fff6a25c1f6 TNodeOperationTask::OperationTaskProc(void*) + 10
16  libsystem_pthread.dylib       	0x00007fff7deec661 _pthread_body + 340
17  libsystem_pthread.dylib       	0x00007fff7deec50d _pthread_body + 0
18  libsystem_pthread.dylib       	0x00007fff7deebbf9 thread_start + 13

Binary Images:
       0x10fa47000 -        0x110100ff7  com.apple.finder (10.13.5 - 10.13.5) <3a746b08-e656-3128-b33a-2e61ef388c7b> /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder
    0x7fff532ed000 -     0x7fff5414bfff  com.apple.AppKit (6.9 - 1561.60.100) <3c27cf6f-e640-3411-a87d-ccb2222cc754> /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
    0x7fff54c9a000 -     0x7fff55034ff7  com.apple.CFNetwork (902.1 - 902.1) <76eb8cb6-bf59-3bda-bf2b-f21b161611b9> /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
    0x7fff5504e000 -     0x7fff55353fff  com.apple.HIToolbox (2.1.1) <bf7f9c0e-c732-3fb2-9bbc-362888bda57b> /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox
    0x7fff55d0e000 -     0x7fff561affef  com.apple.CoreFoundation (6.9 - 1454.90) <e5d594bf-9142-3325-a62d-cf4aaf472642> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff57bc8000 -     0x7fff57c82fff  com.apple.DiscRecording (9.0.3 - 9030.4.5) <35d1ff9a-432c-39ad-9c1f-9022246ec5c0> /System/Library/Frameworks/DiscRecording.framework/Versions/A/DiscRecording
    0x7fff57e6f000 -     0x7fff58235fff  com.apple.Foundation (6.9 - 1454.90) <8ea924f3-adae-3f4b-8482-8b11c027d9a5> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
    0x7fff58de2000 -     0x7fff59ac8fff  com.apple.JavaScriptCore (13605 - 13605.3.8) <e6b49cda-72f0-3e58-85ef-98af10ce04b0> /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore
    0x7fff6980b000 -     0x7fff6983cff3  com.apple.CoreServicesInternal (309.1 - 309.1) <4ecd14ea-a493-3b84-a32f-cf928474a405> /System/Library/PrivateFrameworks/CoreServicesInternal.framework/Versions/A/CoreServicesInternal
    0x7fff6a1d9000 -     0x7fff6a308fff  com.apple.desktopservices (1.12.5 - 1.12.5) <7739c9a5-64d9-31a5-899b-5ffa242ad70d> /System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv
    0x7fff7bb22000 -     0x7fff7bb78fff  libc++.1.dylib (400.9) <7d3daccc-3804-393c-abc1-1a580fd00cb6> /usr/lib/libc++.1.dylib
    0x7fff7db99000 -     0x7fff7dbd2ff7  libdispatch.dylib (913.60.2) <232c69bd-022e-3ab9-8807-79f9fa7cb5ec> /usr/lib/system/libdispatch.dylib
    0x7fff7dbd3000 -     0x7fff7dbf0ff7  libdyld.dylib (551.4) <81bf3a82-5719-3b54-aba9-76c82d932cac> /usr/lib/system/libdyld.dylib
    0x7fff7dd08000 -     0x7fff7dd2eff7  libsystem_kernel.dylib (4570.71.2) <f22b8d73-69d8-36d7-bf66-7f9ac70c08c2> /usr/lib/system/libsystem_kernel.dylib
    0x7fff7dee9000 -     0x7fff7def4fff  libsystem_pthread.dylib (301.50.1) <0e51ccba-91f2-34e1-bf2a-feefd3d321e4> /usr/lib/system/libsystem_pthread.dylib
    0x7fff7df1e000 -     0x7fff7df4bff7  libxpc.dylib (1205.70.9) <0bc7ad67-671d-31d4-8b88-c317b8379598> /usr/lib/system/libxpc.dylib

Environment

• Operating System: macOS High Sierra 10.13.6

We need to upgrade Guava to 25.0 in order to fix CVE-2018-10237.

OS: MacOS High Sierra 10.13.5
Adapter Version: 1afbe5d

Under MacOs currently the used filesystem is mounted with the option -noappledouble. If the filesystem contains already AppleDouble files, they cannot be deleted and can prevent directories from being deleted.

Correctly show:

• File modification date
• File creation date
• File access date (if available)

Delegate user/group ownership, if the underlying file supports it (must be fail-save, as apparently Windows doesn't have such attributes).

From the WinFSP 2017.2 debug logs we can see that close is called:

javaw[TID=0f80]: FFFFDB82FA6F0D90: >>Create [UT---C] "\", FILE_OPEN, CreateOptions=200000, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=000000000000054C[PID=1364], DesiredAccess=80, GrantedAccess=0, ShareAccess=7
javaw[TID=0f80]: FFFFDB82FA6F0D90: <<Create IoStatus=0[1] UserContext=0000000000000000:0000000002933290, GrantedAccess=80, FileInfo={FileAttributes=10, ReparseTag=0, AllocationSize=0:1000, FileSize=0:1000, CreationTime=2017-12-07T09:10:54.326Z, LastAccessTime=2017-12-13T14:40:48.932Z, LastWriteTime=2017-12-13T14:40:48.932Z, ChangeTime=1970-01-01T00:00:00.000Z, IndexNumber=0:0}
javaw[TID=0f80]: FFFFDB82F9A53010: >>Close 0000000000000000:0000000002933290
javaw[TID=0f80]: FFFFDB82F9A53010: <<Close IoStatus=0[0]

However our release function is not called (debugger doesn't stop at breakpoint).

@SerCeMan @billziss-gh Does one of you know if:

1. close/release is skipped under certain conditions?
2. some other functions such as unlink expect release to be called implicitly? (no such thing is documented in the fuse docs, though)

As suggested in this thread, there might be room for improvements in the locking scheme implemented in #15:

Since the modification of the directory resource happens atomically when actually adding/removing a child, it might be worth trying to only get a shared read lock on a directory when adding or removing children. This is more of a eventual consistency model. I.e. a concurrent directory listing will be undefined.

When creating, deleting or moving a directory, we still need an exclusive write lock on that directory. During that time no read lock can be obtained, thus modifications to ancestors will not be possible.

Todo

• profile application while adding a single large directory (deeply nested)
• decide whether lock contention can be reduced on directory locks
• (optionally) only "read-lock" directories when adding or removing children

Currently the OpenFlags O_RDONLY, O_WRONLY and O_RDWR are not handled correctly:

According to https://linux.die.net/man/3/open those three flags MUST be handled mutually exclusive.

Since O_RDONLY has the value 0 in the bitmask, it falsely is always present, which leads to errors (such as cryptomator/cryptomator#753).

In version 1.2.5 the reveal command for in the AbstractMount class was changed from java.awt.Desktop.open() to ...browse(), which is not supported on all systems. (e.g. Linux system with KDE):

In tests the old open() command still does the job and hence should be used again.

Basically, I am wondering what is needed to be done to consider this project an MVP ready for production use, i.e. integration with Cryptomator and release.

Understanding of your roadmap would make it easier for me to organize my own.

Basic Info

OS: Fedora 27
fuse-nio-adapter: commit 2073845

Description

Accessing a file on Linux results in a I/O-error and throws a java.io.IOException, even with the ReadOnlyAdapter
howto:

1. start adapter (e.g. mount ~\test at ~\mnt)
2. place a textfile into ~\test (e.g. asd.txt)
3. switch with console into ~\mnt
4. try to output some content of asd.txt , e.g. with cat

Attachments (optional)

LOOKUP /fuse-nio-adapter.log
getattr /fuse-nio-adapter.log
   NODEID: 7
   unique: 352, success, outsize: 144
unique: 353, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 4458
   unique: 353, success, outsize: 32
unique: 354, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 4458
getattr /
   unique: 354, success, outsize: 120
unique: 355, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 4458
readdir[0] from 0
   unique: 355, success, outsize: 440
unique: 356, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 4458
   unique: 356, success, outsize: 16
unique: 357, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 357, success, outsize: 16
unique: 358, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3125
getattr /
   unique: 358, success, outsize: 120
unique: 359, opcode: OPEN (14), nodeid: 7, insize: 48, pid: 4459
open flags: 0x8000 /fuse-nio-adapter.log
[Thread-220] ERROR org.cryptomator.frontend.fuse.ReadOnlyFileHandler - Error opening file.
java.io.IOException: Unsupported open flags: __UNKNOWN_CONSTANT__
	at org.cryptomator.frontend.fuse.ReadOnlyFileHandler.open(ReadOnlyFileHandler.java:56)
	at org.cryptomator.frontend.fuse.ReadOnlyFileHandler.open(ReadOnlyFileHandler.java:36)
	at org.cryptomator.frontend.fuse.ReadOnlyAdapter.open(ReadOnlyAdapter.java:126)
	at ru.serce.jnrfuse.AbstractFuseFS.lambda$init$2(AbstractFuseFS.java:131)
	at jnr.ffi.provider.jffi.NativeClosureProxy$$impl$$1.invoke(Unknown Source)
   unique: 359, error: -5 (Eingabe-/Ausgabefehler), outsize: 16

Basic Info

OS: Windows 10
branch: feature/cryptofsIntegration
commit: 426439b

Description

Some applications have problems open files in the vault. They must be copied from the vault to a place outside the vault to open them in the specific program. Currently it is unclear, if it depends on the application or not. It does not depend on the file-type.

Steps to Reproduce:
1 Have an exisiting vault with for example a pdf-file in it.
1 Open the "open-with"-dialogue and select microsoft edge

Expected Result: The file is shown in the program
Actual Result: An empty file and/or an error message is shown

##Examples:
.pdf-files and Microsoft Edge
.txt-files or .odt-files and Wordpad

Does performance benefit from eager getattr? Maybe we can at least return file stats that are easy to determine. E.g. instead of using Files.list() we might better be using walkFileTree with zero depth, which gives us basicfileattributes for each file.

Some logging has shown me that after I return the list of file names, getattr is then called once for each file [...]
-- Source

See: cryptomator/cryptomator#900

Improve #23 by no longer hard coding it but also allowing to define a custom value during file system construction.

At the moment the filesystem gets closed even if unmount failed:

This is incorrect and needs to be improved.

jnr-fuse doesn't properly stat files when running on arm: SerCeMan/jnr-fuse#14

This means that several things that rely on stat (e.g. basically anything that checks filesizes) breaks.

A possibility might be to replace the library with https://github.com/EtiennePerot/fuse-jna

Basic Info

OS: Windows 10
branch: feature/cryptofsIntegration
commit: 426439b

Description

Deleting a directory with a lot of small files in it takes a long time to finish. My test directory contains 432 files, every file has an average file size of 17 KB. It takes nearly 4 minutes to delete the directory. Under Linux the deletion is executed within seconds.

If a file channel is still open when one executes the unmount method ( which in the end calls the linux helper command fusermount), the unmount fails with a CommandFailedException:

Nonetheless, with the call fuseAdapter.unmount() the actual filesystem is marked as not-mounted, make it impossible to recover from the failed regular unmount command:

Additionally, the CommandFailedException for the fusermount only means that the operation did not succeeded, but does not indicate for what reason.

I propose to change the regular unmount to _ only_ execute fusermount and marking the filesystem as unmounted, if no internal file channels are open and otherwise throw a different exception. If the filesystem need to be unmoutned, the unmountForced() method can be used.

Ideas how to implement can be taken from cryptomator/dokany-nio-adapter@9a65c64

I'm trying to implement something similar to Overlay FS, where I would need to save changes to files separately from the original files. It doesn't look like the Path API would allow that by itself. When a file is opened, I would need to read from the same file and write to a different one. Is there anything in this project that would allow me to intercept read/write requests?

Currently, file and directory permissions are hard-coded in getattr.
I think it would be better to return the real permissions from the cryptofs instead.

Basic Info

OS: Fedora 27
fuse-nio-adapter: commit 1c2cd0c

Description

On Linux a file accessed over the mountpoint has no content/is empty.
howto:

1. start adapter (e.g. mount ~\test at ~\mnt)
2. place a textfile into ~\test with content (e.g. $ echo Hello > asd.txt)
3. switch with console into ~\mnt
4. output content of asd.txt , e.g. with cat

Expected Result: Hello
Actual Result: (meaning nothing)

There is a NOFOLLOW_LINKS missing in getFileAttributeView():

This leads to errors when attempting to set the file times of symlinks (as described in cryptomator/cryptomator#853).

Before blindly fixing this, we need to find out how AT_SYMLINK_NOFOLLOW is to be handled correctly in FUSE.

Basic Info

OS: Windows 10, Ver. 1709, Build 16299.64
fuse-nio-adapter: commit 2073845

Description

Creating Files not via explorer results in a I/O-Error thrown by windows and an java.nio.file.AccessDeniedException thrown by the adapter (visible in errorlog)

howto:

1. start adapter (e.g. mount X:\test at J:)
2. open notepad.exe, write something into file
3. save file at J:

Attachments (optional)

java[TID=163c]: FFFFAC019A4702A0: >>Create [UT---C] "\asd.txt", FILE_OPEN_IF, CreateOptions=60, FileAttributes=0, Security=NULL, AllocationSize=0:0, AccessToken=00000000000008C8, DesiredAccess=12019f, GrantedAccess=0, ShareAccess=3
java[TID=20f0]: FFFFAC019A61CEA0: <<QueryDirectory IoStatus=c0000034[0]
java[TID=20f0]: FFFFAC019D25F280: >>Close 0000000000000000:000001FE3B68CBE0
java[TID=20f0]: FFFFAC019D25F280: <<Close IoStatus=0[0]
java[TID=20f0]: FFFFAC019D25F280: >>Create [UT---C] "\", FILE_OPEN, CreateOptions=21, FileAttributes=10, Security=NULL, AllocationSize=0:0, AccessToken=0000000000000934, DesiredAccess=100000, GrantedAccess=0, ShareAccess=3
[Thread-164] ERROR org.cryptomator.frontend.fuse.ReadWriteFileHandler - Error opening file.
java.nio.file.AccessDeniedException: Y:\test\asd.txt
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:116)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
	at org.cryptomator.frontend.fuse.OpenFile.<init>(OpenFile.java:27)
	at org.cryptomator.frontend.fuse.OpenFileFactory.open(OpenFileFactory.java:52)
	at org.cryptomator.frontend.fuse.ReadWriteFileHandler.createAndOpen(ReadWriteFileHandler.java:55)
	at org.cryptomator.frontend.fuse.ReadWriteAdapter.create(ReadWriteAdapter.java:67)
	at ru.serce.jnrfuse.AbstractFuseFS.lambda$init$14(AbstractFuseFS.java:194)
	at jnr.ffi.provider.jffi.NativeClosureProxy$$impl$$17.invoke(Unknown Source)
java[TID=163c]: FFFFAC019A4702A0: <<Create IoStatus=c0000185[0]
java[TID=20f0]: FFFFAC019D25F280: <<Create IoStatus=0[1] UserContext=0000000000000000:000001FE3B68D240, GrantedAccess=100000, FileInfo={FileAttributes=10, ReparseTag=0, AllocationSize=0:1000, FileSize=0:1000, CreationTime=2017-11-27T17:24:19.924Z, LastAccessTime=2017-12-13T20:45:24.231Z, LastWriteTime=2017-12-13T20:45:24.231Z, ChangeTime=1970-01-01T00:00:00.000Z, IndexNumber=0:0}
java[TID=20f0]: FFFFAC019B662EA0: >>Close 0000000000000000:000001FE3B68D240
java[TID=20f0]: FFFFAC019B662EA0: <<Close IoStatus=0[0]```

When there is no internet connection, the selection of FUSE as volume Type is not possible.

The Logs shows this error:
13:09:31.295 [JavaFX Application Thread] ERROR o.c.frontend.fuse.mount.MacMounter - Could not read /Library/Filesystems/osxfuse.fs/Contents/version.plist to detect version of OSXFUSE.

The error occurs when the MacMounter class tries to find the version of FUSE. When the method getVersionString parses the version.plist file, the DocumentBuilder tries to download the DTD file referenced in the verision.plist file. This ends in a Exception when there is no direct connection to the internet. This will also fail on a system with a automatic proxy discovery.

When the DTD Definition is remove from the version.plist file, the error doesn't occur.

Implement lock and potentially flock:

int lock(String path, FuseFileInfo fi, int cmd, Flock flock);
int flock(String path, FuseFileInfo fi, int op);

Once cryptomator/cryptofs#47 is fixed, we can support xattrs. Regarding error codes, we should stick to the osx manpage, which has a more detailed distinction between different errors.

Noteworthy: E2BIG and ENAMETOOLONG should be handled correctly, since the size might increase due to the encryption layer.