darkmattercore / libusbhsfs Goto Github PK

**Is your feature request related to a problem? Most platforms support EXFAT however exfat is not exactly robust. However an varient called texfat which provides a redundant FAT

Describe the solution you'd like
Support for TEXFAT

Describe alternatives you've considered
None exist

Additional context
NTFS is my current filesystem but its not as universal as exfat texfat is backwords compatable with exfat however this can result in the fats getting out of sync. So a routine to validate that the fat is in tact and to update the redundant fat would be nice

https://docs.microsoft.com/en-us/previous-versions/cc907927(v=msdn.10)?redirectedfrom=MSDN

Is your feature request related to a problem? Please describe.
Horizon OS treats directories with numbered files (e.g. 00, 01, etc.) *and* the archive bit flag set as a single file, which helps overcome the max file size limitation in FAT-based filesystems. This feature is known as ConcatenationFile.

In order to provide the user a hassle-free way to manage big files under FAT-based filesystems from UMS devices, it'd be desirable to implement a similar approach into the library.

Describe the solution you'd like
Implement a ConcatenationFile-like ABI that's capable of opening, reading and writing filesystem objects that follow the previously described layout.

Support for ConcatenationFile objects would be completely optional, would be exclusive to FAT-based filesystems, and would ideally be controlled by an additional mount flag. The feature itself would be enabled by default.

Details on how the actual implementation would be accomplished have not yet been planned. This is just something I'd like to do, because it's extremely useful for apps such as nxdumptool.

Additional context
Max file size for each part file would match the one used by Horizon OS.

1. Plug-in USB drive.
2. Run hbl
3. Run example

Example hangs and do not react on any buttons.
If plug out flash drive - then receive exit signal and example exits.

Example works only if first run it and only after that insert usb drive.

What about NX native FsFilesystem interface along with devoptab?

Is your feature request related to a problem? Please describe.
Back when I designed the interface for this library in 2020, it seemed like a no-brainer to use an event-based system based on the ABI used by multiple Switch background services, which in turn makes it possible for the application using the library to span a single additional background thread that uses waitMulti() to handle multiple events and call it a day.

However, this very same reasoning falls short if the application using the library doesn't use events at all, which is something I never stopped to consider. This forces the developer to span an additional background thread to exclusively handle changes from the library, which is a bit overkill.

Please consider this library was specifically designed with nxdumptool in mind before it became a much broader project, and that application relies heavily on background services that do use this kind of ABI.

Describe the solution you'd like
A callback-based system that uses a function pointer provided by the user to handle UMS status changes under the library's very own background thread.

This new system should be able to co-exist with the current event-based system, because both designs have their own particular use cases.

Additional context
Initially proposed by averne and discussed in #26.

Describe the bug
Hello, would you be able to help diagnose this bug?

After using a homebrew that makes use of libusbhsfs returning to that homebrew can result in a black screen or crash.

Please see rashevskyv/dbi#7 for full reference including atmosphere crash report. I was able to reproduce this with the libusbhsfs example nro

To Reproduce
Steps to reproduce the behavior:

1. Run libusbhsfs example
2. Exit
3. Return to open libusbhsfs example
4. Black screen in application mode, crash in applet mode

Please fill the following information:

• Horizon OS (Switch FW) version: 10.2.0 + 11.0.0 tested
• CFW: Atmosphere 0.16.0 prerelease 2
• Atmosphère launch method (if applicable): fusee-primary or hekate fss0
• libusbhsfs version: 0.0.3
• Homebrew launch method: application or applet
• USB Mass Storage device specs: Kingston datatraveler 64GB exFAT pendrive

Logfile
I'm not getting the logfile, would you be able to provide a build that produces this?

Describe the bug
We faced performance degression in XorTroll/Goldleaf#590, it uses this project as a submodule, so I think we may have the root cause here.

To Reproduce
Steps to reproduce the behavior:

Compare old version of GoldLeaf and latest version and install some nsp files.

Please fill the following information:

• Horizon OS (Switch FW) version: [e.g. 10.0.0] 12.1.0
• CFW: [e.g. Atmosphère, SX OS, etc.] Atmosphère
• CFW version: [e.g. 0.11.1, 2.9.4, etc.] 0.19.5
• Atmosphère launch method (if applicable): [e.g. Hekate, fusee-primary] Hekate
• libusbhsfs version: [e.g. 0.0.2] https://github.com/DarkMatterCore/libusbhsfs/tree/782aa51e0aa149427664cc3a9c2e520937576fcc
• Homebrew launch method: [e.g. title override, applet]
• USB Mass Storage device specs: [e.g. WD My Passport 1 TB, exFAT partition] exFAT MicroSD/Samsung 128 GB PRO Endurance

Logfile
Make sure you're using an application that's linked against the debug version of the library, and upload your logfile located at sdmc:/libusbhsfs.log.

Additional context
Add any other context about the problem here.

As per title, a clean install of devkit pro on both PC and Ubuntu, I get the following error trying to build the GPL version

ntfs.c
ntfs_dev.c
/home/mrdude/Desktop/libusbhsfs/source/ntfs-3g/ntfs_dev.c: In function 'ntfsdev_statvfs':
/home/mrdude/Desktop/libusbhsfs/source/ntfs-3g/ntfs_dev.c:825:10: error: implicit declaration of function 'NVolFreeSpaceKnown' [-Werror=implicit-function-declaration]
  825 |     if (!NVolFreeSpaceKnown(vd->vol) && ntfs_volume_get_free_space(vd->vol) < 0) ntfs_set_error_and_exit(ENOSPC);
      |          ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[1]: *** [/opt/devkitpro/devkitA64/base_rules:19: ntfs_dev.o] Error 1
make: *** [Makefile:154: lib/libusbhsfs.a] Error 2

Building ISC type builds fine with no errors.

ISC build fine

GPL (which is built while building nxdumptool, I guess?) halts on an implicit declaration of function: NVolFreeSpaceKnown in ntfs_dev.c

If I add -Wno-implicit-function-declaration to the cflags, it then hangs on an incompatible pointer type in ntfs_disk_io.c. (int vs long unsigned int).

Trying to make install on ISC build throws a ton of errors, even though actually building it works without error.

Can you use this to make the switch notice an sd card via dock?

libusbhsfs.log

Logfile provided by @fennectech. They're using a compressed 4TB NTFS volume -- the error itself seems to originate from NTFS-3G:

[2022-02-19 12:49:40.812947616] ntfs_delete -> Could not delete inode cache entry for /test_dir/libusbhsfs-example.txt
 (file "dir.c", line 2051, level 0x80).
[2022-02-19 12:49:40.925302924] ntfs_delete -> Could not delete inode cache entry for /test_dir/test.txt
 (file "dir.c", line 2051, level 0x80).
[2022-02-19 12:49:40.958309248] ntfs_delete -> Could not delete inode cache entry for /test_dir
 (file "dir.c", line 2051, level 0x80).

Gotta take a look and fix the error ASAP.

For anyone compiling on clean installed devkitpro mysy2 install on windows, if the following packages are not installed the build hangs when trying to create the tar.bz2 file to install.

pacman -S tar
pacman -S diffutils
pacman -S pkg-config

Also I installed bzip2 (pacman -S bzip2), not sure if this was needed or not.

Is your feature request related to a problem? Please describe.
For performance reasons, I've formatted some of my drives with mkfs.ext4 -O '^has_journal'. I don't store a lot of important things on these drives (they're merely used as temporary extra storage) so doing this made sense to me. However, when trying the example program in this repo, mounting the drive stops at https://github.com/DarkMatterCore/libusbhsfs/blob/main/source/lwext4/ext.c#L68

Describe the solution you'd like
Don't replay or start a new journal if the has_journal filesystem flag isn't set.

Describe alternatives you've considered
tune2fs -O has_journal allows me to create a journal without formatting the drive, which is what I'm doing for now, but eventually it'd be nice to just have it work without a journal.

I am using the latest build of DBI 195. ( DBI is using the latest release of libusbhsfs)
The developer mentioned that I should post this concern on this github
Source: rashevskyv/dbi#8

Running atmosphere 15 on 10.2 firmware
I am also running DBI on NSP forwarder.
I have three hardrives all formated to exfat

Two of my hardrives are using an enclosure (1 tera western digital, 2 tera toshiba). These are working well with DBI.
The third hardrive is a western digital passport external hardrive. DBI simply hangs after I tried accessing the files inside. I already tried reformating it but still, it won't read. The light indicator for the hardrive stays still.

Note:
I have tried all three hardrives with tinfoil and they all work. I do not know what causes the error for DBI.

Describe the bug
Applications hangs with Western Digital Elements SE Portable (WDBABV). I tried both 500GB and 1TB variants

To Reproduce
Steps to reproduce the behavior:
Connect a Western Digital Elements SE Portable (WDBABV) hard disk to the Switch (I tried also by using an Y cable with external power supply) and launch libusbhsfs-example (released one and a debug-rebuilt one)

Please fill the following information:

• Horizon OS (Switch FW) version: 11.1.0
• CFW: Atmosphère
• CFW version: 0.17.0 (with or without mesosphere)
• Atmosphère launch method (if applicable): Hekate or fusee-primary
• libusbhsfs version: v0.2.2
• Homebrew launch method: title override or applet
• USB Mass Storage device specs: Western Digital Elements SE Portable (WDBABV)

Logfile
Make sure you're using an application that's linked against the debug version of the library, and upload your logfile located at sdmc:/libusbhsfs.log.

libusbhsfs.log

Additional context
The hard disk(s) used to work with SX OS libusbfs