dirs-dev / directories-jvm Goto Github PK

At the moment support is provided for user-level directories of various kinds, but it would be great if there was support for system-level configuration directories too, as they're often used as a fallback in applications if user-level configuration can't be found.

I printed all the directories from new ProjectDirectories.from("com", "Foo Corp", "Bar App"), BaseDirectories and UserDirectories on Windows 10 and I got the following:

ProjectDirectories

projectPath=Foo Corp/Bar App
tcacheDir=C:\Users\<myName>\AppData\Roaming/Foo Corp/Bar App/data/cache
configDir=C:\Users\<myName>\AppData\Roaming
dataDir=C:\Users\<myName>\AppData\Roaming
tdataLocalDir=C:\Users\<myName>\AppData\Local/Foo Corp/Bar App/data
runtimeDir=null

BaseDirectories

homeDir=C:\Users\<myName>
dataLocalDir=C:\Users\<myName>\AppData\Local
cacheDir=C:\Users\<myName>\AppData\Local
executableDir=null

UserDirectories

audioDir=null
fontDir=null
desktopDir=D:\<myName>\Desktop
documentDir=null
downloadDir=null
pictureDir=null
publicDir=null
templateDir=C:\Users\<myName>\AppData\Roaming\Microsoft\Windows\Templates
videoDir=null

Nearly none of the UserDirectories is detected.

Note: The Desktop location is correct, I remapped it to a secondary drive

I have had trouble on a windows machine using coursier, metals, and other tools which have a transitive dependency on directories-jvm. After doing some tracing, I was able to discover that the issue is that directories-jvm invokes powershell for windows without the -noprofile flag.

Pulling the code down locally and simply adding the -Noprofile flag fixed the path resolutions which previously failed for me.

  static String[] getWinDirs(String... guids) {

    int guidsLength = guids.length;
    StringBuilder buf = new StringBuilder(guidsLength * 68);
    for (int i = 0; i < guidsLength; i++) {
      buf.append("[Dir]::GetKnownFolderPath(\"");
      buf.append(guids[i]);
      buf.append("\")\n");
    }

    String encodedCommand = SCRIPT_START_BASE64 + toUTF16LEBase64(buf.toString() + "}");

    return runCommands(guidsLength, Charset.forName("UTF-8"),
        "powershell.exe",
        "-Noprofile",
        "-EncodedCommand",
        encodedCommand
    );
  }

Upon inspection, it's not really clear if there are enough shared semantics across platforms to carve out something like this.

Also, there is the question of /tmp vs. /var/tmp on Linux.

The $XDG_STATE_HOME contains state data that should persist between (application) restarts, but that is not important or portable enough to the user that it should be stored in $XDG_DATA_HOME.

Hello,
Because I want to use your jar with jlink, having an automatic-module-name isn't enough, a module-info.java file is necessary.
However, it means compiling with java 9, thus droping the java 6 support, except if you create a multi-release jar or use another branch for java 9.
I have found no way with the configured version of sbt, neither to compile a module-info.java file, nor to create a multi-release jar. But sbt isn't a build tool i'm used to.
For the moment, I have created a real modular version in my fork of your repository, using maven to build.
Do you see a better solution ?

See Col-E/Recaf#278 for more info.

Hi! Thanks for the great work on the library!

I noticed that there are recent fixes that might benefit some users of our project (Metals). I was wondering if it's possible to make a new release with those recent changes? I there is anything blocking maybe we can help out? Thanks!

The related issue in Metals: scalameta/metals-vscode#208

@SackCastellon Could you have a look at dfba1ea and test it on your system to make sure it returns all the results we expect?

I'm a bit concerned about the Linux change, as instead of running xdg-user-dirs directly, we are now running bash to execute multiple commands. I should probably benchmark this, to make sure we are actually faster...

I just found java-native-access/jna and I think instead of calling the PowerShell we could use Shell32Util.getKnownFolderPath(Guid.GUID guid) together with KnownFolders to get the corresponding folder path.

For example:

String desktopPath = Shell32Util.getKnownFolderPath(KnownFolders.FOLDERID_Desktop);

What do you think?

It looks like on Windows (though not reproducible on every machine) configDir comes with an UTF-8 BOM prefix.

It was suggested by @alexarchambault that we should strip it automatically similar to coursier/coursier@fac264f

The full discussion is here scalameta/metals#1925

Should we just do that in this case?

Related to coursier/coursier#904.

On Windows, if the user name contains a special character (e.g. é), the directories provided by ProjectDirectories seems right (toString prints chars that seem correct), but can't be passed to java.io.File (the corresponding file / directory doesn't exist). On the other hand, sys.env("LOCALAPPDATA") gives a path with an apparently wrong special character in toString… but java.io.File accepts it, and it actually correspond to the underlying file / directory.

See https://github.com/alexarchambault/directories-windows-special-char for a small repro. From Windows with a username having a special character, sbt run fails for me, like

[info] Compiling 1 Scala source to C:\Users\HalÞx\projects\coursier\foo\target\scala-2.12\classes ...
[info] Done compiling.
[info] Packaging C:\Users\HalÞx\projects\coursier\foo\target\scala-2.12\foo_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Running Check
dirNameViaDirectories=Halèx, isDirectory: false
dirNameViaLocalAppData=HalÞx, isDirectory: true
[error] (run-main-0) java.lang.AssertionError: assertion failed: Halèx != HalÞx
[error] java.lang.AssertionError: assertion failed: Halèx != HalÞx
[error]         at scala.Predef$.assert(Predef.scala:219)
[error]         at Check$.main(Check.scala:16)
[error]         at Check.main(Check.scala)
[error]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.lang.reflect.Method.invoke(Method.java:498)
[error] Nonzero exit code: 1
[error] (Compile / run) Nonzero exit code: 1

Many contributors have spent heroic efforts to keep Windows support working, for which I'm greatly thankful.

Though it appears to me that Windows support keeps breaking time and time again – perhaps it's time to think whether a different approach could be less painful and more respectful to the time & efforts of contributors?

I'm short on actual ideas though:

• Alternative approaches that DON'T work:
  • Reading environment variables.
  • Reading registry values.
  • Shipping pre-compiled native code.
• Alternative approaches that may work:
  • Shipping a .NET assembly that replaces the current Powershell + .NET part?

I'm open to suggestions, thoughts, ideas, etc. – what do people (@alexarchambault, @eatkins, @fthomas. @phongngtuan, ...) think?

OS: Windows 10

A java program calling Util.getWinDirs() triggers an antivirus scan on my laptop (Avecto Defendpoint) and takes almost 2 minutes to complete.
E.g this class:

public class A {
  public static void main(String[] a) throws Exception {
    System.out.println("Hello! " + new java.util.Date());
    io.github.soc.directories.ProjectDirectories.from("crash", "boom", "bang");
    System.out.println("Bye! " + new java.util.Date());
  }
}

prints

Hello! Sat Jan 18 21:30:56 GMT 2020
Bye! Sat Jan 18 21:32:40 GMT 2020

Interestingly, if I save the powershell script that ProjectDirectories use into a file a.ps1 and run it like this: powershell -File a.ps1 then it completes in a couple of seconds.

Version 26

// LIB=C:\nope

import dev.dirs.ProjectDirectories
ProjectDirectories.from("com", "Foo Corp", "Bar App")

It's stuck on a powershell call:
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -version 2 -NoProfile -EncodedCommand JgAg...

When I try running it manually it fails:

Add-Type : (0) : Warning as Error: Invalid search path 'C:\nope' specified in '
LIB environment variable' -- 'The system cannot find the path specified.

When running the program using PowerShell 7, version 26 of directories-jvm returns a path with leading null.

Output in PowerShell:

ProjectDirectories (Windows 10):
  projectPath   = 'foo\bar'
  cacheDir      = 'null\foo\bar\cache'
  configDir     = 'PowerShell 7.1.3\foo\bar\config'
  dataDir       = 'PowerShell 7.1.3\foo\bar\data'
  dataLocalDir  = 'null\foo\bar\data'
  preferenceDir = 'PowerShell 7.1.3\foo\bar\config'
  runtimeDir    = 'null'

Expected output (which you get for example in the default system embedded PowerShell or using cmd):

ProjectDirectories (Windows 10):
  projectPath   = 'foo\bar'
  cacheDir      = 'C:\Users\jakob\AppData\Local\foo\bar\cache'
  configDir     = 'C:\Users\jakob\AppData\Roaming\foo\bar\config'
  dataDir       = 'C:\Users\jakob\AppData\Roaming\foo\bar\data'
  dataLocalDir  = 'C:\Users\jakob\AppData\Local\foo\bar\data'
  preferenceDir = 'C:\Users\jakob\AppData\Roaming\foo\bar\config'
  runtimeDir    = 'null'

Latest version 23 creates invalid paths with leading null\ on Window 10.

I encountered this when updating SBT / coursier and dependencies where downloaded to a null dir inside my project.

Actual

import dev.dirs.ProjectDirectories
import $dep.`dev.dirs:directories:23`

ProjectDirectories.from("org", "scalameta", "metals")
// res0: ProjectDirectories = ProjectDirectories (Windows 10):
//   projectPath   = 'scalameta\metals'
//   cacheDir      = 'null\scalameta\metals\cache'
//   configDir     = 'null\scalameta\metals\config'
//   dataDir       = 'null\scalameta\metals\data'
//   dataLocalDir  = 'null\scalameta\metals\data'
//   preferenceDir = 'null\scalameta\metals\config'
//   runtimeDir    = 'null'

Expected

import dev.dirs.ProjectDirectories
import $dep.`dev.dirs:directories:22`

ProjectDirectories.from("org", "scalameta", "metals")
// res0: ProjectDirectories = ProjectDirectories (Windows 10):
//   projectPath   = 'scalameta\metals'
//   cacheDir      = 'C:\Users\user\AppData\Local\scalameta\metals\cache'
//   configDir     = 'C:\Users\user\AppData\Roaming\scalameta\metals\config'
//   dataDir       = 'C:\Users\user\AppData\Roaming\scalameta\metals\data'
//   dataLocalDir  = 'C:\Users\user\AppData\Local\scalameta\metals\data'
//   preferenceDir = 'C:\Users\user\AppData\Roaming\scalameta\metals\config'
//   runtimeDir    = 'null'

Environment

OS: Windows 10 1903
JVM: AdoptOpenJDK 1.8.0_222 / 1.8.0_282

I've encountered this issue while using SBT and coursier and I've opened the issue there too: coursier/coursier#1919

Issue

I'm using coursier with SBT under a WSL environment. After updating SBT from version 1.3.13 to 1.4.1 coursier is looking at the wrong path for the cache, specifically it's using a windows path, and dependency resolution fails.

Environment

OS: Windows 10, but running under WSL/Ubuntu 18.04.4 LTS
SBT: 1.4.1 (don't know how to check coursier version)
JVM: Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

Current behaviour

Under WSL, coursier uses a Windows path to resolve the cache:

$ sbt
...
[info] loading project definition from /mnt/c/Users/JamesSmith/Projects/myproject/project
[error] java.nio.file.NoSuchFileException: C:\Users\JamesSmith\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\config\1.3.3\config-1.3.3.jar
...
[error] (Compile / bloopGenerate) java.nio.file.NoSuchFileException: C:\Users\JamesSmith\AppData\Local\Coursier\cache\v1\https\repo1.maven.org\maven2\com\typesafe\config\1.3.3\config-1.3.3.jar

Expected behaviour

Coursier should use a WSL path when working inside WSL

This was originally reported by @xshi0001 as sbt/sbt#5386

steps

sbt version: 1.3.4 window10

problem

xshi0@LAPTOP-L904TFAI MINGW64 /d/WorkPlace/Flink/2-demo/springBoot-flink
$ sbt
[warn] No sbt.version set in project/build.properties, base directory: D:\WorkPlace\Flink\2-demo\springBoot-flink
[error] java.io.IOException: Cannot run program "powershell.exe": CreateProcess error=2, 系统找不到指定的文件。
[error] Use 'last' for the full log.
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? l

Hi! I just finished an initial port of this library to .NET. I would like to donate it (and offer continued maintenance) to the dirs-dev org. Is that something you might be interested in?

Hi,

I can't reproduce it myself since I don't have that OS but it seems that ProjectDirectories#fromPath does not always work on Windows 8.1 (background: the Functional Programming class at EPFL has assignments that use scalafix for linting, scalafix uses coursier, coursier uses directories), here's the stack trace from one student:

 [error] java.lang.NullPointerException
[error]         at com.geirsson.shaded.directories.ProjectDirectories.fromPath(ProjectDirectories.java:221)
[error]         at com.geirsson.shaded.directories.ProjectDirectories.from(ProjectDirectories.java:272)
[error]         at com.geirsson.shaded.coursier.CoursierPaths.cacheDirectory(CoursierPaths.java:61)
[error]         at com.geirsson.shaded.coursier.CachePath.defaultCacheDirectory(CachePath.java:88)
[error]         at com.geirsson.shaded.coursier.Cache$.default$lzycompute(Cache.scala:1131)
[error]         at com.geirsson.shaded.coursier.Cache$.default(Cache.scala:1131)
    [error]         at com.geirsson.shaded.coursier.Cache$.fetch$default$1(Cache.scala:1023)
[error]         at com.geirsson.coursiersmall.CoursierSmall$.fetch(CoursierSmall.scala:39)
    [error]         at scalafix.internal.sbt.ScalafixCoursier$.scalafixCliJars(ScalafixCoursier.scala:22)
[error]         at scalafix.internal.sbt.ScalafixInterface$.$anonfun$fromToolClasspath$1(ScalafixInterface.scala:20)
[error]         at scala.util.Try$.apply(Try.scala:209)
    [error]         at scalafix.internal.sbt.ScalafixInterface$LazyValue._value$lzycompute(ScalafixInterface.scala:12)
    [error]         at scalafix.internal.sbt.ScalafixInterface$LazyValue._value(ScalafixInterface.scala:12)
[error]         at scalafix.internal.sbt.ScalafixInterface$LazyValue.apply(ScalafixInterface.scala:13)
    [error]         at scalafix.sbt.ScalafixPlugin$.scalafixArgs(ScalafixPlugin.scala:96)
[error]         at scalafix.sbt.ScalafixPlugin$.$anonfun$scalafixInputTask$4(ScalafixPlugin.scala:110)
[error]         at sbt.std.FullInstance$.$anonfun$flattenFun$3(TaskMacro.scala:79)
[error]         at scala.Function1.$anonfun$compose$1(Function1.scala:44)
    [error]         at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
    [error]         at sbt.std.Transform$$anon$4.work(System.scala:67)
    [error]         at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
    [error]         at sbt.Execute.work(Execute.scala:278)
    [error]         at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error]         at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
    [error]         at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
    [error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    [error]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error]         at java.lang.Thread.run(Thread.java:748)
[error] (Compile / scalafix) java.lang.NullPointerException

I am using scala metals library in vscode. I got error stating as below

"main" java.nio.file.InvalidPathException: Illegal char <:> at index 7: WARNING: (0) : No source files specified\bsp\data

In scala metals library we are following this issue in #1085 (scalameta/metals#1085). We came to know the issue is with directories-jvm as scala metals is using directories-jvm.

Below code gives us an info about ProjectDirectories configured

System.out.println(ProjectDirectories.fromPath("bsp"));

Output:

ProjectDirectories (Windows 10): projectPath = 'bsp' cacheDir = '\bsp\cache' configDir = 'WARNING: (0) : No source files specified\bsp\config' dataDir = 'WARNING: (0) : No source files specified\bsp\data' dataLocalDir = '\bsp\data' runtimeDir = 'null'

I understand that in configDir path a warning message is append to the path which causing the error. But i don't have any idea what it is happening, can please check on this issue.

This was originally reported as sbt/sbt#5206 On Windows, Coursier creates "./null/Coursier/cache/v1" directories as cache and coursier/coursier#1438

steps

1. Use Windows.
2. Download the AdoptOpenJDK 8 232-b09 zip file (https://github.com/AdoptOpenJDK/openjdk8-binaries/releases/download/jdk8u232-b09/OpenJDK8U-jdk_x86-32_windows_hotspot_8u232b09.zip)
3. Launch sbt as follows:

sbt --java-home /cygdrive/c/Users/foo/Downloads/OpenJDK8U-jdk_x86-32_windows_hotspot_8u232b09/jdk8u232-b09/

4. add directories-jvm to libraryDependencies and call

scala> io.github.soc.directories.ProjectDirectories.fromPath("Coursier")

problem

ProjectDirectories (Windows 7):
  projectPath  = 'Coursier'
  cacheDir     = 'null\Corusier\cache'
  configDir    = 'null\Corusier\config'
  dataDir      = 'null\Corusier\data'
  dataLocalDir = 'null\Corusier\data'
  runtimeDir   = 'null'

It points a relative path named "null".

expectation

https://github.com/soc/directories-jvm#projectdirectories

notes

https://github.com/soc/directories-jvm/blob/8849872b9b71a378e6356f69bcf58427cb681e2f/src/main/java/io/github/soc/directories/Util.java#L113-L145

I think it'd be nice / convenient if directories-jvm had a CLI, to print the directories of an app, from its name, on the current OS. It could be used like

$ directories coursier --cache-dir
/Users/alex/Library/Caches/Coursier

Solaris is not supported.

Conseqently sbt 1.3.2 fails with a supprising error lmcoursier.internal.shaded.coursier.cache.shaded.io.github.soc.directories.UnsupportedOperatingSystemException: directories are not supported on SunOS

patch necessary (at least) on
io/github/soc/directories/Util.java:33
io/github/soc/directories/UserDirectories.java:305

If PowerShell is in Constrained Language Mode, that is PS C:\> $ExecutionContext.SessionState.LanguageMode prints ConstrainedLanguage instead of FullLanguage, Util.getWinDirs throws an exception which can be observed by running dev.dirs.DirectoriesTest:

sbt:directories> testOnly dev.dirs.DirectoriesTest
[info] Total time: 0 sec
[info] Compiling 1 Java source to C:\Project\prj\directories-jvm\target\test-classes ...
[error] Test dev.dirs.DirectoriesTest.testBaseDirectories failed: java.lang.RuntimeException: Couldn't find pwsh.exe or powershell.exe on path or in default system locations, took 2.545 sec
[error]     at dev.dirs.Util.windowsFallback(Util.java:280)
[error]     at dev.dirs.Util.getWinDirs(Util.java:161)
[error]     at dev.dirs.BaseDirectories.<init>(BaseDirectories.java:274)
[error]     at dev.dirs.BaseDirectories.get(BaseDirectories.java:246)
[error]     at dev.dirs.DirectoriesTest.testBaseDirectories(DirectoriesTest.java:9)
[error]     ...
[error] Caused by: java.io.IOException: no output from process
[error]     at dev.dirs.Util.runCommands(Util.java:206)
[error]     at dev.dirs.Util.runWinCommands(Util.java:241)
[error]     at dev.dirs.Util.windowsFallback(Util.java:278)
[error]     ... 41 more
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]         dev.dirs.DirectoriesTest
[info] Total time: 3 sec
[info]   root / Test / testOnly: 3 sec
[error] (Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 3 s, completed 19.11.2020 14:08:57

The inner exception java.io.IOException: no output from process is thrown because the PowerShell process exits with only printing to stderr and nothing to stdout.

PowerShell exits with errors when trying to execute the SCRIPT_START_BASE64 script. The error can be reproduced in a PowerShell console by executing the first line of that script:

PS C:\> $ExecutionContext.SessionState.LanguageMode
FullLanguage
PS C:\> [Console]::OutputEncoding = [System.Text.Encoding]::UTF8

PS C:\> $ExecutionContext.SessionState.LanguageMode = "ConstrainedLanguage"

PS C:\> $ExecutionContext.SessionState.LanguageMode
ConstrainedLanguage
PS C:\> [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
Die Eigenschaft kann nicht festgelegt werden. Das Festlegen der Eigenschaft wird in diesem Sprachmodus nur für Kerntypen unterstützt.
In Zeile:1 Zeichen:1
+ [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : PropertySetterNotSupportedInConstrainedLanguage

Note that [Console]::OutputEncoding = [System.Text.Encoding]::UTF8 does not exit with an error in FullLanguage mode.

That Util.getWinDirs throws an exception if PowerShell is in Constrained Language Mode is the reason why the current sbt 1.4 versions are not usable in such environments as noted in #40 (comment) and coursier/coursier#1913.

A possible workaround to make sbt usable again would be removing this line.

is this on purpose? at least https://github.com/soc/directories/blob/master/src/main/java/io/github/soc/directories/BaseDirectories.java looks quite weird.