System cannot find the path specified about runhouse HOT 8 OPEN

Cc @@concretevitamin @Michaelvll, this looks like it might be an issue in the SkyPilot launch flow on Azure, any ideas?

from runhouse.

It may be related to Windows paths. Not sure if we tested Windows. Cc @romilbhardwaj

from runhouse.

We haven't tested running natively on Windows. However, I can confirm that SkyPilot works in a python environment inside Windows Subsystem for Linux (WSL). Perhaps windows users can use WSL as a workaround?

from runhouse.

@romilbhardwaj I did some more looking into the code and found that one of the issues appears to be from the temp yml file being generated that specifies the properties of the machine.

file_mounts: {
  "~/.sky/sky_ray.yml": "C:\Users\stollbak\.sky\generated\test.yml.tmp",
  "~/.sky/wheels/5f10c8a76630d0f617f0312055a347bf": "C:\Users\stollbak\AppData\Local\Temp\5f10c8a76630d0f617f0312055a347bf",
  "~/.azure/azureProfile.json": "~/.azure/azureProfile.json",
  "~/.azure/clouds.config": "~/.azure/clouds.config",
  "~/.azure/config": "~/.azure/config",
  "~/.azure/msal_token_cache.json": "~/.azure/msal_token_cache.json",
}

Looks like backslashes are creating issues since they are being treated as escape characters. Where is this file being generated in the library? I can take a look and see about a fix.

from runhouse.

Hi @BStoller, the tmp yml file is generated here.

This method fills in a template YAML. The offending paths in your example above are called sky_ray_yaml_local_path and sky_local_path in our template.

• To fix sky_ray_yaml_local_path (i.e., C:\Users\stollbak\.sky\generated\test.yml.tmp in your example), you may want to modify _get_yaml_path_from_cluster_name here.

• To fix sky_ray_yaml_local_path (i.e., C:\Users\stollbak\AppData\Local\Temp\5f10c8a76630d0f617f0312055a347bf in your example), you may want to modify the temp_wheel_dir.absolute() returned here.

We're very open to contributions and I look forward to your PR in the SkyPilot repo :)

from runhouse.

@BStoller , I'm happy to help write up a change if you'd test them on your windows box. I actually have a few other changes I was thinking to contribute up to SkyPilot shortly.

from runhouse.

@dongreenberg No problem, I can test the changes. Let me know if you need any more information.

from runhouse.

This is problem from os.devnull usage (in skypilot) in windows. Windows does not offer os.devnull at default, but WSL offers it. Thus the result is different.

import os
log_path = os.devnull
log_dir = os.path.expanduser(os.path.dirname(log_path))

This will result '' for some windows environment, but will work in linux or its subsystems.

It is headache for those windows systems- windows can't do that.

from runhouse.