ml-tooling / ml-workspace Goto Github PK
View Code? Open in Web Editor NEW🛠 All-in-one web-based IDE specialized for machine learning and data science.
Home Page: https://mltooling.org/ml-workspace
License: Apache License 2.0
🛠 All-in-one web-based IDE specialized for machine learning and data science.
Home Page: https://mltooling.org/ml-workspace
License: Apache License 2.0
Hi! check out Pytorch Lightning as an option for your backend!
https://github.com/williamFalcon/pytorch-lightning
Describe the issue:
Technical details:
when we use vscode(http://localhost:8080/tools/vscode/), there is no reaction if we debug. But using vscode in VNC, it's ok.
Describe the bug:
When the folder name contains notebooks, seems this line of code will cause two unexpected popup, showing 404 as a result.
Console says js tried to load tooling/tools and tooling/storage/check after the baseURL was reconginized as the current full url, which caused the bug.
Expected behaviour:
Works fine without the popup.
Steps to reproduce the issue:
Technical details:
Possible Fix:
A better way of the regex?
Additional context:
I discovered this and thought it was related to #62 at first, and was somehow caused by the pydeck extension. Then later I found out that it was actually a different bug ;)
Hi,
According to the README, in the Authentication section: "You can also use to let Jupyter generate a random token that is printed out on the container logs."
I am trying to execute:
docker run -p 8080:8080 --env AUTHENTICATE_VIA_JUPYTER="<generated>" mltooling/ml-workspace:latest
but I cannot find any token in the logs produced by the command docker logs
.
Where is that generated token logged?
Thanks!
Feature description:
vscode, vnc, ungit, rstudio proceeds without password
Problem and motivation:
we can escape the risk of hacking.
Is this something you're interested in working on?
Can I set a password?
Describe the bug:
Not sure, what the root cause is or how to go about debugging but when running ml-workspace the cells return very slowly. htop shows the python process pinned at a 100 and ZMQ process also appears a lot. I'm using this command:
docker run -d -p 8080:8080 --name "ml-workspace" -v "${PWD}:/workspace" --env AUTHENTICATE_VIA_JUPYTER=")*==&11" --shm-size 512m --restart always mltooling/ml-workspace:latest
If I switch to using vanilla jupyter lab outside of docker everything is speedy again.
The time for cell execution is still accurate, it's in the order of ms, but the results arrive seconds later and the cell keeps being shown as busy with python process at a 100.
Expected behaviour:
Return values with the same speed, vanilla jupyter lab does.
Describe the bug:
ML Workspace fails to start and serve up Jupyter Notebooks
Started ML WORKSPACE with the Following Docker Command:
$ docker run -d \
-p 8080:8080 \
--name "ml-workspace" \
-v "/Users/<myuserid>/Documents/Notebooks:/workspace" \
--env AUTHENTICATE_VIA_JUPYTER="mytoken" \
--shm-size 512m \
--restart always \
mltooling/ml-workspace:latest
57fc561fa9eb1c38261e9c6a09a66a1a4c2ca9e686fbb1991996cb4befa1f8a6
$
It launched and reported the container id
Expected behaviour:
When I go to http://localhost:8080 I expect to get to my Jupyter Notebooks. Instead I get a waiting forever display and the logs as you see below.
Steps to reproduce the issue:
Run the docker command as described above. Open http://localhost:8080 in the browser.
DOCKER LOGS:
2020-09-30 14:18:47,850 [INFO] Starting...
2020-09-30 14:18:47,960 [INFO] Start Workspace
2020-09-30 14:18:48,082 [INFO] Running config backup restore.
2020-09-30 14:18:48,336 [INFO] Configure ssh service
Agent pid 41
Identity added: /root/.ssh/id_ed25519 (root@6d2fe55b74ff)
2020-09-30 14:18:48,449 [INFO] Configure nginx service
2020-09-30 14:18:48,532 [INFO] Configure tools
2020-09-30 14:18:48,599 [INFO] Configure cron scripts
2020-09-30 14:18:48,876 [INFO] Scheduling cron check xfdesktop task with with cron: 0 * * * *
2020-09-30 14:18:48,898 [INFO] Running cron jobs:
2020-09-30 14:18:48,899 [INFO] @hourly /opt/conda/bin/python '/resources/scripts/check_xfdesktop_leak.py' check> /proc/1/fd/1 2>/proc/1/fd/2
2020-09-30 14:18:48,996 [INFO] Starting configuration backup.
2020-09-30 14:18:49,316 [INFO] Scheduling cron config backup task with with cron: 0 * * * *
2020-09-30 14:18:49,323 [INFO] Running cron jobs:
2020-09-30 14:18:49,324 [INFO] @hourly /opt/conda/bin/python '/resources/scripts/check_xfdesktop_leak.py' check> /proc/1/fd/1 2>/proc/1/fd/2
2020-09-30 14:18:49,325 [INFO] @hourly . /resources/environment.sh; /opt/conda/bin/python '/resources/scripts/backup_restore_config.py' backup> /proc/1/fd/1 2>/proc/1/fd/2
2020-09-30 14:18:49,341 [INFO] Configure and run custom scripts
2020-09-30 14:18:50,338 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/filebrowser.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/glances.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/jupyter.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/netdata.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/novnc.conf" during parsing
2020-09-30 14:18:50,339 INFO Included extra file "/etc/supervisor/conf.d/rsyslog.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/sslh.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/ungit.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/vncserver.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/vscode.conf" during parsing
2020-09-30 14:18:50,340 INFO Included extra file "/etc/supervisor/conf.d/xrdp.conf" during parsing
2020-09-30 14:18:50,340 INFO Set uid to user 0 succeeded
2020-09-30 14:18:50,385 INFO RPC interface 'supervisor' initialized
2020-09-30 14:18:50,386 CRIT Server 'inet_http_server' running without any HTTP authentication checking
2020-09-30 14:18:50,388 INFO RPC interface 'supervisor' initialized
2020-09-30 14:18:50,388 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2020-09-30 14:18:50,389 INFO supervisord started with pid 95
2020-09-30 14:18:51,392 INFO spawned: 'rsyslog' with pid 98
2020-09-30 14:18:51,396 INFO spawned: 'nginx' with pid 99
2020-09-30 14:18:51,399 INFO spawned: 'sslh' with pid 100
2020-09-30 14:18:51,405 INFO spawned: 'sshd' with pid 101
2020-09-30 14:18:51,410 INFO spawned: 'jupyter' with pid 102
2020-09-30 14:18:51,413 INFO spawned: 'vncserver' with pid 104
2020-09-30 14:18:51,417 INFO spawned: 'cron' with pid 107
2020-09-30 14:18:51,422 INFO spawned: 'filebrowser' with pid 108
2020-09-30 14:18:51,427 INFO spawned: 'glances' with pid 109
2020-09-30 14:18:51,433 INFO spawned: 'netdata' with pid 113
2020-09-30 14:18:51,440 INFO spawned: 'novnc' with pid 116
2020-09-30 14:18:51,447 INFO spawned: 'ungit' with pid 118
2020-09-30 14:18:51,454 INFO spawned: 'vscode' with pid 123
2020-09-30 14:18:51,460 INFO spawned: 'xrdp' with pid 124
2020-09-30 14:18:52,395 INFO success: rsyslog entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,395 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,418 INFO success: sslh entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,457 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,457 INFO success: jupyter entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,461 INFO success: vncserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,461 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,461 INFO success: filebrowser entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,461 INFO success: glances entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,461 INFO success: netdata entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,462 INFO success: novnc entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,462 INFO success: ungit entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,463 INFO success: vscode entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-09-30 14:18:52,463 INFO success: xrdp entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020/09/30 14:19:34 [error] 136#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080"
2020/09/30 14:19:39 [error] 136#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080", referrer: "http://localhost:8080/"
2020/09/30 14:19:44 [error] 136#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080", referrer: "http://localhost:8080/"
2020/09/30 14:19:49 [error] 136#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080", referrer: "http://localhost:8080/"
....
2020/09/30 14:21:05 [error] 136#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080", referrer: "http://localhost:8080/"
...
SCREENSHOT:
Technical details:
docker exec -it 57fc561fa9eb1c38261e9c6a09a66a1a4c2ca9e686fbb1991996cb4befa1f8a6 /bin/sh; exit
# echo $WORKSPACE_VERSION
0.9.1
#
$ docker version
Client: Docker Engine - Community
Version: 19.03.12
API version: 1.40
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:41:33 2020
OS/Arch: darwin/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.12
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:49:27 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
$ docker run -d \
-p 8080:8080 \
--name "ml-workspace" \
-v "/Users/<myuserid>/Documents/Notebooks:/workspace" \
--env AUTHENTICATE_VIA_JUPYTER="mytoken" \
--shm-size 512m \
--restart always \
mltooling/ml-workspace:latest
$ docker exec -it 57fc561fa9eb1c38261e9c6a09a66a1a4c2ca9e686fbb1991996cb4befa1f8a6 /bin/sh; exit
# env
LANGUAGE=en_US:en
WORKSPACE_PORT=8080
HOSTNAME=57fc561fa9eb
XDG_CACHE_HOME=/root/.cache/
LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/opt/conda/lib
SHUTDOWN_INACTIVE_KERNELS=false
HOME=/root
KMP_BLOCKTIME=0
NB_USER=root
VNC_COL_DEPTH=24
WORKSPACE_HOME=/workspace
CONFIG_BACKUP_ENABLED=true
DATA_ENVIRONMENT=/workspace/environment
TERM=xterm
WORKSPACE_FLAVOR=full
PATH=/opt/node/bin:/opt/conda/bin:/usr/local/openresty/nginx/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
XDG_RUNTIME_DIR=/tmp
DISPLAY=:1
CONDA_PYTHON_DIR=/opt/conda/lib/python3.7
KMP_DUPLICATE_LIB_OK=True
KMP_AFFINITY=granularity=fine,compact,1,0
LANG=en_US.UTF-8
SHARED_LINKS_ENABLED=true
AUTHENTICATE_VIA_JUPYTER=mytoken
DEBIAN_FRONTEND=noninteractive
WORKSPACE_BASE_URL=/
SHELL=/usr/bin/zsh
RESOURCES_PATH=/resources
PYTHON_VERSION=3.7.6
SSL_RESOURCES_PATH=/resources/ssl
CONDA_DIR=/opt/conda
LC_ALL=en_US.UTF-8
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
VNC_RESOLUTION=1600x900
INCLUDE_TUTORIALS=true
WORKSPACE_VERSION=0.9.1
PWD=/root
USER_GID=0
VNC_PW=vncpassword
MAX_NUM_THREADS=auto
#
Possible Fix:
Additional context:
when I run an ml-workspace GPU container, jupyter_tensorboard raise error ' Invalid response: 500 Internal Server Error'. And when I run tensorboard on a terminal, it raises error 'ValueError: Duplicate plugins for name projector'. So how can I fix this problem?
Describe the bug:
When opening up a new notebook, using developer tools in Chrome shows 404 error for this path: /nbextensions/widgets/notebook/js/extension.js
and
/nbextensions/pydeck/extension.js
Expected behaviour:
Loads the js files successfully
Steps to reproduce the issue:
Create a new notebook, open developer tools in Chrome, console shows the 404 error.
Technical details:
Possible Fix:
Reinstall the extension in next build?
Additional context:
MLWorkspace strikes me as a phenomenal tool with broad applicability. I'm wondering if you might be able to provide an example of how to fit it into a standard multi-user jupyterhub + dockerspawner, where a 'start-singleuser' command spawns an image in which the primary user is not root, but jupyter's standard jovyan, and permissions are fixed accordingly. I've been attempting this, but it's proving a bit elusive to me.
Basically, is there a way of combining jupyter's base-notebook stack with mlworkspace to create a single-user compatible image that could be spawned from jupyterhub?
(MLHub seems like a natural step in this direction, but this would be for users who are interested in integrating MLWorkspace into their existing workflows and who might be gunshy about having single users have root permissions).
My thanks in advance.
Feature description:
can support the docker env BASE_URL lile jupyter NotebookApp.base_url, so that the url will like http://$IP/$BASE_URL/tree?
Problem and motivation:
in the way, it can be deployed behind the nginx, the nginx conf can be written like this:
location /$BASE_URL {
proxy_pass http://$INNER_IP:8080;
}
the user can use this url like http://$NGINX_IP/$BASE_URL
to visit the inner ml-workspace.
Describe the bug:
Cannot plot charts in RStudio due to graphics device error
Expected behaviour:
Charts should appear in RStudio's Viewer panel
Steps to reproduce the issue:
Technical details:
Server Docker Engine Community:
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 04:13:06 2019
OS/Arch: linux/amd64
Experimental: true
Possible Fix:
You may need to install Cairo in the Docker image before RStudio - https://stackoverflow.com/questions/19513705/error-in-rstudiogd-shadow-graphics-device-error-r-error-4-r-code-execution
Additional context:
Describe the bug:
I write some libary which uses the Plotly FigureWidget to create some interactive plot. This plot runs in the normal jupyter notebook. But in the jupyter lab I only see the message: "loading widget..."
Expected behaviour:
TODO
Steps to reproduce the issue:
TODO
Technical details:
Possible Fix:
While searching for the Jupyter-Lab extension I saw that the @jupyterlab/plotly extension is deprecated.
This package is being deprecated. Please use the Plotly-supported jupyterlab-plotly (https://www.npmjs.com/package/jupyterlab-plotly). See the plotly.py README for more info: https://github.com/plotly/plotly.py#jupyterlab-support-python-35)
Additional context:
Installation in the Dockerfile:
Line 736 in ae85a02
Hello, just wanted to ask if one can expect that this repository will get updates in the future or if it got abonded?
I use the jupyter labs and vscode on a daily basis but each of their own received a lot of updates since the last half year or so and ml-tooling hasnt had any updates since january.
Is it because of corona?
Describe the bug:
The Ungit link, from the workspace "Open Tool", is not loading the web interface for the Ungit.
Expected behaviour:
Open the web interface for Ungit
Steps to reproduce the issue:
Open the ML-Hub login to the server.
2. Create a new server with a given name
3. After the server spawn, access the server.
4. Open Tool -> Ungit
Technical details:
Possible Fix:
Additional context:
The file browser opens up without authentication (By typing link in browser directly) and runs as root. This pose a serious security issue.
when I try running jupyter labextension install @oriolmirosa/jupyterlab_materialdarker
it installs perfectly but fails to build. Any comments on that? BTW here are the logs for that:
Node v11.15.0
Building jupyterlab assets (build:prod:minimize)
> node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js install --non-interactive
yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning Pattern ["plotly-icons@latest"] is trying to unpack in the same destination "/root/.cache/yarn/v4/npm-plotly-icons-1.
3.13-31cbc2d8832df8a7ed81399ec32f03ef9043c783/node_modules/plotly-icons" as pattern ["plotly-icons@^1.1.5"]. This could resul
t in non-deterministic behavior, skipping.
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning "nbdime-jupyterlab > [email protected]" has unmet peer dependency "codemirror@^5.0.0".
warning "jupyterlab-chart-editor > react-chart-editor > [email protected]" has unmet peer dependency "[email protected].
x".
warning "jupyterlab-chart-editor > react-chart-editor > [email protected]" has unmet peer dependency "[email protected].
x".
warning "jupyterlab-chart-editor > react-chart-editor > [email protected]" has unmet peer dependency "[email protected]".
warning "jupyterlab-chart-editor > react-chart-editor > [email protected]" has incorrect peer dependency "react@^0.14.0
|| ^15.0.0".
warning "jupyterlab-chart-editor > react-chart-editor > draft-js-import-html > [email protected]" has unmet peer
dependency "[email protected]".
[5/5] Building fresh packages...
success Saved lockfile.
Done in 8.03s.
> node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer
yarn run v1.15.2
$ /opt/conda/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer
Done in 0.52s.
> node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js run build:prod:minimize
yarn run v1.15.2
$ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.minimize.config.js
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
JupyterLab failed to build
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/jupyterlab/debuglog.py", line 47, in debug_logging
yield
File "/opt/conda/lib/python3.6/site-packages/jupyterlab/labextensions.py", line 105, in start
core_config=self.core_config)
File "/opt/conda/lib/python3.6/site-packages/jupyterlab/commands.py", line 378, in build
command=command, clean_staging=clean_staging)
File "/opt/conda/lib/python3.6/site-packages/jupyterlab/commands.py", line 583, in build
raise RuntimeError(msg)
RuntimeError: JupyterLab failed to build
Exiting application: jupyter
/bin/bash ./mainscript.sh script that sneakily download xmrig v5.11.1 in the background as root.
Did the same thing happen to you?
Hi! Looks like a great all-in-one tool that you're building :)
Unfortunately, the version of GPflow you are pinning, 1.5.1 (see https://github.com/ml-tooling/ml-workspace/blob/master/resources/libraries/requirements-full.txt - same in develop branch) is out of date, and doesn't work with the also-pinned tensorflow 2.x. I would have simply opened a pull request on that file, but it looks like it is autogenerated and it is not clear to me how to change that. Also, GPflow requires TensorFlow 2.1 (which contains numerous bugfixes over TensorFlow 2.0, for example fixed memory leaks). What if anything could I do to help you upgrade the dependencies and bring ml-workspace up to date?
Describe the issue:
In the flavor "GPU" the python package "tensorflow-gpu" is installed here. This leads to problems if other python packages have the package "tensorflow" in their requirments.
Tensorflow changes the python package name from tensorflow-gpu
to tensorflow
. This package support CPU and GPU (https://www.tensorflow.org/install/pip)
Dockerfile:
ml-workspace/gpu-flavor/Dockerfile
Line 139 in ae85a02
Technical details:
Awesome project that I am using to make ML available to students.
Picked up this issue:
After following docs, cannot access webservices, log shows
exited: nginx (terminated by SIGILL (core dumped); not expected)
Possible Fix:
This is likely the same issue described here:
docksal/service-vhost-proxy#35
metwork-framework/mfext#384
Running on older hardware means no SSE4.2.
It would be great if this can be worked around so that older hardware can be used by students to access current tools. ML-Tooling already helps a lot when connecting to a remote server, which is great. This issue prevents running the container locally. Alternatively, SSE4.2 should be mentioned as a hardware requirement, or tested for in the container with appropriate logging.
Describe the bug:
Attempts to connect to VNC desktop fail with the following error:
Expected behaviour:
Ability to view desktop via novnc
Steps to reproduce the issue:
Technical details:
0.8.7
3.0.6
Ubuntu 18.04
docker run \
-p 8080:8080 \
--name "ml-workspace" \
--env AUTHENTICATE_VIA_JUPYTER="######" \
--shm-size 56G mltooling/ml-workspace:latest
Possible Fix:
Wish I knew!
Additional context:
Thanks a lot for providing this wonderful tool! Has been immensely helpful already.
Describe the issue:
I have a server that does not allow external internet connection. So it is not applicable to download docker image from hub. I wondered whether ml-workspace offers an offline deployment? Thanks in advance.
Technical details:
Feature description:
Slim ML-Workspace to some base set of features. e.g. Conda, Jupyter/Lab and HW Monitoring. Then have begin layering features atop, e.g. VNC, then File Browser, followed by Port Management, etc. (Presumably, these could be individually pulled without needing the prior steps.)
I can (kinda) foresee some wasted time when building, e.g. needing to run: apt-get install <some-pkg>
multiple times where <some-pkg>
is the same, but apt-get
already handles that for us.
Problem and motivation:
Generally, ML tools are pretty beefy, so it's expected that you've got copious disk space for this. 😅 However, I've found that I personally favor only a few of the built-ins of ML-Workspace. For instance, I've never used the VNC or Ungit. Additionally, I rarely use Glances or Netdata, instead preferring gotop
(but that's honestly a familiarity thing).
Is this something you're interested in working on?
Sure!
Hi, I would like to request update OpenCV to 4.4, this version support GPU
Below are the confis -
After every few hours, we are seeing the contain has been crashed.... Below are the pod logs along with yaml file..
Any one have set mount point as S3 and getting these errors -
[I 07:31:19.808 NotebookApp] Build is up to date
[I 07:31:51.717 NotebookApp] Starting buffering for 50495f03-f431-47a9-a64a-ec2135277139:56eab2bd-8c17-43d4-9bc3-9dbf5353d4eb
[I 07:32:04.503 NotebookApp] EOF on FD 75; stopping reading
[I 07:32:04.603 NotebookApp] Terminal 1 closed
[I 07:32:05.407 NotebookApp] EOF on FD 77; stopping reading
[I 07:32:05.507 NotebookApp] Terminal 2 closed
[I 07:32:10.601 NotebookApp] New terminal with automatic name: 1
[I 07:33:45.350 NotebookApp] Saving file at /package.json
[W 07:35:12.626 NotebookApp] delete /package-lock.json
[W 07:35:23.241 NotebookApp] delete /package-lock.json
[W 07:35:23.296 NotebookApp] 404 DELETE /api/contents/package-lock.json?1582270523109 (127.0.0.1): File or directory does not exist: /workspace/package-lock.json
[W 07:35:23.296 NotebookApp] File or directory does not exist: /workspace/package-lock.json
[W 07:35:23.297 NotebookApp] 404 DELETE /api/contents/package-lock.json?1582270523109 (127.0.0.1) 56.64ms referer=http://internal-a202f0609446511eaaa74020d1583377-1568446533.us-east-1.elb.amazonaws.com:8080/lab
[I 07:36:14.259 NotebookApp] Saving file at /package.json
2020-02-21 08:00:01,622 [INFO] Starting configuration backup.
2020-02-21 08:03:01,824 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
[I 08:03:48.643 NotebookApp] Saving file at /package.json
[I 08:03:51.421 NotebookApp] Saving file at /package.json
[I 08:09:33.208 NotebookApp] Saving file at /js_components.ipynb
[I 08:09:33.209 NotebookApp] Saving js_components.ipynb
[I 08:13:22.048 NotebookApp] New terminal with automatic name: 2
[W 08:18:31.169 NotebookApp] 404 GET /api/contents?type=directory&=1582272988521 (127.0.0.1): No such file or directory:
[W 08:18:31.169 NotebookApp] No such file or directory:
[W 08:18:31.170 NotebookApp] 404 GET /api/contents?type=directory&=1582272988521 (127.0.0.1) 1.32ms referer=http://internal-a202f0609446511eaaa74020d1583377-1568446533.us-east-1.elb.amazonaws.com:8080/tree?
[I 08:18:46.798 NotebookApp] Starting buffering for 77021263-d219-4458-aa86-4ad79285f8e4:86432548-a225-4331-a246-3445198a675c
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 09:03:02,056 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 10:03:01,301 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 11:03:01,547 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 12:03:01,778 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 13:03:02,046 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 14:03:01,289 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
Traceback (most recent call last):
File "/resources/scripts/backup_restore_config.py", line 53, in
os.makedirs(CONFIG_BACKUP_FOLDER)
File "/opt/conda/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/opt/conda/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
OSError: [Errno 107] Transport endpoint is not connected: '/workspace/.workspace'
2020-02-21 15:03:01,520 [INFO] Leak check: xfdesktop (mem=34578432.0 cpu=0.0); xfsettingsd (mem=17633280.0 cpu=0.0); xfce4-panel (mem=24084480.0 cpu=0.0)
YAML file is -
Wondering if there's an easy way to run other tools under port 8080, without exposing extra ports. For example Ray scheduler UI would listen on 8081?
For instance the open port feature would generate a URL but it can't be used with something like Ray's UI (both of them share the query string token)
http://172.17.0.2:8081/?token=c638fa7a06765f25408eefee6761091fd3716e7ae3fe177a
A working docker image was moved to new machine instead of downloading from scratch but the VNC part is not wrking it keeps disconnecting and reconnecting every few seconds or so.
Describe the bug:
Expected behaviour:
Steps to reproduce the issue:
Technical details:
Possible Fix:
Additional context:
It would be great if ml-workspace can be easily deployed on Kubernetes through helm or yaml scripts, and can expose metrics to promethues and grafana.
Feature description:
DataLad is a "data management and publication multitool". Learn more by visiting http://datalad.org and even better http://handbook.datalad.org/ or listening to any introductory talk on datalad like a recent debconf 2020 or slides from a past presentation at MILA, Montreal which was a little tuned for ML use-cases.
Problem and motivation:
DataLad solves many problems of version control, scalable access to remote data resources; growing amount of data is becoming available through git-annex (and thus DataLad): e.g. via http://datasets.datalad.org/ we provide access to over 200TB of (largely) neural data, and more to come!
Disclaimer: I am "involved" in DataLad project ;-)
Is this something you're interested in working on?
I guess. Although it might be just provision installation of DataLad within the ML workspace. We provide Debian packages (in stock Debian and backports from http://neuro.debian.net), it is available from PyPI and also from conda-forge. If ML is friendly to docker or singularity containers -- please also install datalad-container extension.
DataLad comes with command line and Python APIs. So it should work nicely for Jupyter notebooks users to access the data they want or to establish version control of their results, or even reproducible (re)execution via datalad run
Feature description:
The new major release of the JupyterLab 3.0 is released!
Problem and motivation:
The new version has more functionalities: Visual Debugger and a lot of other features and bugfixes
Here is the migration guide: JupyterLab 2.x to 3.x
Is this something you're interested in working on?
No
Having both master and main is confusing. I have initiated a PR first from master then only to arrive to conflicts. If master
is no longer used, best would be just to remove that branch git push origin :master
to avoid confusion etc
Describe the bug:
When I build the image with docker, I get errors.
I run the following command inside the project root folder:
docker image build -t ml-workspace .
After a while I get the error at step 50/97:
RuntimeError: JupyterLab failed to build
Expected behaviour:
Success build.
Steps to reproduce the issue:
I run the following command inside the project root folder:
docker image build -t ml-workspace .
After a while I get the error at step 50/97:
Step 50/97 : RUN jupyter lab build && jupyter labextension install @jupyter-widgets/jupyterlab-manager && if [ "$WORKSPACE_FLAVOR" = "minimal" ]; then jupyter lab clean && jlpm cache clean && rm -rf $CONDA_DIR/share/jupyter/lab/staging && clean-layer.sh && exit 0 ; fi && jupyter labextension install @jupyterlab/toc && jupyter labextension install jupyterlab_tensorboard && jupyter labextension install @jupyterlab/git && pip install jupyterlab-git && jupyter serverextension enable --py jupyterlab_git && jupyter labextension install jupyter-matplotlib && if [ "$WORKSPACE_FLAVOR" = "light" ]; then jupyter lab clean && jlpm cache clean && rm -rf $CONDA_DIR/share/jupyter/lab/staging && clean-layer.sh && exit 0 ; fi && pip install --pre jupyter-lsp && jupyter labextension install @krassowski/jupyterlab-lsp && jupyter labextension install @jupyterlab/plotly-extension && jupyter labextension install jupyterlab-chart-editor && jupyter labextension install @pyviz/jupyterlab_pyviz && jupyter labextension install @lckr/jupyterlab_variableinspector && jupyter labextension install @ryantam626/jupyterlab_code_formatter && pip install jupyterlab_code_formatter && jupyter serverextension enable --py jupyterlab_code_formatter && jupyter lab build && jupyter lab clean && jlpm cache clean && rm -rf $CONDA_DIR/share/jupyter/lab/staging && clean-layer.sh
---> Running in 62ce5b6a94e6
[LabBuildApp] JupyterLab 1.2.5
[LabBuildApp] Building in /opt/conda/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (build:prod:minimize)
An error occured.
RuntimeError: JupyterLab failed to build
Technical details:
Possible Fix:
Additional context:
Feature description:
It would be great if there could be a script in tools for installing a Julia kernel that would be available from Jupyter.
Problem and motivation:
It's extremely easy to get Python or R going with ml-workspace, but it would be nice if Julia was as straightforward.
Personally, I'm interested in getting more familiar with Julia and am also interested in using ml-workspace for my working environment. That's pretty much it. I think it would be useful for other Julia users.
Is this something you're interested in working on?
I'd be willing to try to figure it out.
Describe the issue:
Unable to build dockerfile and continually returns errors repeatedly.
Simply just trying to build the dockerfile while adding one extension that we have created into the dockerfile to have set up upon launching jupyterlab. Any help would be really appreciated.
Technical details:
can ml-workspace add the support for cuda9.0?
R notebook install package failed to use remotes::install_github
remotes::install_github("r-lib/devtools")
Error in system_check(unzip, args): Command failed sh: 1: : Permission denied
Traceback:
1. remotes::install_github("r-lib/devtools")
2. install_remotes(remotes, ...)
3. vapply(remotes, install_remote, ..., FUN.VALUE = logical(1))
4. FUN(X[[i]], ...)
5. source_pkg(bundle, subdir = remote$subdir)
6. decompress(path, outdir)
7. my_unzip(src, target)
8. system_check(unzip, args)
9. stop("Command ", command, " failed ", res$stderr)
remotes::install_github("r-lib/httr")
Error in system_check(unzip, args): Command failed sh: 1: : Permission denied
Traceback:
1. remotes::install_github("r-lib/httr")
2. install_remotes(remotes, ...)
3. vapply(remotes, install_remote, ..., FUN.VALUE = logical(1))
4. FUN(X[[i]], ...)
5. source_pkg(bundle, subdir = remote$subdir)
6. decompress(path, outdir)
7. my_unzip(src, target)
8. system_check(unzip, args)
9. stop("Command ", command, " failed ", res$stderr)
But use install.packages
is ok
I updated vscode from it's UI and now I always get redirected to localhost/login which is not a valid URL, all the other tools work fine, but not VScode. Any ideas how to debug or reset it?
Describe the bug:
We have a setup where we have some cloud machines with one gpu and multiple cpus, which we use for training some models. In such a context, we've tried running our usual notebooks from the ml-workspace docker image with Jupyter Notebook as well as Jupyter Lab, however we've noticed that during training or multi-process operations, only one CPU on the machine is being used. We've noticed this in netstat as well as using htop.
Expected behaviour:
When running jupyter notebooks that use multiprocess operations, all cpus on the host machine should be used.
Steps to reproduce the issue:
Here is a piece of code we tried to run specifically to test this behaviour:
def f(x):
while True:
x*x
pool = multiprocessing.Pool(3)
pool.map(f,range(100))
Technical details:
Possible Fix:
Our suspicion is that maybe there are some restrictions in the setup/run config for Jupyter Notebook.
Additional context:
Describe the bug:
Jupyterlab terminal fails to open from Launcher screen with error message i is not a constructor
. Also, trying to start a terminal in Jupyterlab by selecting file->new->terminal fails to open a new terminal tab
Expected behaviour:
A new terminal tab should be opened in Jupyterlab
Steps to reproduce the issue:
docker run -p 8080:8080 mltooling/ml-workspace:latest
Open Tool
, Jupyterlab
Terminal
iconTechnical details:
Server Docker Engine Community:
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.6
Git commit: 6247962
Built: Sun Feb 10 04:13:06 2019
OS/Arch: linux/amd64
Experimental: true
Possible Fix:
Additional context:
Using Safari as browser gives the error message undefined is not a constructor (evaluating 'new i(s,n)')
Describe the bug:
when I use VSCODE,I can't debug.There is no reaction while I press the bottom.
Another question is that why I open vscode from VNC,debug is OK.
And when I open vscode from VNC,it's unable to recognize Chinese.
Expected behaviour:
Steps to reproduce the issue:
Technical details:
Possible Fix:
Additional context:
Describe the bug:
imported an example zeppelin notebook from stock zeppelin docker image for testing spark
I got this:
java.lang.IllegalArgumentException: Unsupported class file major version 55
at org.apache.xbean.asm6.ClassReader.(ClassReader.java:166)
at org.apache.xbean.asm6.ClassReader.(ClassReader.java:148)
at org.apache.xbean.asm6.ClassReader.(ClassReader.java:136)
at org.apache.xbean.asm6.ClassReader.(ClassReader.java:237)
at org.apache.spark.util.ClosureCleaner$.getClassReader(ClosureCleaner.scala:49)
at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:517)
at org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:500)
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)
at scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:134)
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
at org.apache.spark.util.FieldAccessFinder$$anon$3.visitMethodInsn(ClosureCleaner.scala:500)
at org.apache.xbean.asm6.ClassReader.readCode(ClassReader.java:2175)
at org.apache.xbean.asm6.ClassReader.readMethod(ClassReader.java:1238)
at org.apache.xbean.asm6.ClassReader.accept(ClassReader.java:631)
at org.apache.xbean.asm6.ClassReader.accept(ClassReader.java:355)
at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(ClosureCleaner.scala:307)
at org.apache.spark.util.ClosureCleaner$$anonfun$org$apache$spark$util$ClosureCleaner$$clean$14.apply(ClosureCleaner.scala:306)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$clean(ClosureCleaner.scala:306)
at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:162)
at org.apache.spark.SparkContext.clean(SparkContext.scala:2326)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1.apply(RDD.scala:850)
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1.apply(RDD.scala:849)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
at org.apache.spark.rdd.RDD.mapPartitionsWithIndex(RDD.scala:849)
at org.apache.spark.sql.execution.aggregate.HashAggregateExec$$anonfun$doExecute$1.apply(HashAggregateExec.scala:102)
at org.apache.spark.sql.execution.aggregate.HashAggregateExec$$anonfun$doExecute$1.apply(HashAggregateExec.scala:95)
at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)
at org.apache.spark.sql.execution.aggregate.HashAggregateExec.doExecute(HashAggregateExec.scala:95)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.WholeStageCodegenExec.liftedTree1$1(WholeStageCodegenExec.scala:606)
at org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:600)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.prepareShuffleDependency(ShuffleExchangeExec.scala:92)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:128)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeExec$$anonfun$doExecute$1.apply(ShuffleExchangeExec.scala:119)
at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:52)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.doExecute(ShuffleExchangeExec.scala:119)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.InputAdapter.inputRDDs(WholeStageCodegenExec.scala:391)
at org.apache.spark.sql.execution.aggregate.HashAggregateExec.inputRDDs(HashAggregateExec.scala:151)
at org.apache.spark.sql.execution.WholeStageCodegenExec.doExecute(WholeStageCodegenExec.scala:627)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.TakeOrderedAndProjectExec.executeCollect(limit.scala:136)
at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collectFromPlan(Dataset.scala:3389)
at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2550)
at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2550)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3370)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3369)
at org.apache.spark.sql.Dataset.head(Dataset.scala:2550)
at org.apache.spark.sql.Dataset.take(Dataset.scala:2764)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.zeppelin.spark.SparkZeppelinContext.showData(SparkZeppelinContext.java:108)
at org.apache.zeppelin.spark.SparkSqlInterpreter.interpret(SparkSqlInterpreter.java:136)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:103)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:632)
at org.apache.zeppelin.scheduler.Job.run(Job.java:188)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Expected behaviour:
Steps to reproduce the issue:
So it seems that some java version mismatch is going on.
Technical details:
Possible Fix:
Additional context:
Hi,
i receive a Error: 400 Client Error: Bad Request ("Unknown runtime specified nvidia")
my-nvidia setup
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64 Driver Version: 440.64 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN RTX Off | 00000000:0A:00.0 Off | N/A |
| 41% 37C P8 19W / 280W | 16MiB / 24217MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 TITAN RTX Off | 00000000:0B:00.0 Off | N/A |
| 40% 34C P8 9W / 280W | 1MiB / 24220MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1302 G /usr/lib/xorg/Xorg 9MiB |
| 0 1673 G /usr/bin/gnome-shell 4MiB |
+-----------------------------------------------------------------------------+
Running "docker run --gpus all nvidia/cuda:10.0-base nvidia-smi" runs without problems. When iam running the mltooling/ml-workspace-gpu manually, tensorflow cant find any GPU.
what iam doing wrong?
Describe your request:
If JupyterLab is used it isn't possible to passing JavaScript objects to the IPython kernel. This is described in the Jupyter Tipps & Tricks tutorial section 4.6 Run Javascript. I added some information (#5 ) but it's maybe a good idea to add this information also to a more general area to avoid nasty behaviors/bugs.
Which page or section is this issue related to?
Using docker run -p 8080:8080 mltooling/ml-workspace:latest
the corresponding section is at: http://localhost:8080/notebooks/tutorials/jupyter-tipps.ipynb#Run-Javascript
For Jupyter labs, I have a specific requirement - Instead of showing multiple notebooks as a tree view, i would like to show all the notebooks as in the TILE view as attached.
Please suggest of Jupyter Labs has any existing extension to achieve the same?
I searched a lot but could not figured it out!!
Please add pyCaret, https://github.com/pycaret/pycaret
and darts https://github.com/unit8co/darts.
Best regards ..
sudo docker run -d -p 8080:8080 --gpus all --name "ml-workspace" -v "${PWD}:/workspace" --env AUTHENTICATE_VIA_JUPYTER="mytoken" --shm-size 512m --restart=no mltooling/ml-workspace-gpu:latest
sudo docker run -d -p 8080:8080 --gpus all --name "ml-workspace" -v "${PWD}:/workspace" --env AUTHENTICATE_VIA_JUPYTER="mytoken" --shm-size 512m --restart=no mltooling/ml-workspace-gpu:latest
ml-workspace If you click filebrowser in the right tool, i can connect to port 0000. Can you change the default port?
It's troublesome to change one by one
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.