jupyterhub / jupyter-remote-desktop-proxy Goto Github PK
View Code? Open in Web Editor NEWRun a Linux Desktop on a JupyterHub
License: BSD 3-Clause "New" or "Revised" License
Run a Linux Desktop on a JupyterHub
License: BSD 3-Clause "New" or "Revised" License
We have no tests setup, and we have seen how changes we make have broken things either for users of TigerVNC or users of TurboVNC. With this in mind, I'd like us to ensure that we can at least startup things successfully without erroring when using all of the VNC servers we decide to support.
I think a way to help us test TurboVNC also is to go for #88 and provide a TurboVNC image as well.
Extracted from @manics in #84 (comment):
Just a thought, is tigervnc-xorg-extension actually needed?
https://packages.debian.org/sid/tigervnc-xorg-extension
This package contains an X server connector so VNC clients can connect to your local X desktop directly.
I don't know the implications of this, @yuvipanda do you know?
Note that it sais Provides: vnc-server, xserver
below, so the actual "x server" seems to be bundled with the standalone-server apt package.
sudo apt show tigervnc-standalone-server
Package: tigervnc-standalone-server
Version: 1.12.0+dfsg-4ubuntu0.22.04.1
Priority: optional
Section: universe/x11
Source: tigervnc
Origin: Ubuntu
Maintainer: Ubuntu Developers <[email protected]>
Original-Maintainer: TigerVNC Packaging Team <[email protected]>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 2 874 kB
Provides: vnc-server, xserver
# ...
This is written to work with Python 3.6+ currently in setup.py, can we reduce that to Python 3.8+? If we get Python version specific tests, we won't need to conclude failures and then decide on adding patches to support for very old versions etc.
I've concluded that the current images in :main
image doesn't work when started with jupyterhub-singleuser
4.1.3.
It doesn't matter if jupyterhub 4.0.2 etc is used on the server side - this is user environment side stuff where the jupyter server starts rejecting things.
I've tested to conclude that if jupyterhub-singleuser 4.1.0-4.1.3 is used in the user environment, we error, but if 4.0.2 or 4.1.4 is used we are OK and things work. So, this can be closed once we can publish an image that will use jupyterhub 4.1.4. I've triggered a rebuild of jupyter/docker-stacks to get 4.1.4 jupyterhub released today in a base-notebook image.
the desktop proxy appears to go into a lock screen if left alone.
lock screen never appears
lock screen appears if the system is left alone. or locked explicitly.
you can download and run the following docker container launch the desktop and leave it for a while.
this is the base docker container: dandiarchive/dandihub:latest
https://github.com/dandi/dandi-hub/blob/dandi/docker/Dockerfile#L40
Remote desktop running on a remote server and port forwarded to a local desktop starts, but clicking Desktop gives an error and the desktop does not start.
Clicking the desktop icon opens a remote desktop.
On a remote server:
docker build -t jupyter-remote-desktop-proxy .
docker run --rm -p 8888:8888 jupyter-remote-desktop-proxy
Locally:
Distributor ID: Ubuntu
Description: Ubuntu 21.04
Release: 21.04
Codename: hirsute
Local: OSX 13.4 Ventura
# paste output of `pip freeze` or `conda list` here
# jupyterhub_config.py
The image tests are failing, but the bash is too complex for me to really debug. I think it's ok for now.
Using git master (to test the unix socket support)
PR to follow - issue at https://github.com/jupyterhub/jupyter-remote-desktop-proxy/blob/main/jupyter_remote_desktop_proxy/__init__.py#L18
Xvnc server starts and is rendered through browser and able to
Traceback in docker logs
[W 2023-07-14 15:44:59.270 ServerApp] jupyter_server_proxy | extension failed loading with message: TypeError('expected str, bytes or os.PathLike object, not tuple')
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 356, in load_extension
extension.load_all_points(self.serverapp)
File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 228, in load_all_points
return [self.load_point(point_name, serverapp) for point_name in self.extension_points]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 228, in <listcomp>
return [self.load_point(point_name, serverapp) for point_name in self.extension_points]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 219, in load_point
return point.load(serverapp)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 147, in load
return loader(serverapp)
^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/jupyter_server_proxy/__init__.py", line 47, in _load_jupyter_server_extension
server_processes += get_entrypoint_server_processes(serverproxy_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/jupyter_server_proxy/config.py", line 100, in get_entrypoint_server_processes
server_process_config = entry_point.load()()
^^^^^^^^^^^^^^^^^^^^
File "/opt/install/jupyter_remote_desktop_proxy/__init__.py", line 24, in setup_desktop
with open(vncserver) as vncserver_file:
^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not tuple
git clone https://github.com/jupyterhub/jupyter-remote-desktop-proxy.git
cd jupyter-remote-desktop-proxy
docker build --no-cache -t $(whoami)/$(basename ${PWD}) .
docker run --rm -p 8888:8888 $(whoami)/$(basename ${PWD})
# Install TurboVNC (https://github.com/TurboVNC/turbovnc)
# ARG TURBOVNC_VERSION=2.2.6
# RUN wget -q "https://sourceforge.net/projects/turbovnc/files/${TURBOVNC_VERSION}/turbovnc_${TURBOVNC_VERSION}_amd64.deb/download" -O turbovnc.deb \
# && apt-get install -y -q ./turbovnc.deb \
# # remove light-locker to prevent screen lock
# && apt-get remove -y -q light-locker \
# && rm ./turbovnc.deb \
# && ln -s /opt/TurboVNC/bin/* /usr/local/bin/
I did the build and failed
Should build
Status: Downloaded newer image for jupyter/base-notebook:python-3.7.6
---> 43c995183265
Step 2/10 : USER root
---> Running in 7f701b6fa23c
Removing intermediate container 7f701b6fa23c
---> 205da6c2c11a
Step 3/10 : RUN apt-get -y update && apt-get install -y dbus-x11 firefox xfce4 xfce4-panel xfce4-session xfce4-settings xorg xubuntu-icon-theme
---> Running in 70736492727a
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Err:1 http://archive.ubuntu.com/ubuntu focal InRelease
At least one invalid signature was encountered.
Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
At least one invalid signature was encountered.
Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
At least one invalid signature was encountered.
Err:2 http://security.ubuntu.com/ubuntu focal-security InRelease
At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://archive.ubuntu.com/ubuntu focal InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu focal-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal-updates InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu focal-backports InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu focal-backports InRelease' is not signed.
W: GPG error: http://security.ubuntu.com/ubuntu focal-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.ubuntu.com/ubuntu focal-security InRelease' is not signed.
The command '/bin/bash -o pipefail -c apt-get -y update && apt-get install -y dbus-x11 firefox xfce4 xfce4-panel xfce4-session xfce4-settings xorg xubuntu-icon-theme' returned a non-zero code: 100
# paste output of `pip freeze` or `conda list` here
# jupyterhub_config.py
# paste relevant logs here, if any
I think it could be reasonable for us to build two separate docker images if we intend to support two VNC servers. This is a way to help ourselves test them as well.
Practically I'm thinking that we would allow a build arg to determine if the same Dockerfile builds to include TigerVNC or TurboVNC, and that we then build/push the image twice and varying the build arg.
Then, with such images in place, we could also do a simple smoke test for both TigerVNC and TurboVNC and resolve #68 in a relatively simple manner.
TurboVNC can be installed via apt
as described here. I've confirmed that the apt installation method seem to be providing us with up to date versions of TurboVNC.
vnc-server
build arg defaulting to tigervnc
, and refactor the Dockerfile to install TigerVNC towards the end, after everything else except copying of files from this repo to setup the Python package.apt
according to official docs-turbovnc
suffix to the tag-tigervnc
suffix to the tagThe https://mybinder.org/v2/gh/jupyterhub/jupyter-remote-desktop-proxy/HEAD?urlpath=desktop badge fails to launch a working server
You should end up at a Linux desktop on mybinder.org
https://mybinder.org/v2/gh/jupyterhub/jupyter-remote-desktop-proxy/HEAD?urlpath=desktop
The repo now contains breaking changes so the next release will be v2.0.0.
I think before release we should at least get the following open PRs to land:
I think also if we can't drive development of tests now, it won't get done - and specifically now before a release is when we would want to see tests go green as well. Due to that, I'd like to see us not settle for the above, but push to resolve:
By @goekce in #96 (comment)
I believe the command line arguments to tigervnc and turbovnc may be different. In #69 I had mentioned that
-xstartup
is not accepted by tigervnc:jupyter-remote-desktop-proxy/jupyter_remote_desktop_proxy/__init__.py
Lines 34 to 35 in 024ab7d
I searched for but could neither find any reference to
xstartup
in the source code of tigervnc nor in the documentation:https://github.com/search?q=repo%3ATigerVNC%2Ftigervnc%20xstartup&type=code
Further arguments must also be tested:
jupyter-remote-desktop-proxy/jupyter_remote_desktop_proxy/__init__.py
Lines 37 to 46 in 024ab7d
This is the reason why I think in the long term the arguments should be configurable by the user.
This project relies on websockify, and that typically needs to get installed via conda-forge with its pre-compiled stuff bundled there.
Due to that, having a conda-forge feedstock becomes extra relevant to simplify things downstream.
I am on Arch Linux. I installed my own TigerVNC, because the bundled TigerVNC segfaults when I start Firefox.
jupyter-remote-desktop-proxy picks the configuration based on whether the string TigerVNC
can be found in vncserver
:
The newest version of vncserver
does not include TigerVNC
as a string, which can lead to wrong configuration of the installed VNC server.
The workaround is to use tigervnc
as a string instead of TigerVNC
. I can happily prepare a PR, however I also saw that there is a discussion about which vncservers to support, so I wanted to confirm if my workaround makes sense.
I can include more details, if the error description is not clear.
A general remark about configurability:
The configuration can change between the software versions, so a more sustainable solution would be to make the command for starting the VNC session configurable. Currently the config is mostly hardcoded. Another argument that calls for a configuration capability: the newest vncserver
does not support -xstartup
argument used here:
jupyter-remote-desktop-proxy/jupyter_remote_desktop_proxy/__init__.py
Lines 34 to 35 in 024ab7d
I can open a new issue for this if needed.
We moved the upstream of that repo here, but there are more issues in that repo. We should transfer them. I'm not sure how.
EDIT: Link is https://github.com/yuvipanda/jupyter-desktop-server/issues
Ctrl-Alt-Del
is removed in #78
It's occasionally useful as an escape hatch so we should add it back, albeit in a menu instead of a top-level button
Do nothing
Users who do something strange in their environment and need an escape hatch. User with custom/alternative desktops other than XFCE where Ctrl-Alt-Del may be more useful.
Add it back somewhere
I've not been able to pinpoint the reason, but it seems like the initial attempt to connect to a VNCserver via websockets when visiting /desktop can sometimes fail. I suspect its a race condition that may make it never happen in some contexts, and often in others.
In #101 there is now a test that often but not always in our CI system on the first attempt, but not the second.
This repository is licensed under BSD-3-Clause.
However it bundles and distributes TigerVNC which is GPL-2.0:
https://github.com/jupyterhub/jupyter-remote-desktop-proxy/tree/v1.0/jupyter_remote_desktop_proxy/share/tigervnc
Either:
Users start a jupyterhub session on cluster nodes and open VNC Desktop.
This does not always work due to conflicts when users are on the same node.
__init__.py
/tmp/.X11-unix
vncserver
is running:
vncserver -list
TurboVNC sessions:
X DISPLAY # PROCESS ID NOVNC PROCESS ID
:1 2003660
If an another user wants to start a VNC Desktop on the same node it will not work.
Currently, there's no way to go back to the hub home page / control panel without modifying the URL. We should instead have a button in the UI that lets people go back home / to JupyterLab launcher.
This is a response to #51 extracted to a dedicated issue to have a more focused discussion without cluttering the PR.
I think having a minimal image as part of this project is great because it represents the functionality tied to jupyterhub that this project introduces, and being based on a jupyter/docker-stacks image we are able to focus on the specific things relevant to this project. I see it as a surgical addition that:
Anything beyond that is something I'm still reflecting on, which makes me very greatful to see an explicit policy about this right away @yuvipanda!!
From the policy section:
Policy for adding images
To reduce maintenance burden, we will only accept maintaining images here that
match the following criteria:
- They add an application with a popular scientific use case.
- The application is already package and maintained in
conda-forge
.Users are encouraged to build and maintain their own images where possible,
inheriting from the base image here for ease of use.
With a jupyterhub hat on, I'd like to think a bit extra before committing to maintaining anything beyond the minimal image.
I've brainstormed a few policy ideas below to help us discuss what we think makes sense.
Going beyond providing just a minimal desktop image maintaing a flora of images has a significant risk of being too taxing for the jupyterhub team. Most flora projects stop flourishing because unsustainable growth (helm/helm-charts, jupyterhub/oauthenticator, kubernetes in-tree software for third party components, maybe terraform also?).
I think this project receives the best value per maintenance by maintaining only one minimal image, or possibly one minimal and one advanced example. With that in mind I think we shouldn't consider maintaing a flora of images.
I think having a single advanced image example alongside the minimal desktop image could help this project become aware of and resolve likely challenges in real world situations. I think the coupling to some specific well used software like QGIS could be acceptable as it becomes a real world use case that helps us test and validate the project.
I've written this as "an example" as I don't think it would be in scope for this project to ensure it would be an image that works out well for direct use given various periphiary needs users may have. I also think its reasonable that we don't need to maintain a changelog if we introduce a breaking change to the advanced example image, but I think we should for the minimal image if we have an advanced image example in this repo.
I see this as a slim version of 2), where a advanced image example isn't provided in the repo, but documentation in the "explanation" category (https://diataxis.fr) to help create an advanced image is. Such explanation docs could include info about MIME types etc.
Beyond such explanation based docs (which is easier to maintain tha how-to docs), it would make perfect sense to link out to known externally maintained advanced images for practical examples.
Even slimmer than 3), where we don't commit to maintaining explanation based docs, but just link out to other projects.
To not introduce any image and to clear away the Dockerfile we have.
Commit 447bf86 breaks the use of TurboVNC.
The image should work whether TigerVNC or TurboVNC is used.
The image does not work when TurboVNC is used.
Build image with TurboVNC installed.
Something went wrong, connection is closed
.~/.vnc/<CONTAINER ID>:<DISPLAY NUMBER>.log
for details.See #47 (comment) for logs and additional information.
This project ships with TigerVNC, but has support for use with TurboVNC as well, and really any vncserver
binary:
If a vncserver executable is found in PATH it will be used, otherwise a bundled TightVNC server is run. You can use this to install vncserver with support for other features, for example the Dockerfile in this repository installs TurboVNC for improved OpenGL support.
Should we aim for something simpler to reduce complexity and ensure robustness on what we really can manage to test and maintain? I'm thinking that we could remove TigerVNC for example.
I'd love to use and contribute with maintenance of this project, but I struggle to understand enough about it to feel confident on how to install it etc.
Here are some points that i think could be addressed within the README:
websockify
is a required dependency, but that is not documented
pip install
of websockify doesn't seem to cut it, but conda install -c conda-forge
does. Perhaps that is why it is not listed under install_requires in setup.py?__init__.py
points to share/xstartup, which in turn does exec /usr/bin/dbus-launch xfce4-session
, making me think that the apt
packages dbus-x11
and xfce4-session
are strictly required.apt
packages are required?xfce4-session
is required, perhaps is xorg
(x11?) and other xfce4
packages strictly required as well?/panel
path?I think all projects benefit greatly from a technical overview as that allows anyone to better onboard and for example debug issues or contribute to maintenance in various ways.
Here are some notes that I've distilled from learning how this project works in order to help me debug and maintain things in this project. They should absolutely be refined further, but this is similar to the level of overview I think we should provide in the README, for example under a "How it works" heading like for jupyterhub-idle-culler.
/desktop-websockify/
Requests here are listened to by websockify. It terminates websocket traffic and proxies data to a VNC server. setup_websockify
returns a command to start websockify
, that in turn wraps startup of a vncserver, that in turn wraps startup of a desktop session via a passed xstartup
script. When websockify
wraps the startup of a vncserver, it intercepts the socket bind request to hijack listening on that port, and instead lets the vncserver listen to some other port it then will forward traffic to.
The purpose of websockify
is to bridge websocket traffic into a normal socket traffic.
At the most basic level, websockify just translates WebSockets traffic to normal socket traffic. Websockify accepts the WebSockets handshake, parses it, and then begins forwarding traffic between the client and the target in both directions.
The associated JupyterLab launcher entry redirects users to /desktop/, which in turn provides an index.html referencing a viewer.js that interacts via websockets with /desktop-websockify/.
The way jupyter-server-proxy proxies communication can be either via a unix socket or via a TCP port. Only TigerVNC supports listening to unix sockets though.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.