Error when creating spark session about jupyterlab-sparkmonitor HOT 6 CLOSED

This is a difficult error to debug; the kernel extension is complaining that the Scala Listener has no comms; this is usually because the listener wasn't able to find a Spark Instance.

One solution might be to restart your kernel and refresh the page when you encounter this. I've found that sometimes installing the extension on a kernel that was previously running can result in some wonky errors that are simply fixed by a simply stop and start of the kernel and a refresh of the page.

from jupyterlab-sparkmonitor.

I'm also having this problem. @samj1912 Figure anything out?

from jupyterlab-sparkmonitor.

I note it happens even when the extension is loaded directly from the notebook (in lieu of kernel initialization):

from jupyterlab-sparkmonitor.

Extension thinks its loaded:

from jupyterlab-sparkmonitor.

So my current read is that register_comm is begin called, but the self.target_func callback isn't being called before comm is being dereferenced in send.

from jupyterlab-sparkmonitor.

It can be worked around by checking if the frontend has already connected or not when using the comm object. It might be interesting to queue messages maybe ?

I suspect #18 is an other manifestation of this thread crash (but from the Spark listener side).

diff --git a/sparkmonitor/kernelextension.py b/sparkmonitor/kernelextension.py
index 6a94f5e..fbf3c04 100644
--- a/sparkmonitor/kernelextension.py
+++ b/sparkmonitor/kernelextension.py
@@ -41,6 +41,7 @@ class ScalaMonitor:
         ipython is the instance of ZMQInteractiveShell
         """
         self.ipython = ipython
+        self.comm = None

     def start(self):
         """Creates the socket thread and returns assigned port"""
@@ -53,7 +54,8 @@ class ScalaMonitor:

     def send(self, msg):
         """Send a message to the frontend"""
-        self.comm.send(msg)
+        if self.comm:
+            self.comm.send(msg)

     def handle_comm_message(self, msg):
         """Handle message received from frontend

from jupyterlab-sparkmonitor.