New hook point for messages / message transforms. about ipykernel HOT 14 CLOSED

Also, there are issues with threading. Since the ZMQDisplayPublisher is a global object, installing a message filter there naively will affect messages from everywhere. In the case of a context manager, it seems that the "correct" behavior is to only have the output from the block of code in the current thread. @minrk suggested making the hooks thread local attributes in the ZMQDisplayPublisher object (see https://docs.python.org/2/library/threading.html#threading.local), maybe something like

from threading import local
class ZMQDisplayPublisher(...):
    def __init__(...):
        self.threadlocal = local()

    def register_hook(self, hook):
        if not hasattr(self.threadlocal, 'hooks'):
            self.threadlocal.hooks = []
        self.threadlocal.hooks.append(hook)

from ipykernel.

Code to touch for making a display hook registration system includes the following.

• https://github.com/ipython/ipykernel/blob/master/ipykernel/displayhook.py - this is used to 'display' the last output of a computation
• https://github.com/ipython/ipykernel/blob/master/ipykernel/iostream.py - this is used to buffer and package messages for stdout/stderr
• https://github.com/ipython/ipykernel/blob/master/ipykernel/zmqshell.py#L56 - this is used for display_data messages.

@minrk, can you verify this is right?

from ipykernel.

Interestingly, since clear_output is implemented by just sending a clear_output message to the front end, this works too:

with output_widget.capture_messages():
    clear_output()

from ipykernel.

Yup, that should be right. Also to clarify, implementing this hook should need no changes in any other repos, but the intention is to use it in a new implementation of the OutputWidget downstream.

from ipykernel.

@dwillmer, @minrk: next steps:

We can get a handle on the display publisher with get_ipython().display_pub

• Make a context manager that will register a hook and store a list of display outputs. You may need to flush something when exiting the context manager to make sure you get all of the display messages.
• Make a widget that has such a context manager. It should take over any display_pub messages and store the messages in a list as its synced widget state. A clear_output message should clear the message queue (and probably doesn't need to be sent, since the empty message queue will be synced).
• Make a front end widget that will render the display pub message queue (this will likely involve using a rendermime instance that is passed in...)

@minrk - can you remind us what needs to be done for capturing stdout/stderr?

from ipykernel.

@jasongrout - is this ^^ what you're expecting?

The print calls go to the normal output area, but any display_data messages within the new context manager are sent to the custom widget area.

Clearly we'll change the name from MessageWidget ;) just wanted some early feedback!

from ipykernel.

Yep, that's the idea. Thanks!

Things to still do:

• hook into the stdout/stderr (we also want to capture stdout). We have to write these extension points as well.
• make sure a clear_output() message clears the messages

And when I finish the widgets in the new notebook, actually render the messages sent to the message widget.

from ipykernel.

Just checking, the hook unregisters itself when the context manager exits, right? (that's not illustrated in your screenshot, but it should work that way)

from ipykernel.

yep, it does, i'll post an updated screenshot in a bit.

from ipykernel.

For stdout/err, the simple thing is to swap out sys.stdout/err to an io.StringIO object, but that won't have the same threadlocal behavior the hooks now have. To preserve that, some amount of threadlocal things will need to be added to the IOStream code, so that writes go to the short-circuit buffer rather than the default.

from ipykernel.

@dwillmer - no need to post a new screenshot for the exit status. I trust it's there.

from ipykernel.

Update:

• Using with widget.capture(): instead of 'with widget:`.
• capture() takes an optional positional arg for the message type, so you can intercept different messages in each context.
• Moved context manager to ipykernel, left widget code in ipywidgets

from ipykernel.

I think this issue is solved with #115. Let's document the widget work on an issue in the ipywidgets repo. @dwillmer, can you open a work-in-progress PR over there?

from ipykernel.

@minrk or @dwillmer, if this issue is solved by #115, can you close this? Apparently I don't have permissions to close issues on this repo.

from ipykernel.