manzt / anywidget Goto Github PK

https://anywidget.dev/en/jupyter-widgets-the-good-parts/#2-custom-messages explains how to send custom messages to the frontend but is there an equivalent to send data to the backend? I know traitlets that are synchronized can be used for this but I wonder whether there is a way to send messages that I don't need to synchronize back to the frontend.

Hi, I would like to try to use deck.gl in anywidget. Would you have any advice on how to go about doing this?

I moved deployment of my anywidget-based library from local to CI and forgot to build and include the relevant JS files. I got a confusing error mentioning regexes:

Obviously the issue here is on my side, but if it's possible to have a more descriptive error here, that would be great!

Goal: Consolidate common patterns for communicating between JS and Python.

Why: Someone comes to anywidget and wants to use framework X. How do you make sure they can get the information they need and be productive as quickly as possible.

How: Perhaps some additional utility libraries (a la @anywidget/react) that provide the minimal boilerplate to make synchronizing state between JS & Python a breeze. @xnought has a really nice svelte implementation in https://github.com/xnought/svelte-store-anywidget, for example. I guess the question is how we should share this kind of code:

1. Official libraries (i.e.,@anywidget/react, @anywidget/svelte)
2. Snippets or recipes that can be copied and pasted
3. An awesome-anywidget collection of cool anywidget projects

Out of the above options, I think I'd prefer 1. Since many coming to anywidget for the first time might not be familiar with front-end tools / JS, having an official "one-stop-shop" for your anywidget needs would be the best experience IMO. Perhaps we could even have some templates like popular frontend frameworks to bootstrap any anywidget project:

npm create anywidget@latest my_widget --tempate=react

Biggest question on my mind is maintenance. I think the client-specific libraries shouldn't be too big of a lift, but the templates would be really tricky to keep up to date (I do not want cookiecutters part 2).

Musing on project philosophy: One thing I like about anywidget in its current state is that the starting point is ESM. For beginners, this means you start by handwriting JS code that the browser understands and thereby get a better understanding and demystify of the front end. Front-end frameworks introduce magic (like bundlers) that obscure this fact, meaning the code you write is NOT what is executed in the browser, which can lead to misconceptions / confusion about front end.

Ideally one would only reach for a framework if the abstractions it provided outweighed the cost (often not a consideration), but the reality is that many come to the front-end for the first time having heard about framework X and now they want to use X for their first project.

IMO widgets should primarily utilize vanilla JS, with the majority of logic to connect Jupyter Widget's backbone model with an existing API. If a widget is sufficiently complex and necessitates a front-end framework, it likely signifies a more general front-end library that would be better off in a separate package.

With that said, I think folks will still come to anywidget and want to use framework X, so I want to make sure they avoid common pitfalls and are productive quickly.

Is there support for adding a webworker script tag for the component? I am looking to add a tag like this <script src="https://unpkg.com/@hpcc-js/[email protected]/dist/graphviz.umd.js" type="javascript/worker"></script> to implement a d3-graphviz component.

Sorry, if its obvious, but I am not able to figure out, how I could load js-libraries from my filesystem (not CDN, but offline from my local-disk) in "_esm" script-base.

Can you please let me know how to do it ?

Now that Deno allows for async displays, we should only need to "just-in-time" initialize the comm when we render the widget.

-- let model = await widget(config);
++ let model = widget(config);

The change would be to keep a promise for the comm and only await it when we need to display the widget.

I am interested in creating a map widget based on the leaflet javascript library. Here is an example. I am not a JavaScript expert. Do you know why the map widget does not show up?

https://leafletjs.com/examples/quick-start/example-overlays.html

import anywidget
import traitlets

class MapWidget(anywidget.AnyWidget):
    # Widget front-end JavaScript code
    _esm = """

    import * as L from "https://unpkg.com/[email protected]/dist/leaflet.js"
    export function render(view) {

        // create a div element
        const container = document.createElement("div");
        container.id = "map";
        container.style.height = "300px";

        // create a Leaflet map
        const map = L.map(container).setView([51.505, -0.09], 13);
        L.tileLayer("https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png", {
        attribution:
            'Map data &copy; <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors',
        maxZoom: 18,
        }).addTo(map);

        canvas.appendChild(map)
        view.el.appendChild(canvas);
        }
    """
MapWidget()

I have a widget like this

import duckdb
import anywidget
import pyarrow as pa

class CustomWidget(anywidget.AnyWidget):
    _esm = """
    import * as arrow from 'https://cdn.jsdelivr.net/npm/[email protected]/+esm'
    
    export function render(view) {
        view.el.classList.add("custom-widget");

        view.model.on("msg:custom", msg => {
            console.log("received message", msg);
            
            if (msg.type === "result") {
                const table = arrow.tableFromIPC(msg.result);
                console.log("table", table);
            }
        });
        
        const btn = Object.assign(document.createElement("button"), { innerText: "Send message" });
        btn.addEventListener("click", () => {
            const msg = {type: "arrow", sql: "SELECT count(*) FROM df"};
            console.log("sending message", msg);
            view.model.send(msg);
        });
        view.el.appendChild(btn);
    }
    """
    
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.on_msg(self._handle_custom_msg)

    def _handle_custom_msg(self, data, buffers):
        print(f"{data=}, {buffers=}")

        if data["type"] == "arrow":
            result = duckdb.query(data["sql"]).arrow()
            sink = pa.BufferOutputStream()
            with pa.ipc.new_stream(sink, result.schema) as writer:
                writer.write(result)
            buf = sink.getvalue()

            msg = {"type": "result", "result": buf.to_pybytes()}
            print(f"{msg=}")
            self.send(msg)

widget = CustomWidget()
widget

df if any pandas dataframe.

I would expect the client to get binary data but instead, the messages are strings and so the client doesn't parse the Arrow IPC correctly.

This solution is no longer possible since 0.5.0 because the raw AnyView object is no longer exposed to the render() callback (it's now being filtered by extract_context).

Because of this, I see no good way to build a container widget (my current effort). I would like to use AnyWidget to bootstrap my project, but without access to the lower level ipywidgets stuff, I don't think it's feasible.

I can happily submit a PR to expose the AnyView object. My local version is exposing it as another property on the result of extract_context: e.g. return { model, el: view.el, view};

@manzt - thank you for this python package. It looks like it will be so much easier to work with than cookiecutter - which on a fresh install for jupyterlab, fails to run to completion.
Alas, I have a small problem with your CounterWidget demo. I hope you can help.
I use the code you provide here - verbatim, as best I can tell.
On executing this line

w = CounterWidget()

the javascript console shows:

Any suggestions? Any more information I can supply?

@manzt and i have been experimenting with anywidget + @carbonplan/maps in https://github.com/manzt/carbonplan. The notebook works perfectly fine when connected to a live kernel. however, i am facing an issue with persisting the widget's state within the notebook and having external services like nbviewer render the initial state of the widget.

to illustrate the problem, i have shared a sample notebook at https://nbviewer.org/gist/andersy005/f8bca6c542135a75ab3e11203eada3a1. as you can observe, the last cell of the notebook is not being rendered.

any guidance on how to resolve this issue?

Cc @katamartin

Hi,

right now, only a render method can be defined, which will be triggered when a view is created. It may also be useful to perform operations when a widget gets created (not a view), and when a widget gets destroyed. This allows for widgets similar to the one powering jslink etc.
Original questions that triggered this:
widgetti/ipyreact#44
cc @giswqs

Hi,

I'm having trouble trying to save a widget created with anywidget to an HTML file with ipywidgets.embed.embed_minimal_html().

Here is a simple example:

import anywidget
import traitlets


class CounterWidget(anywidget.AnyWidget):
    _esm = """
    export function render({ model, el }) {
      let getCount = () => model.get("count");
      let button = document.createElement("button");
      button.classList.add("counter-button");
      button.innerHTML = `count is ${getCount()}`;
      button.addEventListener("click", () => {
        model.set("count", getCount() + 1);
        model.save_changes();
      });
      model.on("change:count", () => {
        button.innerHTML = `count is ${getCount()}`;
      });
      el.appendChild(button);
    }
    """
    _css = """
    .counter-button { background-color: #ea580c; }
    .counter-button:hover { background-color: #9a3412; }
    """
    count = traitlets.Int(0).tag(sync=True)


counter = CounterWidget()
counter.count = 42
counter

from ipywidgets.embed import embed_minimal_html

embed_minimal_html("/tmp/out.html", views=[counter])

Once opened in a browser, the resulting HTML file throws a JavaScript error:

Class null not found in module @jupyter-widgets/[email protected]

I'm not sure if it is a problem related to anywidget or if I don't use this feature correctly.
I'm using anywidget 0.6.5.

Many thanks for all your work !

I'm fairly confident I'm encountering CORS problems due to the fact that the URL created by by URL.createObjectURL and executed by import() returned has a different host. Specifically, it is of the form blob:https://example.com/.... If you make a request for a resource from the Jupyter server in my widget I get a CORS error. Presumably because the host of my widget code has a blob: prefixed host name.

It's hard to be 100% sure of this since window.location does not show the blob: prefix. However, I've convinced myself by observing that the CORS error does not show up if I run the same query in the developer console.

I am exploring the maplibre library with anywidget. The following code works perfectly with Colab. Jupyter Lab can display the map, but the map size is always fixed no matter what div.style.width and div.style.height values. VS Code throws errors. Any advice?

import anywidget
import traitlets

class MapWidget(anywidget.AnyWidget):
    _esm = """
    function loadScript(src) {
      return new Promise((resolve, reject) => {
        let script = Object.assign(document.createElement("script"), {
          type: "text/javascript",
          async: true,
          src: src,
        });
        script.addEventListener("load", resolve);
        script.addEventListener("error", reject);
        document.body.appendChild(script);
      });
    };
    
    await loadScript("https://unpkg.com/[email protected]/dist/maplibre-gl.js");
    
    export function render(view) {
      const div = document.createElement("div");
      div.style.width = "100%";
      div.style.height = "500px";
      
        var map = new maplibregl.Map({
        container: div,
        style: 'https://demotiles.maplibre.org/style.json', // stylesheet location
        center: [0, 20], // starting position [lng, lat]
        zoom: 2 // starting zoom
        });
      view.el.appendChild(div);
    }
    """
    # make sure to include styles
    _css = "https://unpkg.com/[email protected]/dist/maplibre-gl.css"
    token = traitlets.Unicode().tag(sync=True)

m = MapWidget()
m

Colab:

Jupyter Lab:

VS Code:

Hard to know if this is something that should be solved upstream in ipywidgets, but it is certainly something we can try to resolve for anywidget users. I ran into this issue whilst re-implementing the low-level ipywidgets.Widget base class which anywidget.AnyWidget inherits from.

Description

Third-party widgets don't work in Google Colab for ipywidgets v8. I've seen various issues related to core ipywidgets not working in Colab, but that has been resolved (read: ipywidgets.IntSlider works in v8 but custom widgets do not) and I haven't seen any other solution except "install an older version of ipywidgets".

ipywidgets v7.7.1 (default in Google Colab)

Everything works appropriately.

ipywidgets v8.0.1 (manually installed into notebook)

Fails to find third-party widget. No output, obscure error in the browser console.

Explanation

I haven't seen this documented anywhere, so I'll do my best to describe the issue to my understanding. Of course Google Colab kernel is closed source, so I can't easily explore the source code to see how objects are displayed like in Jupyter. (read: I dug around in Colab notebook for a couple of hours trying to reverse engineer what the required protocol is.)

To my underestanding, the core issue is a related to a change in the protocols used to display Jupyter Widgets in notebooks from ipywidgets v7 to v8.

• v7 uses the _ipython_display_ method (which doesn't return anything but as a side effect displays the object with IPython.display)
• v8 uses the _repr_mimebundle_ method which returns a dictionary of multiple formats keyed by mime-type and then defers to the kernel to display the appropriate mime-type

You can learn more about what each of these methods do in the IPython docs, but the important caveat is _ipython_display_ takes the highest priority of all _repr_* methods. "If [_ipython_display_] is defined, all other display methods are ignored.". It seems that there were some issues with the _ipython_display_ protocol for some built-in widgets which act as containers for other widgets (e.g., ipywidgets.Output), which lead to the use of _repr_mimebundle_ in the latest version.

For whatever reason, Google Colab only supports rendering core ipywidgets with the new _repr_mimebundle_ protocol, not third-party widgets. However, manually calling IPython.display from within _ipython_display_ works for third-party widgets seems to work?

Solution

I was able to get anywidget or any other third-party widgets to render in Google Colab by adding a blanket polymorphic _ipython_display_ method. I.e., the following will work in Google Colab for ipywidgets v7 and v8:

import anywidget
import ipywidgets

class PatchedAnyWidget(anywidget.AnyWidget):
    _esm = "export function render(view) { view.el.innerText = 'Hello, world'; }"

    def _ipython_display(self, **kwargs):
        if hasatter(super(), "_ipython_display_"):
            # ipywidgets v7
            super()._ipython_display_(**kwargs)
        else:
            from IPython.display import display
            
            # ipywidgets v8
            data = self._repr_mimebundle__(**kwargs)
            display(data, raw=True)

My main question is whether we should try to patch _ipython_display_ only in Google Colab (i.e., conditionally with if "colab" in sys.modules) or across all environments. I don't fully understand the reasoning from the ipywidgets core team for preferring _repr_mimebundle_ but importantly if added _ipython_display_ to anywidget.AnyWidget we would revert the changes since the latter protocol takes precedence.

First, many thanks for the creation of anywidget. I wanted to create a Jupyter Widget for a while but found the documentation quite intimidating. With anywidget I have been able to create a working proof of concept very rapidly.

I've got one question, though: if I create a widget by running a notebook cell (for example the CounterWidget given in the documentation) everything works fine and the widget is displayed. But if I close the notebook and reopen it (I work with VSCode), the widget is not displayed anymore but is replaced by a textual representation : CounterWidget(value=60)

Is it the normal behavior ? Is there a way to make widget output "persistent" ?

Many thanks again for your work.

Using the example from https://deno.land/x/anywidget, I see this error:

Error: Module anywidget, version 0.0.1 is not registered, however,         0.6.5 is
    at f.loadClass (134.402424ef4079078b2e0e.js?v=402424ef4079078b2e0e:1:74976)
    at f.loadModelClass (150.1a6d6a3a0542a41bec5a.js?v=1a6d6a3a0542a41bec5a:1:10729)
    at f._make_model (150.1a6d6a3a0542a41bec5a.js?v=1a6d6a3a0542a41bec5a:1:7517)
    at f.new_model (150.1a6d6a3a0542a41bec5a.js?v=1a6d6a3a0542a41bec5a:1:5137)
    at f.handle_comm_open (150.1a6d6a3a0542a41bec5a.js?v=1a6d6a3a0542a41bec5a:1:3894)
    at _handleCommOpen (134.402424ef4079078b2e0e.js?v=402424ef4079078b2e0e:1:73392)
    at y._handleCommOpen (jlab_core.be6103fe6f6cc2c18378.js?v=be6103fe6f6cc2c18378:1:1237749)
    at async y._handleMessage (jlab_core.be6103fe6f6cc2c18378.js?v=be6103fe6f6cc2c18378:1:1239739)

It's not clear that the ultimate issue here is in Anywidget, but I figured it would be ok to start with an issue here. I'm making lonboard as a fast map rendering library and widget. It would be nice to support standalone HTML export and nbconvert conversion to HTML. When I create a minimal map, e.g. with:

import geopandas as gpd
from lonboard import viz
import ipywidgets.embed

gdf = gpd.read_file(gpd.datasets.get_path('naturalearth_cities'))

map_ = viz(gdf, radius_min_pixels=10)
map_

ipywidgets.embed.embed_minimal_html('minimal.html', views=[map_], drop_defaults=False)

The map renders fine within Jupyter Notebook:

But loading the exported HTML fails with

TypeError: Cannot read properties of undefined (reading 'prototype')
    at enhancePointerEventInput (cd357310-fde6-48b7-a3a9-c8ad15fa6425:65439:41)
    at cd357310-fde6-48b7-a3a9-c8ad15fa6425:65475:1

That's specifically coming from a bundled library that's a dependency of mine (via deckgl).

// node_modules/mjolnir.js/dist/esm/utils/hammer.browser.js
var hammerjs = __toESM(require_hammer());

...

// node_modules/mjolnir.js/dist/esm/utils/hammer.browser.js
enhancePointerEventInput(hammerjs.PointerEventInput);

...

function enhancePointerEventInput(PointerEventInput2) {
  const oldHandler = PointerEventInput2.prototype.handler;
  PointerEventInput2.prototype.handler = function handler(ev) {
    const store = this.store;
    if (ev.button > 0 && ev.type === "pointerdown") {
      if (!some(store, (e) => e.pointerId === ev.pointerId)) {
        store.push(ev);
      }
    }
    oldHandler.call(this, ev);
  };
}

It's weird that this works fine in the notebook but not standalone. My hunch is that there's a difference in how the JS gets run inside the notebook vs as a standalone file.

If we look at what ipywidgets.embed.embed_minimal_html does, it's described here and essentially creates:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>IPyWidget export</title>
</head>
<body>


<!-- Load require.js. Delete this if your page already loads require.js -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@jupyter-widgets/html-manager@^1.0.1/dist/embed-amd.js" crossorigin="anonymous"></script>

<script type="application/vnd.jupyter.widget-state+json">
{
  "version_major": 2,
  "version_minor": 0,
...

So it uses standard script tags for require.js and @jupyter-widgets/html-manager as well as for the custom widget state. I imagine there must be some difference in the above generated HTML that means the anywidget code isn't getting run as ESM...? (I'm definitely not an expert on this part of JS).

As repro, I'm attaching a zipfile with minimal.ipynb that has the same Python code as above, and minimal.html with the export. This is using lonboard 0.2.0 but with minify set to false so the exported HTML can be somewhat readable.

minimal.zip

Hi @manzt!
I'm diving into some more anywidget ideas, and right now considering how to structure the folder layout.
Version A has a src/ipymafs/ layout, while version B has a src/ipymafs/static layout.
Also, should it be static or _static?

Is there a convention we can follow?
It would be great if we could even come up with a Best Practice that can be promoted over the whole anywidget ecosystem.
By promoting a specific structure, we could potentially minimize confusion and bugs, enhancing coherence among all anywidget projects.

A

📁 Project Root (ipymafs)
┣━━ 📂 .venv
┃   ┗━━ ... (content of .venv folder)
┣━━ 📂 node_modules
┃   ┗━━ ... (content of node_modules folder)
┣━━ 📂 js
┃   ┗━━ 📄 bezier.jsx
┃       📄 ellipse.jsx
┃       📄 line.jsx
┣━━ 📂 src
┃   ┗━━ 📂 ipymafs
┃       ┣━━ 📄 __init__.py
┃       ┣━━ 📄 bezier.css
┃       ┣━━ 📄 bezier.js
┃       ┣━━ 📄 ellipse.css
┃       ┣━━ 📄 line.js
┃       ┣━━ 📄 ellipse.js
┃       ┗━━ 📄 line.css
┣━━ 📄 README.md
┣━━ 📄 pyproject.toml
┣━━ 📄 package.json
┣━━ 📄 package-lock.json
┣━━ 📄 example_ellipse.ipynb
┣━━ 📄 example_bezier.ipynb
┗━━ 📄 example_line.ipynb

B)

📁 Project Root (ipymafs)
┣━━ 📂 .venv
┃   ┗━━ ... (content of .venv folder)
┣━━ 📂 node_modules
┃   ┗━━ ... (content of node_modules folder)
┣━━ 📂 js
┃   ┗━━ 📄 bezier.jsx
┃       📄 ellipse.jsx
┃       📄 line.jsx
┣━━ 📂 src
┃   ┗━━ 📂 ipymafs
┃       ┣━━ 📄 __init__.py
┃       ┗━━ 📂 static
┃           ┣━━ 📄 bezier.css
┃           ┣━━ 📄 bezier.js
┃           ┣━━ 📄 ellipse.css
┃           ┣━━ 📄 line.js
┃           ┣━━ 📄 ellipse.js
┃           ┗━━ 📄 line.css
┣━━ 📄 README.md
┣━━ 📄 pyproject.toml
┣━━ 📄 package.json
┣━━ 📄 package-lock.json
┣━━ 📄 example_ellipse.ipynb
┣━━ 📄 example_bezier.ipynb
┗━━ 📄 example_line.ipynb

I had a pretty tough time debugging an issue that resulted from some bad code that was generated during a build step and which got executed at "import-time" rather than when rendering. This was difficult to figure out because ipywidgets simply reports something like failed to render model id <the-id> without logging the original error. This could be mitigated if anywidget reported errors that arose when importing the ESM module.

I've just made an experiment to see if I can run a minimal react app with Anywidget+Vite

export default function App() {
  return <h1>Hello Anywidget + Vite!</h1>;
}

but I get this error: ReferenceError: process is not defined 👇👇👇

Steps to reproduce

I've followed the official vite tutorial and the tutorial from the anywidget docs in order to set up a sample project at https://github.com/Octoframes/anywidget-react-vite-test.

The setup process ran without any errors, only displaying the widget does not work.
Here are the steps that I did:

1. npm create vite@latest

• Project name: .
• Select a framework: React
• Select a variant: JavaScript (note that there's also JavaScript+SWC, but I did not choose that)

2. npm install
3. npm run dev
4. Minify the vite default template till the react app is only <h1>Hello Anywidget + Vite!</h1>
5. Add the following files:

anywidget-react-vite-test/
├── pyproject.toml
├── hello_widget/
│  └── __init__.py
└── hello.ipynb

6. Change vite.config.js to

// vite.config.js
import { defineConfig } from "vite";
import react from '@vitejs/plugin-react'

export default defineConfig({
	plugins: [react()],
	build: {
		outDir: "hello_widget/static",
		lib: {
			entry: ["src/main.jsx"],
			formats: ["es"],
		},
	},
});

7. npm run build , which adds

anywidget-react-vite-test/
├── pyproject.toml
├── hello_widget/
│  └── __init__.py
│  └── static/
+       └── main.js
└── hello.ipynb

8. Setup python

• python3.11 -m venv .venv && source .venv/bin/activate
• pip install "anywidget[dev]"
• pip install jupyterlab

9. Run Jupyter Lab and execute the cell. Now the process is not defined error will show.

If you currently have some bandwidth, I'd be curious to hear your thoughts on this @manzt @maartenbreddels ✨

I get the following error:
jan-hendrik@JanHendriksAir okapi4 % cd js
jan-hendrik@JanHendriksAir js % npm install
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:^

package.json

"dependencies": {
"@anywidget/react": "workspace^",
"react": "^18.2.0",
"react-dom": "^18.2.0"
},

I'm really excited that a project like anywidget exists. I myself worked on making bidirectional JS <> Python communications works across Colab / Jupyter but was missing VS Code support.

It looks your project support all backends (VSCode, Colab,...), so I was trying to migrate my current code to anywidget. However, I didn't found a straightforward way to call Python from Javascript and get the result.

Currently, a hack is to use model.set("some_attribute", "value") on an attribute that is decorated with @traitlets.validate('some_attribute'). The validate Python code is executed when some_attribute is changed from JS. However this is boilerplate, limited and hard to use.

It would be nice if there was a native model.call function, for example with signature:

async function call(function_name, args, kwargs)

This would make it much easier to call Python code from the widget, like:

In js:

out = await model.call("my_function", [], {});  # args, kwargs

In Python:

class MyWidget(anywidget.AnyWidget):

  def my_function(self, *args, **kwargs):
    return ...

Input/output could be json.

This was my implementation (that only support Colab/Jupyter): https://github.com/google/etils/blob/main/etils/ecolab/pyjs_com/py_js_com.js
It would be awesome if anywidget was having something similar.

One concrete use-case for example is an interactive variable inspector: https://etils.readthedocs.io/en/latest/ecolab.html#inspect-any-python-objects

Let me take this opportunity to express my heartfelt thanks for this package. This is such a pain-reliever.. Never thought it could get so smooth to build a totally custom widget.

Right now, to style any of the existing ipywidgets too, I usually load tailwind-css to avoid writing vanilla-css.

Thought adding it as a default offering with "anywidget" bundle could make it more approachable to the user-base.

And thank you so much once again, even if you don't care my request.

Just figured out how we can observe traitlets using anywidget+react 🎉
Therefore, I want to share this minimal example.
Hope this is useful as a reference.

import anywidget
from traitlets import Int, Unicode, observe
import pathlib

class Widget2(anywidget.AnyWidget):
    label = Unicode("Click me").tag(sync=True)
    count = Int(0).tag(sync=True)

    @observe("count")
    def _observe_count(self, change):
        print(f"Old value: {change['old']}")
        print(f"New value: {change['new']}")
        print(f"--------------------------")

    _esm = pathlib.Path.cwd() / "src" / "myprojectname" / "static" / "testing_observe.js"
            
w2 = Widget2()
w2

// this is at "js/testing_observe.jsx"
import * as React from "react";
import { createRender, useModelState } from "@anywidget/react";

export const render = createRender(() => {
  const [label] = useModelState("label");
  let [count, setCount] = useModelState("count");
  function handleClick() {
    console.log("button clicked");
    setCount(count + 1);
  }

  return (
    <div>
      <button onClick={handleClick}>
        {count == 0 && label}
        {count} times clicked
      </button>
    </div>
  );
});

and for reference, here is how the same code would look like in ipyreact:

from traitlets import Int, Unicode, observe
import ipyreact

class Example(ipyreact.ReactWidget):
    label = Unicode("Click me").tag(sync=True)
    count = Int(0).tag(sync=True)

    @observe("count")
    def _observe_count(self, change):
        print(f"Old value: {change['old']}")
        print(f"New value: {change['new']}")
        print(f"--------------------------")

    _esm = """
    import * as React from "react";

    export default function ({ label, count, set_count }) {
    function handleClick() {
        console.log("button clicked");
        set_count(count + 1);
    }
    return (
        <div>
        <button onClick={handleClick}>
            {count == 0 && label}
            {count} times clicked
        </button>
        </div>
    );
    }
    """


Example()

We added in widget decorator in #126.

This is still a little clunky to use as it adds several imports:

from anywidget.experimental import widget
from psygnal import evented
from dataclasses import dataclass

@widget(esm="index.js")
@evented
@dataclass
class Counter:
    value: int = 0

It would be nice if widget could be smarter and reduced to something like:

@widget(esm="index.js")
class Counter:
    value: int = 0

but the point of the descriptor API to to allow flexibility for modeling/serializing data and emitting events, so I'm not sure I want this to be too magical/opinionated.

I guess the use case I am really interested in is prototyping and avoiding imports. I'd like to have one import and get away with type-hints only if possible. Some prior art, but maybe we can just add a dataclass API of our own like pydantic.

from anywidget.experimental import dataclass

@dataclass(esm="index.js")
class Counter:
    value: int = 0

Not sure if you have any thoughts @tlambert03

I really like what I see so far from anywidget. A couple of questions that would probably be addressed by a roadmap:

1. Where do you see anywidget fitting into the ipywidgets ecosystem, alongside related projects like dash, bokeh, holoviz?
2. Does anywidget have a story for packaging your widget up into a python package?
3. Does anywidget have a story for typescript?
4. How can anywidget help with testing? I would love a guide and examples for how to test the js/ts of a widget without having to restart/reevaluate the widget cell each time I make a change. Some examples of tests and explanations of why they work to avoid that workflow would get me out of the whole pattern.

What else are you planning to build into anywidget?

Keep up the good work, Paddy.

Following the tutorial from https://anywidget.dev/en/jupyter-widgets-the-good-parts/ (installed with pip install anywidget
) will lead to an error because not all dependencies are installed:
ModuleNotFoundError: No module named 'watchfiles'
This can be easily fixed:

Maybe one can add the watchfiles to the pyproject.toml dependency file?
Or one can mention in the https://anywidget.dev/en/getting-started/ page, that one should install pip install --upgrade "anywidget[dev]"

Thanks for this great package!

Hi,

I recently encountered an issue with my pyobsplot widget on Windows (see juba/pyobsplot#17). It seems that when the _esm or _css elements of a widget contain some special characters, Windows tries to read them with the default cp1252 encoding and fails, generating an UnicodeDecodeError. The problem doesn't appear on Linux (and I guess on MacOS).

After fiddling a bit, it seems that adding an encoding argument to read_text() in FileContents.__str__ solves the issue:

def __str__(self) -> str:
    if self._contents is None:
        self._contents = self._path.read_text(encoding="utf8")
    return self._contents

I'm really not sure it is a correct way to fix this, but if you think it is I can do a proper PR if you like.

Thanks !

v0.6 dropped support for Python 3.7. Perhaps it was a bit too aggressive? My motivation for dropping support is because 3.7 is now officially EOL, but keeping support longer shouldn't be super complicated. Maybe a good plan would be to defer to whatever ipywidgets is supporting?

@keller-mark raises a good point in vitessce/vitessce-python#208 (comment).

Both anywidget and the Jupyter cookiecutter templates claim that widgets will be compatible with multiple notebook environments. However, neither the cookiecutters nor anywidget have automated end-to-end tests to ensure which check compatibility. Each test JS and Python code separately.

It will be a challenge, but I'd like to implemented automated e2e tests for anywidget in its "target" environments. First, we should be explicit about which environments to test. So far, I've identified:

• classic Jupyter Notebooks
• JupyterLab
• VS Code
• Google Colab
• nbconvert (HTML / reveal.js outputs)

An initial foreseen challenge is for the environments which rely on a CDN to load the widget code (VS Code, Google Colab, nbconvert). I'm not aware of how to "test" that widget code actually is loaded and runs in these prior to making a release to NPM because the code is loaded from a CDN like unpkg.com using the current version.

Hi,
Can i use anywidget to interact with jupyter lab components?

Ive been trying the following as my starting point, but my hunch is this isn't possible or the wrong way to go about it.
(3.6.1 is the notebook version i'm currently using)

import anywidget
class ExampleWidget(anywidget.AnyWidget):
    # anywidget, required #
    _esm = """
    import { JupyterFrontEnd, JupyterFrontEndPlugin } from "https://esm.sh/@jupyterlab/[email protected]";

    """
ExampleWidget()

i get this error

Failed to create view for 'AnyView' from module 'anywidget' with model 'AnyModel' from module 'anywidget'
SyntaxError: The requested module '/v132/[email protected]/es2022/crypto-browserify.mjs' does not provide an export named 'webcrypto'

It would be great to have a simple example of using anywidget to interact with Jupyter itself. I think that is something others will find useful.

Thanks.

Hi

I am a user of and contributor to HoloViz Panel. One of the reasons I prefer Panel for my data apps is that it works with ipywidgets. I would like Panel and Anywidget to work together to enable users to easily develop powerful data apps using Anywidget and derived widgets like mapwidget.

Unfortunately it currently does not work.

I don't know which python package causes the issue. So I've cross posted here bokeh/ipywidgets_bokeh#89.

Reproducible Example

pip install panel==0.14.4 ipywidgets-bokeh==1.3.0 anywidget==0.2.0

script.py

import anywidget
import traitlets

import panel as pn

pn.extension("ipywidgets")

class CounterWidget(anywidget.AnyWidget):
    _esm = """
    export function render(view) {
      let getCount = () => view.model.get("count");
      let button = document.createElement("button");
      button.classList.add("counter-button");
      button.innerHTML = `count is ${getCount()}`;
      button.addEventListener("click", () => {
        view.model.set("count", getCount() + 1);
        view.model.save_changes();
      });
      view.model.on("change:count", () => {
        button.innerHTML = `count is ${getCount()}`;
      });
      view.el.appendChild(button);
    }
    """
    _css="""
    .counter-button { background-color: #ea580c; }
    .counter-button:hover { background-color: #9a3412; }
    """
    count = traitlets.Int(0).tag(sync=True)

counter = CounterWidget()
pn.panel(counter).servable()

panel serve script.py

Open http://localhost:5006/script and see the exception in the browser console.

Class null not found in module @jupyter-widgets/[email protected]

Any help would be appreciated. Thanks.

I'm giving a try to this very new feature https://github.com/manzt/anywidget/tree/main/packages/svelte but my example fails with count having NAN value.
The svelte code is exactly the same as in the readme. The python code is like below

class SvelteCounter(anywidget.AnyWidget):
    _esm = "http://localhost:5173/dist/index.js?anywidget"
    count = traitlets.Int().tag(sync=True)
    
counter = SvelteCounter()
counter

The MimeBundeDescriptor introduces a more library agnostic connection between python model and JS view, but currently doesn't support two important features from ipywidgets.Widget. First a mechanism to both send and receive messages from the front end. The latter could be simple enough to implement with a special method (akin to _anywidget_get_state):

class Foo:
  _repr_mimebundle_ = MimeBundleDescriptor()
  def _anywidget_handle_custom_msg(self, msg):
      ...

But I'm having trouble thinking of a way to allow for custom messages to be sent. Right now this can be accomplished with:

class Foo:
  _repr_mimebundle_ = MimeBundleDescriptor()

Foo()._repr_mimebundle_._comm.send(...) 
# probably do not want to expose Comm directly bc custom messages have a particular format

but I'd be ok with adding a send method to ReprMimeBundle directly (like send_state):

class Foo:
  _repr_mimebundle_ = MimeBundleDescriptor()

Foo()._repr_mimebundle_.send(contents, buffers
# alias for ReprMimeBundle._comm.send({"method": "custom", "content": content}, buffers=buffers)

Of course, using the _repr_mimbundle_ name is a bit verbose, but I like how it makes it clear how these messages are only received if the front end is activated.

Second, we should have a way of marking attributes which shouldn't automatically be synchronized with the frontend. This would be the opposite of sync=True for traitlets where by default everything is synchronized.

I think there's a lot of potential in bringing react components into the ipywidget ecosystem.
This would open the door to projects like an https://www.tldraw.com/ based plot annotation tool, or interactive https://mafs.dev/ components for teaching math.

There's a minimal react example at
https://react.dev/learn/tutorial-tic-tac-toe#setup-for-the-tutorial

export default function Square() {
  return <button className="square">X</button>;
}

I don't know much about the react bundling process, therefore the question to you @manzt : would you have time and interest to investigate if it's possible to incorporate this minimal react component into an anywidget widget? That would be amazing!
And I think that other people from the community would be interested in this as well, e.g. @maartenbreddels https://discourse.jupyter.org/t/need-to-guidance-to-integrate-react-js-application-into-jupyter-extension/18477/3

Hi, I found that I cannot display any widgets created with anywidget (I tried a few examples off the web). When I run the code it finishes without an error, however no widget appears, the output is just empty. Its not a general display issue, as widgets created with ipywidgets are indeed shown, as can be seen here:

The same happens when I try to create widgets with other modules which depend on anywidget such as higlass.
I am new to jupyter so my set up might be wrong.
I installed the following in an empty conda env:

mamba create -n higlass-python
mamba activate higlass-python
mamba install python=3.11 # to install python3.11 (as otherwise aiohttp fails with python version 3.12)
pip install higlass-python
pip install ipykernel 
pip install clodius
pip install higlass-widget
pip install "anywidget[dev]"
pip install matplotlib
pip install ipympl

I am then starting a remote jupyter notebook on our HPC from within this environment with jupyter notebook --no-browser --port=8889 and open the notebook locally on my laptop. I also choose the correct kernel (as otherwise I get the "modules not found" errors).
Any idea what is happening? Perhaps I need to enable anywidgets or similar?

Working with a collaborator who is having issues using the vitessce widget. They see the error

Failed to load model class 'AnyModel' from module 'anywidget'
Error: No version of module anywidget is registered

I believe it is related to https://stackoverflow.com/questions/73715821/jupyter-lab-issue-displaying-widgets-javascript-error and jupyterlab/jupyterlab#12977 (comment)

Based on the environment info they shared with me, I am guessing they must have had ipywidgets==8.0.4 installed before trying to install vitessce (of course I can add a dependency on a specific ipywidgets version in the vitessce package, but probably best to fix at the anywidget level)

Minimal reproducer: https://colab.research.google.com/drive/1QkDXWsvyXISdGbR5ETb8HvOqR719lG9s?usp=sharing

I am trying to create a Box widget alternative to ipywidgets.Box. How to add children to it from _esm attribute which are provided from python side?

import anywidget
import traitlets
import ipywidgets as ipw

class Box(anywidget.AnyWidget):
    _esm = """
    export function render(view) {
        // How to access and display children here?
    }
    """
    children = traitlets.List(trait=traitlets.Instance(ipw.DOMWidget)).tag(sync=True, **ipw.widget_serialization)

Of course there should be nothing in the box as we did not handle it on JavaScript side.
Would it be nicer to add example of a container widget in docs?

          TODO: Right now the anywidget frontend code and the python package are published together with the same version. So `_model_module_version` and `_view_module_version` should be `anywidget.__version__`

Originally posted by @manzt in #46 (comment)

The current implementation of _get_pydantic_state_v2 is:

When you have a model that serializes to binary data, mode="json" is ok only when the binary data is actually a utf8 string. I.e.:

from pydantic import BaseModel

class Model(BaseModel):
    a: bytes

Model(a=b'abc').model_dump(mode='json') 
# {'a': 'abc'}

Model(a=b'\xa1').model_dump(mode='json')
# UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 0: invalid utf-8

But because anywidget uses remove_buffers, setting mode="json" isn't necessary.

state = Model(a=b'\xa1').model_dump()

from anywidget._util import remove_buffers
state2, buffer_paths, buffers = remove_buffers(state)
# {}, [['a']], [b'\xa1']

Though removing mode="json" might have regressions for other data types, like dates?

The demo at https://anywidget.dev/ is quite cool!

Just a minor comment on the behavior of counter.value :

In Jupyter notebooks, the output of counter.value is static, so it won't update immediately, as the website indicates.

This update will only happen after another cell execution.
Maybe this example at anywidget.dev can be tweaked in a way that this is also considered?
That would be amazing! 🚀

Now we need to inherit from AnyWidet and add the _esm. I've noticed many times in the notebook I want to pass a Path or str to the AnyWidget ctor. Does this make sense to add?

Currently some issues with renderer HTML with the latest nbconvert.

• jupyter/nbconvert#1934

In the "Example" section of this docs page there is reference to a value property. However no such value is defined in the snippet above. From my understanding, this should be count.

Would be great if the docs had sth similar to #97 for Svelte. Maybe as a vite subsection.

I did not realize that the cleanup function runs during view.remove(). I added a view.remove() call within the cleanup function returned by render() and then this caused an infinite loop in my widget by accident.

It might be possible to prevent users from getting in this situation in some way, but I am not sure. Maybe instead of passing this to render, a different object could be passed without a remove property or with remove() somehow overridden as a no-op?

If preventing it is not possible, the docs about the cleanup function could note that view.remove() should not be called inside the returned function, or could clarify that the cleanup function is run during DOMWidgetView.remove