It would be nice to use ReadTheDocs to get a documentation site up. This could include some generated API docs (especially for the plugin API) for the typescript side of the house, as well as provide more concrete examples of any built-in (or at least monorepo'd) plugins.

With a bit more work, this could also use jupyterlite (or jupyterlite-sphinx and get per-PR-push previews deployed.

Tasks

• baseline
  • in RTD web ui
    • reserve appropriate site name, e.g jupyterlab-markup.rtfd.io
    • set up preview builds
  • in a PR
    • create .readthedocs.yml
    • capture docs build deps
    • create docs/conf.py
    • create some basic docs examples
• advanced
  • create environment.yml (so the nodejs version can be pinned along with everything else)
  • set up typedoc
  • use jupyterlite
  • add link-checking
  • automate some screenshots with galata/robot

The render of any markdown cell shows empty blank.

Safari browser (13.0.4) work's fine.

Tested with Firefox 72.0.1 and Chrome 79.0.3...

I'm removing the server extension for now; the existing infrastructure is in-place if we need one in future, but given that most of the Jupyter stack is migrating to python>=3.7, the mimetypes fix should no longer be required.

One of the issues with our current approach is that we handle each cell very much in isolation. MyST cannot perform forward references with this limitation.

One option is to hack more deeply into the notebook, and allow a full-notebook render to be performed. I'm not really thinking the ramifications of this through too much yet, but I already spot some problems:

• A full re-render would be slow for big notebooks
• A partial re-render (parse whole doc, but only re-render cell) would be faster, but lead to out-of-sync issues.

Maybe there is something we could do at a different level to allow plugins to declare that they need more information / have dependencies. I'm just rambling at this stage.

Or, we take the easy way out and say that this is a limitation.

Elevator Pitch

Add bi-directional scrolling between an Editor and a Markdown Preview. Allow disabling this behavior.

Motivation

Yesterday during the collaborative editing demo during the community call, it was somewhat painful watching people editing markdown and have weird scrolling happen in an otherwise great experience.

As we can't get that data out marked upstream, this seems like the best place to land it. It's been a tick since I've looked at the code (binder's busted, btw), and I'd make this a plugin, but it feels like a it should be a core feature to this "ecosystem".

The other whammy is the default 300ms delay which is way too long with multiple editors, but that's for another day...

Design Ideas

The upstream demo contains the primary content for basic synced scrolling. Doing it right right is harder, but this would give us someplace to start.

As both the Editor and Markdown Preview documents have empty toolbars (by default), my inclination would be to enable this by default and offer a toggleable in Main Menu item in Edit or something.

Future Work

With more knowledge of the tokens between two documents, we could do even prettier LSP-style things, like jump-to-token and synced highlighting.

Update mermaidjs to support Class diagram

#63

First of all thanks for the awesome extension 😄

When trying it out I found that the syntax highlighting for yaml is broken.
For example this code snipped

```yaml
foo:
    bar: [1,2]
```

Without the extension, it renders like this

And after installing the extension

Html markup without extension

Html markup with extension

Maybe there is a way to tell markdown-it to ignore yaml and yml?

Other code blocks like python and js work normally.

While I love the idea of extensible markup in the Jupyter ecosystem, it's gonna get gross, and not reproducible real fast if content just silently looks bad if plugins are missing.

As this doesn't seem forthcoming in the CommonMark spec, this repo should probably demonstrate an approach to:

• instrumenting and capturing which plugins were actually used per render
• storing this metadata
  • in the notebook, this seems like a top-level metadata field

metadata:
  jupyter-markup:
    plugins:
      - footnote
      - deflist
cells: []

• in plaintext, presumably a comment syntax (gah!) could be added

<!-- jupyter-markup: footnote deflist -->

• providing some feedback (status bar?) when authoring/reading if plugins are missing
  • and how to get the missing plugins... this seems really hard to manage, especially considering...
• demonstrating/testing headless rendering in nbconvert/jupyter-book vs in-browser content
  • this would probably be a dependency of #11, rather than bundled

The extension works like a charm, but I wish there's a better way to view diagrams in dark themes.

Hey, just noticed the legwork for JupyterLab 3 has been done! 🎉 I'd be happy to maintain this on conda-forge, but would prefer a non-pre-release on PyPI as an upstream... keep/add the Alpha trove classifier, but it's all open source anyway!

If there are any other things you'd like to add before then, happy to help!

Support for PlantUML
markdown-it-plantuml

markdown-it-markspec

@bollwyvl I'm thinking that the solution here is a per-plugin options schema that is aggregated by the jupyterlab-markup extension. I'm not familiar yet with the settings configuration; do you think this is the right approach?

As of posting, JupyterLab 3 is still a release candidate, but is a pretty compelling way forward, and already pretty easy to develop against pip or conda. As #10 adds a serverextension to handle the .wasm mime type junk, it would be a relatively small step to move to a pip install-and-there-you-go end user experience.

Some concrete steps on top of #10:

• migrate to a jupyter_server (vs notebook) extension...
  • I haven't done this before, but it can't be that different
• add the labextension build stuff, dumping into src/jupyterlab_markup/static
• redo the build plumbing
  • source the version (and other package information) only from static/package.json
  • I'm a fan of doit for keeping the package in a releasable state from a single package

Because a big part of #10 is to make this extensible, it would still be important to do npm releases, so that others can get at the Token to make extensions with additional markdown-it plugins.

Currently, this extension doesn't support 1.0. This will change - I want to use the newest release myself, so I'll look into this.

The docs just say "Additional markdown rendering support in markdown."

This doesn't help me decide whether I would want to install this or not.

Please provide a list of features, links to documentation for those features and examples.

For example:

Install this extension if you need

• mermaid.js diagrams (examples)

in your Markdown cells

I have installed the following plugin with npm:
https://www.npmjs.com/package/markdown-it-image-figures

But I can't figure out how to activate it in the extension.

Markdown/CommonMark are hard to syntax highlight anyway, and a number of the plugins in #10 change the syntax in a way that is not covered by the existing ipythongfm mode. CodeMirror has been a sticky wicket in supporting Lab3 #11 as CodeMirror's approach doesn't

The diagrams modes like the ...

```mermaid

and...

```bob

syntaxes would be covered by new modes, as existing mode already defers fenced blocks, but other things like footnote and deflist would need a fair amount of massaging.

Generally, there should be a mechanism for a plugin to confidently add (and test) new syntax highlighting features. Hopefully this wouldn't mean rewriting the mode, but who knows!

From #10 (comment):

• the core extension can
• allow disabling built-in math typesetting (and or other features, like link replacement)
• provide access to lifecycle hooks to plugins for when those features would have happened

If a plugin wants to "step up" and overload an existing feature, it should be possible to do so. One certainly wouldn't want mulitple link replacers, math typesetters, or whatever.

Semi-concretely:

• expand options to be something like...

options: {
  MarkdownIt: ...
  renderMarkdown: (something) => Promise<Partial<Omit<renderMarkdown.IRenderOptions, 'host', 'source'>>>
}

• if multiple things try to register options, it's probably broken

• take over more of renderMarkdown

  • at each of the stages, allow each plugin to register things... so maybe like...
    • beforeMarkdownIt
    • afterMarkdownIt
    • beforeLatex
    • afterLatex
    • beforeHTML
    • afterHTML

• Each of these events would probably need a rank or weight for different plugins.

Some things that could be built with this, depending on how much context was given to the plugins:

• given just above, a server-based latex renderer
  • put a placeholder in (or the previous/cached math), wait for it to render
• given the current widget, a jupyter widget replacer
  • find specially-flavored links/images, e.g. ![](widget://foo) replace them with a widget
  • some lovely ones like tanglejs would make my day

Hi there,
i would like to have the ability to extend the rendering functionality of the plugin using preexisting python libraries.
Something similar to the feature in marked that lets you specify a python script to use to preprocess the text before rendering.

I think the implementation could be another option in the settings to specify the python file to load, with options for pre/post processing. Code wise the inputted file will be executed in the server extension. This will enable some pretty cool hooks into the rendering process i think.

I am trying to install jupyterlab-markup using the development steps at the bottom of the README but there is an error when importing 'StaticModule' from 'setuptools.config'. The error occurs when I run pip install -e . (output log). I am using Anaconda's Python 3.9.11 on MacOS 11.4 with a clean Anaconda virtual environment with only pip and jupyter installed. How can I fix this error?

@bollwyvl you're currently the maintainer of conda-forge for this extension, so you're a stakeholder here!

I realised that I had included a markdown-it dependencies that were ultimately not used, and when I went to add a plugin to support it, I realised that we're growing the size of this extension according to my interests.

I think it would be better to keep only a core number of plugins, e.g.

• anchor.ts

and move everything else into a separate jupyterlab-markup-XXX npm, PyPI & conda-forge packages.

This would be a bit more work, is it something you'd be able to help with on the conda side?

Currently we cache the md instance on the renderer. This means that when the settings are modified, the existing IRenderer instances never reflect the new changes.

Either we make getMarkdownIt return the same object where the settings are unmodified, or we listen to the settings changed signal. I can't think that we necessarily need a unique MarkdownIt instance per-renderer, so perhaps the former option is best.

Hi

I note that the plugin mechanism makes it relatively easy to add your own custom handlers for different sorts of structured content.

Do the plugins also have access to the notebook cell metadata? For example, could a plugin access the cell.model.metadata? If so, is there an example of this anywhere?

markdown cell doesn't show content after apply code

The install build process fails. I am not sure how resolve this.

pi@raspberrypi:~ $ sudo jupyter labextension install @agoose77/jupyterlab-markup Building jupyterlab assets (build:prod:minimize)
An error occured.
RuntimeError: JupyterLab failed to build
See the log file for details: /tmp/jupyterlab-debug-vvjmi75o.log

pi@raspberrypi:~ $ sudo cat /tmp/jupyterlab-debug-vvjmi75o.log
Node v10.15.2

Yarn configuration loaded.

/usr/local/bin/npm pack @agoose77/jupyterlab-markup
npm notice
npm notice 📦 @agoose77/[email protected]
npm notice === Tarball Contents ===
npm notice 1.6kB package.json
npm notice 1.3kB README.md
npm notice 238B lib/factories.d.ts
npm notice 384B lib/factories.js
npm notice 145B lib/index.d.ts
npm notice 759B lib/index.js
npm notice 1.6kB lib/renderers.d.ts
npm notice 2.4kB lib/renderers.js
npm notice 921B lib/widgets.d.ts
npm notice 2.7kB lib/widgets.js
npm notice === Tarball Details ===
npm notice name: @agoose77/jupyterlab-markup
npm notice version: 0.2.1
npm notice filename: agoose77-jupyterlab-markup-0.2.1.tgz
npm notice package size: 4.0 kB
npm notice unpacked size: 12.0 kB
npm notice shasum: 788f249c5c18ca3d145d35219e42821d2834669c
npm notice integrity: sha512-P3J6fsHWOCqu4[...]NgOW7zswGblSw==
npm notice total files: 10
npm notice
agoose77-jupyterlab-markup-0.2.1.tgz

Node v10.15.2

Yarn configuration loaded.
Building jupyterlab assets (build:prod:minimize)

node /usr/local/lib/python3.7/dist-packages/jupyterlab/staging/yarn.js install --non-interactive
yarn install v1.21.1
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
Done in 2.66s.

node /usr/local/lib/python3.7/dist-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer --fail
yarn run v1.21.1
$ /usr/local/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer --fail
Done in 3.12s.

node /usr/local/lib/python3.7/dist-packages/jupyterlab/staging/yarn.js run build:prod:minimize
yarn run v1.21.1
$ ensure-max-old-space webpack --config webpack.prod.minimize.config.js
/usr/bin/node[1493]: ../src/node_platform.cc:414:std::shared_ptrnode::PerIsolatePlatformData node::NodePlatform::ForIsolate(v8::Isolate*): Assertion `data' failed.
child_process.js:637
throw err;
^

Error: Command failed: /usr/local/share/jupyter/lab/staging/node_modules/.bin/webpack --config webpack.prod.minimize.config.js
at checkExecSyncError (child_process.js:616:11)
at Object.execFileSync (child_process.js:634:13)
at Object. (/usr/local/share/jupyter/lab/staging/node_modules/@jupyterlab/buildutils/lib/ensure-max-old-space.js:38:17)
at Module._compile (internal/modules/cjs/loader.js:689:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
at Module.load (internal/modules/cjs/loader.js:599:32)
at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
at Function.Module._load (internal/modules/cjs/loader.js:530:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
at startup (internal/bootstrap/node.js:283:19)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

JupyterLab failed to build
Traceback (most recent call last):

File "/usr/local/lib/python3.7/dist-packages/jupyterlab/debuglog.py", line 47, in debug_logging
yield

File "/usr/local/lib/python3.7/dist-packages/jupyterlab/labextensions.py", line 105, in start
command=command, app_options=app_options)

File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 460, in build
command=command, clean_staging=clean_staging)

File "/usr/local/lib/python3.7/dist-packages/jupyterlab/commands.py", line 661, in build
raise RuntimeError(msg)

RuntimeError: JupyterLab failed to build

Exiting application: jupyter
pi@raspberrypi:~ $

Hi

I notice that the code fence label for creating svgbob diagrams is bob whereas the label used in sphinx-contrib/svgbob is svgbob. It would be really useful if a common label were used particularly where notebooks are authored and previewed in JupyterLab/RetroLab and then rendered to Jupyter Book.

It looks like a fair number of bug fixes landed.

The @types haven't been updated, but otherwise seems to work locally, once it's forced with resolutions

I need to check whether it is acceptable for viewers to modify the model. If so, we can add the ability for checkboxes to be synchronised with the underlying source.