jupyter-attic / dashboards Goto Github PK
View Code? Open in Web Editor NEW[RETIRED] See Voilà as a supported replacement
Home Page: https://github.com/voila-dashboards
License: Other
[RETIRED] See Voilà as a supported replacement
Home Page: https://github.com/voila-dashboards
License: Other
For the Meetup Outreach demo, we moved some of our code out to HTML files which are siblings of the notebook file. We then import them like so:
%%html
<link rel="import" href="./dynamic-list.html">
<link rel="import" href="./upcoming-meetups.html">
The top of each of those HTML files has something like this:
<link rel="import" href="/urth_components/iron-list/iron-list.html"
is="urth-core-import" package="PolymerElements/iron-list">
<link rel="import" href="/urth_components/paper-toggle-button/paper-toggle-button.html"
is="urth-core-import" package="PolymerElements/paper-toggle-button#v1.0.10">
<link rel="import" href="/urth_components/paper-material/paper-material.html"
is="urth-core-import" package="PolymerElements/paper-material">
<dom-module id="upcoming-meetups">
...
This works well in the Notebook, but doesn't when we deploy. We get the following error:
GET http://192.168.99.100:9500/files/local_dashboards/meetup-outreach/static/urth_components/iron-list/iron-list.html 404 (Not Found)
To be more realistic, I think we'd want to filter user 1 from the candidate list. We don't want to bother a user who just registered for a meetup one of our meetups to register again for another meetup right away.
The Meetup Outreach demo (see issue #30) makes extensive use of globals in the Python code. While this works well when running the demo locally, it doesn't allow us to make good use of distributed nodes.
We need to replace the use of globals through the use of map/reduce and broadcast vars & accumulators.
I have a few lengthy notebooks, each with a large number of cells, and I want to surface more cells than would fit in a single screen dashboard view. So what I want to show is more like a report than a dashboard.
Say out of 100 cells, I want to select 20 to show up when I switch to View Dashboard. I don't really want to layout the cells, I just want to display them top to bottom, in the same order they show up in Notebook mode.
Ideally, I would be able to select which cells to display while in Notebook mode, by clicking something (icon?) in the top right corner of the cell. Then I could switch to View Dashboard mode to see the result. Layout Dashboard would not be necessary in this case.
We currently protect against the case when declarative widgets aren't installed. But we do still continue to copy bower_components
into every dashboard package, even when the notebook being converted doesn't use the widgets.
jupyter/declarativewidgets#75 will help, but I think this is possible even now if we have a sane way of determining if the notebook is using declarative widgets or not.
Scenario: Create a notebook to give an overview of community sentiment about a project. This notebook should show the community sentiment for a open source project on github. This demo should be streaming.
We frequently use the menu items for selecting a Dashboard view mode ('notebook', 'layout dashboard', 'view dashboard'). It may be beneficial to add a button group to the notebook toolbar to more quickly select a view mode.
When resizing a gridstack cell, the onresizestop
event is fired on mouse up before the CSS transition to snap to the grid completes. This results in incorrect element dimensions when performing sizing calculations based on this event.
To work around this issue, we added a setTimeout
to wait until a cell completes its CSS width/height transition before firing an iron-resize
event to Polymer output elements.
If any dep can't be downloaded, even temporarily, the whole bundle packaging fails with a 500 error server side. We should do something nicer.
Sections of a dashboard can be fetched with row and cell URL args on deployed dashboards. But findings the row/col values requires inspecting the DOM at runtime. Expose these in the dashboard layout UI.
Hi all,
I have been playing with IBM's dashboard following the README, thus deploying within a docker image. I was wondering if it is at all possible to setup the dashboard system on a normal JupyterHub install. For instance, does the docker image use a special fork of jupiter?
The taxi demo notebook should autoload table data as it hooks up a paper-menu
with pre-selected item to an urth-core-function
with an auto
attribute.
This used to load the table whose data is set by the auto
function so that the data is initially visible without user interaction to pick a dataset.
Not sure if this is a dashboard or declarative widgets issue.
1 Create the dashboard layout in the notebook.
2 Switch to 'View Dashboard' in the notebook.
3 Refresh the page and observe the dashboard rendering incorrectly.
This has only been seen on the whiskey dashboard notebook.
Single-column mode is enabled at 768px by default.
(See Gridstack.on_resize_handler function)
Resizing, hiding, and showing dashboard cells causes the page height to recompute, but moving a dashboard cell does not.
This could be as simple as calling the resize function after the Gridstack dragstop
event as we do with the resizestop
event.
When deploying a local dashboard, Polymer v1.2 is installed. But this breaks Urth channels (see jupyter/declarativewidgets#62). For that reason, the declarativewidgets devs have pegged their installs at v1.1.x. We should use the same versions that they define in their bower.json.
(Hey 🐮, let's 🍖 @ 🕤)
When in layout/view dashboard, decrease the window width so the layout becomes single-column and then to back to normal width. Switch back to notebook view and observe the cells are out of order.
It looks like gridstack is rearranging the DOM nodes at the smallest width breakpoint. We need to disable that.
This is related to a larger issue of how to do responsive dashboard layouts.
Long-term solution that probably involves:
The some urth widgets need to update their internal dimensions when their container is resized. This requires that we have some sort of event/notification other than window.onResize().
Our current idea is as follows: There exists PolymerElement/iron-resizable-behavior, which adds a notifyResize() method to the element. We can listen in dashboard for when a cell is resized, then find any polymer elements contained within, and if they implement this behavior, we notifyResize on them so that they can then resize appropriately.
The problem from the Dashboard side is how do we query for Polymer elements and find out if they implement/mixin the iron-resizable-behavior?
Need to document a few high-level suggestions somewhere:
Ideas:
Tooltip / popover that shows on first visit to layout mode in a notebook (set a global), but requires a click on an icon near the Dashboard section of the toolbar thereafter.
Code in error-log.js has to wait for kernel ready event like
IPython.notebook.events.on('kernel_ready.Kernel', {
// do error handling stuff
});
Happens in deployed dashboards, primarily.
gridstack/gridstack.js#238
gridstack/gridstack.js#216
Currently, Gridstack allows us to pass an option to remove_widget()
so that the DOM node isn't removed from the container, but add_widget()
and destroy()
don't have those options.
In the meantime, we have created our own "subclass" of Gridstack which adds these options (code).
We put off moving the code in this repo to supporting the post-Big Split version of the notebook until we understood the future intersection of the work here with the future version of notebook based on Phosphor. After meeting with @sccolbert, @ellisonbg and others about that topic, we agreed that pushing the dashboard layout implementation into jupyter/notebook ahead of the phosphor port of notebook doesn't make much sense: it'll all need to be redesigned and reimplemented later in the context of the more pluggable notebook. That said, it looks like the jQuery-based notebook will be around for a while longer as work proceeds on the Phosphor-based UI encompassing all the current pieces of Jupyter Notebook.
In the interest of keeping the dashboard (and declarative widgets) concepts relevant along the way, we should invest the time in making them 4.x compatible. I suspect that by 4.x compatible, I really mean "any version post-Big Split compatible" since even if the notebook version numbers jump to 5.x, 6.x, etc. major UI changes aren't likely to land in the classic, jQuery impl, but rather in the PhosphorJS one. So, hopefully, migrating the extensions should be a one time cost.
All that said, we agreed that once there's some flavor of Phosphor in Jupyter Notebook, we will collaborate on how dashboards and other incubator features start to plug into that new foundation.
The critical bit is in the render of the jinja template:
jinja2.exceptions.UndefinedError: \'IPython.nbformat.notebooknode.NotebookNode object\' has no attribute \'layout\
Best guess: the layout
loop var is not defined on all iteration of the loop.
[E 17:47:53.751 NotebookApp] Uncaught exception GET /user/9RDX2WXLAjPc/bundle?type=dashboard¬ebook=got_scotch_demo%2Fscotch_dashboard.ipynb (::ffff:45.37.178.151)
HTTPServerRequest(protocol='http', host='jupyter.cloudet.xyz', method='GET', uri='/user/9RDX2WXLAjPc/bundle?type=dashboard¬ebook=got_scotch_demo%2Fscotch_dashboard.ipynb', version='HTTP/1.1', remote_ip='::ffff:45.37.178.151', headers={'Accept-Encoding': 'gzip, deflate, sdch', 'Connection': 'close', 'X-Forwarded-For': '::ffff:45.37.178.151', 'Cookie': '_gat=1; _ga=GA1.2.1320566523.1442145202', 'X-Forwarded-Port': '80', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Referer': 'http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36', 'Upgrade-Insecure-Requests': '1', 'X-Forwarded-Proto': 'http', 'Accept-Language': 'en-US,en;q=0.8', 'Host': 'jupyter.cloudet.xyz'})
Traceback (most recent call last):
File "/opt/conda/lib/python3.4/site-packages/tornado/web.py", line 1413, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "/opt/conda/lib/python3.4/site-packages/tornado/web.py", line 2721, in wrapper
return method(self, *args, **kwargs)
File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 58, in get
self._get_local_app(abs_nb_path)
File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 251, in _get_local_app
md = self._create_app_bundle(abs_nb_path, bundle_root=os.path.join(self.nb_dir, 'local_dashboards'), overwrite=True, template_fn='local.tpl')
File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 133, in _create_app_bundle
converter.to_php_app(abs_nb_path, bundle_dir, template_fn)
File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/__init__.py", line 48, in to_php_app
full_output = to_thebe_html(notebook_fn, {}, 'html', os.getcwd(), template_fn)
File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/__init__.py", line 235, in to_thebe_html
raise RuntimeError('nbconvert wrote to stderr: {}'.format(stderr))
RuntimeError: nbconvert wrote to stderr: b'Traceback (most recent call last):\n File "/opt/conda/bin/ipython", line 6, in <module>\n sys.exit(start_ipython())\n File "/opt/conda/lib/python3.4/site-packages/IPython/__init__.py", line 120, in start_ipython\n return launch_new_instance(argv=argv, **kwargs)\n File "/opt/conda/lib/python3.4/site-packages/IPython/config/application.py", line 574, in launch_instance\n app.start()\n File "/opt/conda/lib/python3.4/site-packages/IPython/terminal/ipapp.py", line 367, in start\n return self.subapp.start()\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 278, in start\n self.convert_notebooks()\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 401, in convert_notebooks\n self.convert_single_notebook(notebook_filename)\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 372, in convert_single_notebook\n output, resources = self.export_single_notebook(notebook_filename, resources)\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 324, in export_single_notebook\n output, resources = self.exporter.from_filename(notebook_filename, resources=resources)\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/exporter.py", line 156, in from_filename\n return self.from_notebook_node(nbformat.read(f, as_version=4), resources=resources, **kw)\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/html.py", line 65, in from_notebook_node\n return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)\n File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/templateexporter.py", line 214, in from_notebook_node\n output = self.template.render(nb=nb_copy, resources=resources)\n File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 989, in render\n return self.environment.handle_exception(exc_info, True)\n File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 754, in handle_exception\n reraise(exc_type, exc_value, tb)\n File "/opt/conda/lib/python3.4/site-packages/jinja2/_compat.py", line 37, in reraise\n raise value.with_traceback(tb)\n File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/jinja_templates/thebe.tpl", line 83, in top-level template code\n data-gs-x={{layout.col}} data-gs-y={{layout.row}}\n File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 408, in getattr\n return getattr(obj, attribute)\njinja2.exceptions.UndefinedError: \'IPython.nbformat.notebooknode.NotebookNode object\' has no attribute \'layout\'\n\nIf you suspect this is an IPython bug, please report it at:\n https://github.com/ipython/ipython/issues\nor send an email to the mailing list at [email protected]\n\nYou can print a more detailed traceback right now with "%tb", or use "%debug"\nto interactively debug it.\n\nExtra-detailed tracebacks for bug-reporting purposes can be enabled via:\n c.Application.verbose_crash=True\n\n'
[E 17:47:53.759 NotebookApp] {
"Accept-Encoding": "gzip, deflate, sdch",
"Connection": "close",
"X-Forwarded-For": "::ffff:45.37.178.151",
"Cookie": "_gat=1; _ga=GA1.2.1320566523.1442145202",
"X-Forwarded-Port": "80",
"X-Forwarded-Proto": "http",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Referer": "http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36",
"Upgrade-Insecure-Requests": "1",
"Accept-Language": "en-US,en;q=0.8",
"Host": "jupyter.cloudet.xyz"
}
[E 17:47:53.759 NotebookApp] 500 GET /user/9RDX2WXLAjPc/bundle?type=dashboard¬ebook=got_scotch_demo%2Fscotch_dashboard.ipynb (::ffff:45.37.178.151) 970.95ms referer=http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb
Since moving the 'Upcoming Meetups' widget into an external Polymer element, the arrows no longer render since they are Font Awesome characters. We could change them to Unicode arrows to avoid including Font Awesome in the Polymer Element.
We've got a slew of content. Users need a roadmap. This should be something like the index.ipynb we put on jupyter.cloudet.xyz but specific to all the notebooks in this repo. The guide should suggest a sequence of reading through them to learn about dashboards and give a brief description of each to avoid having the user open each one to learn what it is.
We are currently running a Jenkins setup internally in IBM. We will for the time being point it to this new repo.
Some useful links about testing with docker and travis-ci
http://docs.travis-ci.com/user/docker/
When working with the Commnunity Sentiment Notebook multiple dom nodes are created for the urth-core-function
elements in dashboard view. It appears there are multiple instances of the notebook represented in gridstack, as well as duplication of the cells within a given output area.
When I reference an element by its id, I see an array returned.
The dom nodes are in the page in various places
One by itself:
And two within the same output area
The kicker is I can reproduce using the following code in that notebook:
%%html
<div id='baz'></div>
//i in dashboard mode
baz.length === 2 // true
However, if I start in a fresh notebook the problem doesn't happen.
Create an example notebook which explains how the Dashboard can interact with resizable Polymer elements (using the iron-resizable-behavior
).
In Layout mode, when I 'Add' a cell from the Hidden Cells section, it moves to the top section, but the cell is automatically sized and positioned in this top section (to optimize space).
If my notebook has quite a few cells, I may add several of them from the Hidden Cells section. When I scroll up, I am disoriented by the lack of order of the newly added cells. I may add a couple of plots, along with some markdown or other output cells to go along with them, but they all become jumbled, and I have to mentally piece the cell puzzle back together.
Admittedly, this is probably only an issue for "dashboards" with quite a few cells. Still, I'd like the option of turning off the auto-layout. When I add a hidden cell, it just gets placed full-width below the existing shown cells. I think this would lend itself to less effort in attaining the final layout.
Once we've got one example under our belt in the form of the community sentiment dashboard (issue #14), create a new notebook that tears it down to the barebones widget definitions and layouts. Document the structure of the data required by these widgets and how it must be bound to them. Then fill in very simple "hello world" versions of functions / DataFrames that generate fake data just so the notebook runs top to bottom.
The notebook should be very tutorial-like and walk people through what bits they need to replace and what bits stay. The goal is to demonstrate how UI doesn't have to be rewritten again and again for very similar use cases where only the domain data differs.
We already have the bower requirement within our dev docker container to support urth-widgets. We can piggy back on this to do the minimal bower install for our own dev purposes. Probably just involves a few swizzles in the Makefile. Pay off: you don't need node/bower installed on the host.
When positioning cells in Layout mode, I constantly find myself clicking and dragging the cell body to move cells, only to be reminded each time that it doesn't work that way. I then squint and locate the little move icon in the upper left corner of the cell and try again. 5 seconds later, I do the same for the next cell. Doh!
To recreate:
from urth.widgets.widget_channels import channel
channel('foobar').set('baz', 'awesome');
AttributeError Traceback (most recent call last)
<ipython-input-1-9054bc59971b> in <module>()
1 from urth.widgets.widget_channels import channel
----> 2 channel('foobar').set('baz', 'awesome');
/opt/conda/lib/python3.4/site-packages/urth/widgets/widget_channels.py in set(self, key, value, **kwargs)
75 def set(self, key, value, **kwargs):
76 global the_channels
---> 77 the_channels.set(key, value, self.chan, **kwargs)
78
79 def watch(self, key, handler):
AttributeError: 'NoneType' object has no attribute 'set'
@lbustelo says that the channel
code is initialized when the notebook receives a kernel ready event. I think we may be executing all code cells before that can happen. Perhaps this is due to how we invoke executeAll
in main.js -- need to wait for Urth widgets to fully initialize?
We think it's a notebook bug, but @parente seems to think it could be something we're tickling by hiding the header and toolbar.
Upon initial investigation it looks like Gridstack is not using saved layout coordinates for steps 1-5 above but is using saved layout coordinates for steps 6-10.
The notebook author may want to include external files (e.g. HTML & CSS) without having to write the entire contents in a notebook cell. This works locally, but when the notebook is converted (and deployed) any links to local files will not work. There should be some way of "marking" files for inclusion during conversion.
Example cell linking files:
%%html
<link rel="stylesheet" type="text/css" href="main.css">
<link rel="import" href="elements.html">
Some ideas:
Cell magics for including files
%%converter include "elements.html", "main.css"
%%linkcss "main.css"
%%linkhtml "elements.html"
The %%link* magics automatically insert the correct html tag, as well as being used to notify the converter.
UI component for manually selecting files to include
Continuation of #59. Move the dynamic list widget out of the meetup outreach notebook and use the association markup to get it packaged with the notebook / dashboard. Requires a fix on the widget side.
Goal: Have a demo based on a real business scenario that incorporates these key points:
Why: Demonstrate how notebooks can be used to prototype dashboards before further investment. Provide it as a tutorial / demo for the community to play with.
I need to drive new sign-ups to my website. One way to do that is to increase attendance at meetups I organize at meetup.com. I want to try to entice people:
To target these users specifically, I'd need to know:
Once I identify these users, I may want to:
After I take these actions, I certainly want to:
We need to show that dashboards don't only work with declarative widgets. Create a really simple demo that uses Bokeh or bqplot or ... whatever in a simple dashboard layout. Bonus: see if it can be deployed locally or if new, special hooks are required for the particular library.
In Layout mode,
When I resize a cell to make it larger, I think I'd like any cells that are being overlapped to be pushed beneath the one I'm enlarging. Less work for me.
Currently, when doing File>Deploy as>Local Dashboard, the converted files are added to the notebook server at the top level, as siblings to existing files and folders. This makes things very confusing when looking at the Notebook tree.
A better solution would be to have a top level folder called "converted" or "local_dashboards", where all the files reside.
jupyter_server already has the base_url on the end of it. Adding it here produces an incorrect URL.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.