0.5.0 may not work correctly with Vega 3.0.1 (My guess)
If i try to run the following example from https://vega.github.io/vega/examples/force-directed-layout/
within a jupyter notebook (with the json path modified), i get an empty rendered window.
jpynb.zip

Can probably copy over the configuration from the altair repo.

I think this will be a function you call, like

import vega
vega.polestar(dataframe)

Thoughts?

I haven't looked, but after #32 is merged, we should probably update the README:

• Provide links to the demo notebooks on GitHub and nbviewer.
• Start to promote that it works on GitHub and nbviewer.
• Audit the installation and dev instructions.
• Update screenshots for vega+vegalite.

So that it will work with the upcoming altair release.

Once it's on PyPI, I can update the conda recipe.

I think it would be useful to have the versions of the javascript packages available at some level, e.g.

from vega import VegaLite
print(VegaLite.library_version)

This would facilitate, e.g. raising a warning in Altair if someone has incompatible versions installed.

This came up in our chat – I think it's a good idea.

Could look something like this:

class Vegalite(object):
    def __init__(self, spec):
        try:
            spec = json.loads(spec)
        except TypeError:
            pass
        if 'data' not in spec:
            raise KeyError('No data provided with spec')
        self.spec = utils.update(spec, DEFAULTS, overwrite=False)

Advantage is more flexible API & ability to out-source dataframe formatting stuff to Pandas; the disadvantage is that in some situations we'd add an extra string->dict->string cycle.

We added some new pieces of this function in Altair. Those changes should be ported here.

Will do very soon. There are some pending changes in compass and voyager.

what happend?

Having a GH Milestone for the release will help us to target issues to the release.

What version number do we want the release to be?

Call this way:

from vega import vegalite
vegalite.display(spec)

Then display the visualization.

Per Dominik's request, I'm attaching my use case, and hope to work with the authors to make this use case work.

My workflow is simulation -> JSON -> pandas -> ? -> vega. Currently the ? is vincent, but vincent is not Vega-2.0-compliant and the author, who did a great job on it, isn't currently maintaining it. My simulation generates a lot of data; pandas is exactly the right tool (in python) for slicing and dicing that data; my hope is this package will fill in the ? where vincent is currently kind-of doing the job.

I am less interested in the ipython case and more interested in just using plain python as a batch script, but I think I will try out ipython since that is a design goal for y'all. I will also have suggestions on how to write good docs/examples that make it easy for others to use your tool. (This is a place where vincent could improve. What I will suggest is that for a given plot type, you explicitly say what it expects as an argument in terms of a pandas DataFrame, for instance, "GroupedBar: expects a pandas DataFrame where the rows are a set of measurements that are grouped together (in the GroupedBar 'group'), and each column is one individual measurement").

It is also my hope that I can generate both web plots (via D3, in the web browser) as well as print plots (SVG, PDF, etc.).

Thanks for writing this!

The following spec should produce a valid plot, and does when converted to JSON and fed to vega-lite directly:

spec = {'layers': [{'config': {'mark': {'color': '#1f77b4'}},
   'data': {'values': [{'x': 0, 'y': 0}, {'x': 1, 'y': 1}]},
   'encoding': {'x': {'field': 'x', 'type': 'quantitative'},
    'y': {'field': 'y', 'type': 'quantitative'}},
   'mark': 'line'},
  {'config': {'mark': {'color': '#ff7f0e'}},
   'data': {'values': [{'x': 0, 'y': 1}, {'x': 1, 'y': 0}]},
   'encoding': {'x': {'field': 'x', 'type': 'quantitative'},
    'y': {'field': 'y', 'type': 'quantitative'}},
   'mark': 'point'}]}

When used with this package, though, it results in an error:

from vega import VegaLite
VegaLite(spec)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-9af8065cc04d> in <module>()
      1 from vega import VegaLite
----> 2 VegaLite(spec)

/Users/jakevdp/anaconda/lib/python3.5/site-packages/vega/base.py in __init__(self, spec, data)
     21         """Initialize the visualization object."""
     22         spec = utils.nested_update(copy.deepcopy(self.DEFAULTS), spec)
---> 23         self.spec = self._prepare_spec(spec, data)
     24 
     25     def _prepare_spec(self, spec, data):

/Users/jakevdp/anaconda/lib/python3.5/site-packages/vega/vegalite.py in _prepare_spec(self, spec, data)
     22 
     23     def _prepare_spec(self, spec, data):
---> 24         return prepare_spec(spec, data)
     25 
     26 

/Users/jakevdp/anaconda/lib/python3.5/site-packages/vega/utils.py in prepare_spec(spec, data)
     91         # Data is either passed in spec or error
     92         if 'data' not in spec:
---> 93             raise ValueError('No data provided')
     94     else:
     95         # As a last resort try to pass the data to a DataFrame and use it

ValueError: No data provided

Currently the package assumes that data must appear at the top level; with the new Layer options, this is no longer strictly required.

The following installation step

jupyter nbextension install --py vega

fails when run inside virtualenv while attempting to modify global distribution directories

(mcmc)madanh@madanh ~/.virtualenvs/mcmc $ jupyter nbextension install --py vega
Installing /home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/vega/static -> jupyter-vega
Traceback (most recent call last):
  File "/home/madanh/.virtualenvs/mcmc/bin/jupyter-nbextension", line 11, in <module>
    sys.exit(main())
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/notebook/nbextensions.py", line 900, in start
    super(NBExtensionApp, self).start()
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/jupyter_core/application.py", line 256, in start
    self.subapp.start()
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/notebook/nbextensions.py", line 678, in start
    self.install_extensions()
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/notebook/nbextensions.py", line 657, in install_extensions
    **kwargs
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/notebook/nbextensions.py", line 225, in install_nbextension_python
    destination=dest, logger=logger
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/notebook/nbextensions.py", line 126, in install_nbextension
    ensure_dir_exists(nbext)
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/site-packages/ipython_genutils/path.py", line 167, in ensure_dir_exists
    os.makedirs(path, mode=mode)
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/os.py", line 227, in makedirs
    makedirs(head, mode, exist_ok)
  File "/home/madanh/.virtualenvs/mcmc/lib/python3.4/os.py", line 237, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/local/share/jupyter'

I might be doing something wrong here, but I couldn't import vega using Python 3.5.

ImportError                               Traceback (most recent call last)
<ipython-input-1-4f3fd77079e7> in <module>()
----> 1 import vega

/.../lib/python3.5/site-packages/vega/__init__.py in <module>()
----> 1 import tool
      2 import polestar
      3 import voyager
      4 import vegalite

ImportError: No module named 'tool'

Does it currently support only Python 2? Relative imports don't work like that in Python 3, if I remember correctly, you'd need to use import .tool. I can try to work on a Python 3 patch if needed. Any thoughts?

Because many python-generated vega plots result in data being embedded into the notebook, notebook size can quickly grow beyond what the browser can handle (see altair-viz/altair#249). Would there be a way to optionally enable PNG-only mode, where on save (or perhaps on render) the live output is replaced by the PNG and removed? I think the JS could be tweaked to do this, but it's not clear to me how this could be easily enabled/disabled from the Python side.

cc/ @ellisonbg

I've encounterd a problem with pyvega and the Python print function.
The following example works, hence the figure is displayed correctly in the notebook:

import altair
altair.Chart(altair.load_dataset('movies')).encode(x='Production_Budget', y='IMDB_Rating')

If I print anything in this same cell, the figure output disappears:

import altair
print('foo')
altair.Chart(altair.load_dataset('movies')).encode(x='Production_Budget', y='IMDB_Rating')

Is this expected behavior?
Niels

Use a path relative to the file location, not the cwd.

https://docs.python.org/2/library/os.path.html

The test case below works fine on http://vega.github.io/vega-editor/?mode=vega-lite but not in a notebook on a freshly built checkout of origin/master. Specifically, the larger of the two areas draws without normalization.

--------------- test case ----------
from vega import VegaLite
data = [
    {'count': 1532, 'date': '2016-07-05', 'label': 'group2'},
    {'count': 4712, 'date': '2016-07-05', 'label': 'group1'},
    {'count': 1736, 'date': '2016-07-06', 'label': 'group2'},
    {'count': 4970, 'date': '2016-07-06', 'label': 'group1'},
    {'count': 1960, 'date': '2016-07-07', 'label': 'group2'},
    {'count': 5248, 'date': '2016-07-07', 'label': 'group1'},
    {'count': 2002, 'date': '2016-07-08', 'label': 'group2'},
    {'count': 5360, 'date': '2016-07-08', 'label': 'group1'},
    {'count': 1969, 'date': '2016-07-09', 'label': 'group2'},
    {'count': 4548, 'date': '2016-07-09', 'label': 'group1'},
    {'count': 1931, 'date': '2016-07-10', 'label': 'group2'},
    {'count': 4631, 'date': '2016-07-10', 'label': 'group1'},
    {'count': 2316, 'date': '2016-07-11', 'label': 'group2'},
    {'count': 5817, 'date': '2016-07-11', 'label': 'group1'}]
graphic = {
    "mark": "area",
    "encoding": {
        "y": {"aggregate": "sum", "field": "count", "type": "quantitative"},
        "x": {"field": "date", "type": "temporal"},
    "color": {"field":"label", "type":"nominal", "scale": {"range": ["#e6aa06", "#1f77b4"]}}
    },
    "config": {"mark": {"stacked": "normalize"}}
}
VegaLite(graphic, data).display()
-----------------

It seems like the config cannot be overridden.

VegaLite({
  "mark": "point",
  "encoding": {
    "y": {"type": "quantitative","field": "Acceleration"},
    "x": {"type": "quantitative","field": "Horsepower"},
    "color": {"type": "nominal","field": "Origin"}
  },
  "config": {
    "cell": {
      "width": 100
    }
  }
}, df)

It would be great to streamline installation as much as possible. To that end, we should create a conda-forge recipe for vega which takes care of the installation/enabling of the jupyter extension. @ellisonbg mentioned that https://github.com/conda-forge/ipyleaflet-feedstock has something like this that we can adapt.

When you are done with your exploration, you probably want to have a visualization in a separate cell. We should make this process simple.

• change the version numbers in the package.json, index.py
• tag a version
• push to pypi and conda

Vega, vega-lite, polestar and voyager are on npm or bower. So we should use those.

I end up sharing a lot of rendered graphs on Slack which has the handy ability to upload images from the clipboard. With matplotlib visualizations there's a vanilla rendered that I can right-click and say 'copy image' and then just paste it into slack which works great. However with vega I have to add the intermediate step of saving the graph as a png first which really slows things down.

I was wondering if it might be possible to add another option beside "Export as PNG" like "Copy as PNG" that would do the same operation but plop the results in the clipboard?

Store the vl spec from polestar in the notebook file and load it if it is present.

I gather that something reads package.json and then updates index.js/index.js.map. We need the commands that do this to be documented somewhere in the package (perhaps in the README?)

Potentially using vega embed

It seems that we don't need it any more if we use images.

Instead of embedding the data as serialized json in the notebook, the data should be served from the kernel (through a server extension) if that's possible.

@ellisonbg could you tell us whether this is possible and how we roughly would go about this?

Currently plots don't render on nbviewer. I'm not sure how to address this, but I think @ellisonbg has some ideas.

An Altair user requested this: https://github.com/ellisonbg/altair/issues/200. The current behavior (silent failure) is not optimal for folks who skip reading the installation instructions 😄

I'm not certain how to do this, but @ellisonbg may have a better idea.

Support for these tools has been removed in #23. It would be great to add it back using the new nbextensions.

Using the nbconvert tool, it would be nice be able to export notebooks containing Vega plots in html (and rst) format. For static plots it just works, but for good reasons all the interactive features of a plot disappear. Links to the d3 and vega javascript libraries need to be added to the HTML template that is passed to nbconvert, and some magic to preserve the plot specification also need to happen in the preprocessing and conversion steps.

Are you able to offer advice on how to write such magic? By calling the private functions _generate_html and _generate_js on the vega.vega.Vega object I can get the relevant snippets of code, but my knowledge about Jupyter and nbconvert is not deep enough to understand how I can make them survive through to the output HTML.

Consistent repo name, package name, folder names.

Repo name should be ipython-vega, all other names should be vega.

vega.vegalite
vega.polestar
vega.voyager

Should be just like polestar. Let's reuse code.

In JupyterLab we are moving to rendering custom outputs using custom MIME types and functions that get registered to rendering those MIME types. I have that working well for JupyterLab, and am working on a version for the classic notebook. This will dramatically simplify the code base and lead to only the actual vega/vegalite JSON spec being stored in the notebook (rather than a bunch of JS code). Will also enable us to get these things working on nbviewer.

This will result in a complete refactor of basically everyfile in this repo...can I hold the lock for a few days?

@domoritz @jakevdp

• Is the spec is not valid, show an error

• Things in utils.py, especially prepare_spec and sanitize_dataframe.
• At least import things from the base.py, vega.py and vegalite.py.

I think the easiest integration of polestar with IPython notebooks is to have a configuration that disables the dataset selector and somehow listens to data changes from the kernel. We should implement a postMessage like interface. regardless, we have to make sure that we keep things contained so it works in jupyter and colab.

When performing a conda install, I'm seeing the following. Any thoughts on what might be going wrong?

% conda install vega --channel conda-forge
Fetching package metadata .........
Solving package specifications: ..........

Package plan for installation in environment /Users/cpd/anaconda/envs/defaultenv:

The following NEW packages will be INSTALLED:

    vega: 0.4.2-py27_1 conda-forge

Proceed ([y]/n)?

Linking packages ...
Install Jupyter notebook extensions

Usage

    jupyter nbextension install path/url

This copies a file or a folder into the Jupyter nbextensions directory. If a URL
is given, it will be downloaded. If an archive is given, it will be extracted
into nbextensions. If the requested files are already up to date, no action is
taken unless --overwrite is specified.

Options
-------

Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.

--quiet
    Minimal output
--symlink
    Create symlink instead of copying files
-s
    Create symlink instead of copying files
--user
    Install to the user's IPython directory
--debug
    Extra output
--overwrite
    Force overwrite of existing files
--prefix=<Unicode> (InstallNBExtensionApp.prefix)
    Default: ''
    Installation prefix
--destination=<Unicode> (InstallNBExtensionApp.destination)
    Default: ''
    Destination for the copy or symlink
--nbextensions=<Unicode> (InstallNBExtensionApp.nbextensions_dir)
    Default: ''
    Full path to nbextensions dir (probably use prefix or user)

To see all available configurables, use `--help-all`

Examples
--------

    jupyter nbextension install /path/to/myextension

[InstallNBExtensionApp] CRITICAL | Bad config encountered during initialization:
[InstallNBExtensionApp] CRITICAL | Unrecognized flag: '--py'
Error: Error: post-link failed for: conda-forge::vega-0.4.2-py27_1

https://github.com/wrobstory/vincent is a python to vega translator. We should have a simple way for users to create a vl visualizations from an ipython notebook.

Something like

import vega
vega.vegalite.create({
{
  "marktype": "point",
  "encoding": {
    "x": {"name": "Acceleration","type": "Q"},
    "y": {"name": "Horsepower","type": "Q"},
  }
}
})

For vegalite visualizations to be seen on sites like nbviewer or github, we need to have static PNG versions of them embedded into the notebooks output. It looks like it is possible to get a base64 encoded PNG from the SVG on the page. Then we would just need to use the output area JS API to put that into the cell. I will try to get an example working that does something like this.