Giter VIP home page Giter VIP logo

vitessce-python's Introduction

vitessce-python

PyPI docs

Binder Open In Colab

Python API and Jupyter widget facilitating interactive visualization of spatial single-cell data with Vitessce.

Installation

To install with pip:

$ pip install vitessce[all]

Getting started

Explore our example notebooks. These contain demos of different use cases and integrations with single-cell data analysis packages.

Development

For a development installation (requires NodeJS and NPM),

$ git clone https://github.com/vitessce/vitessce-python.git
$ cd vitessce-python
$ conda env create -f environment.yml
$ conda activate vitessce-python-dev
$ pip install -e ".[dev,docs,all]"

Conda environments

In this repository, there are multiple conda environments for different purposes:

Linting and testing

flake8
pytest

Formatting

One file:

autopep8 --in-place --aggressive ./path/to/file.py

All .py files:

find . -name '*.py' -exec autopep8 --in-place --aggressive '{}' \;

Documentation

make html

Deployment

To deploy a new version, increment the version of the Python package in pyproject.toml.

Then, when you push or merge the code with the incremented versions to main, the GitHub Action deploy.yml workflow will build and push the package to PyPI.

Troubleshooting

Check JavaScript console

Check the JavaScript console in the web browser for error messages. To do so, shift+right click in Jupyter, then click Inspect element. You may need to enable your browser's develop mode in its preferences for this option to appear when you right-click.

Check your Python environment

Check that you have activated the correct conda or other virtual environment as you expect. When sharing errors, it can be helpful to share the current versions of packages in the environment, e.g., by conda list or pip list.

Restart Kernel and Clear All Outputs, then refresh the browser tab

To ensure that older widget JavaScript output is not causing conflicts with newer JavaScript output (which may persist via notebook outputs even without running notebook cells), clear old outputs by Kernel -> Restart Kernel and Clear All Outputs. Then refresh the browser tab (containing Jupyter) to ensure all JavaScript outputs are fresh.

This is also typically required when running !pip install vitessce[all] from within a Jupyter notebook cell in the same session as running vc.widget() for the first time. The Kernel restart/Output clearing is required when a new Jupyter widget is installed in the environment, otherwise Jupyter will not yet recognize the newly-installed widget (as the notebook was opened with a different set of Jupyter widgets installed, and widget discovery is performed only at certain stages in the notebook lifecycle).

Check the widget configuration

If the widget renders successfully, you can get its current configuration by

vw = vc.widget()
vw
vw.config

Usage when Jupyter is running on a remote machine

If Jupyter is running on a remote machine, then use proxy=True. You may need to specify host_name as well (widget should be able to detect this but the plain display cannot).

vw = vc.widget(proxy=True)
vw

or

vc.display(proxy=True, host_name="http://localhost:8888")

Ensure that Vitessce and Jupyter are installed in the same environment

With the environment activated, run pip list on the command line:

Package                       Version
----------------------------- -----------
...
jupyterlab                    3.x.x
...
vitessce                      3.x.x

Resources

Getting/Offering Help

If you have a specific bug or feature request, please feel free to open an issue. Otherwise our discussions section is a great place to get help or offer it. If you aren't sure if something is a bug or not, don't have all the reproduction steps, or just have a general question, feel free to open a discussion post.

vitessce-python's People

Contributors

ilan-gold avatar ivababukova avatar keller-mark avatar manzt avatar mccalluc avatar nickakhmetov avatar sehilyi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

vitessce-python's Issues

Add a `split` parameter to hconcat and vconcat functions to specify fractions

Currently, hconcat and vconcat split the views equally. If two views are provided, they split into halfs. If three are provided, then thirds, etc.

We could add a new parameter to allow the exact breakdown to be specified by the user such as

hconcat(v1, v2, split=[1,2])

which would make v1 1/3 width and v2 2/3 width.

The denominator would just be the sum of the values

Clean Up Temp Directory After "Close"

@keller-mark Correct me if I'm wrong, but I think the temporary directory persists which can get a bit hairy with system memory since it is hard to find. We should add a function or something to clean up/delete the directory once you're "done" with it. Not sure what "done" looks like in code though...what actions cause it to fire.

`to_dict` -> `to_widget` Removes Parts of the View Config

For example

{
  "version": "1.0.0",
  "name": "277152f17b5a2f308820ab4d85c5a426",
  "description": "",
  "datasets": [
    {
      "uid": "A",
      "name": "277152f17b5a2f308820ab4d85c5a426",
      "files": [
        {
          "type": "cells",
          "fileType": "anndata-cells.zarr",
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "options": {
            "mappings": {
              "UMAP": {
                "key": "obsm/X_umap",
                "dims": [
                  0,
                  1
                ]
              }
            },
            "factors": [
              "obs/marker_gene_0",
              "obs/marker_gene_1",
              "obs/marker_gene_2",
              "obs/marker_gene_3",
              "obs/marker_gene_4"
            ]
          }
        },
        {
          "type": "cell-sets",
          "fileType": "anndata-cell-sets.zarr",
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "options": [
            {
              "groupName": "Leiden",
              "setName": "obs/leiden"
            }
          ]
        },
        {
          "type": "expression-matrix",
          "fileType": "anndata-expression-matrix.zarr",
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "options": {
            "matrix": "X",
            "matrixGeneFilter": "var/marker_genes_for_heatmap"
          }
        }
      ]
    }
  ],
  "coordinationSpace": {
    "dataset": {
      "A": "A"
    },
    "embeddingType": {
      "A": "UMAP"
    }
  },
  "layout": [
    {
      "component": "scatterplot",
      "coordinationScopes": {
        "dataset": "A",
        "embeddingType": "A"
      },
      "x": 0,
      "y": 0,
      "w": 4,
      "h": 6
    },
    {
      "component": "cellSetExpression",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 4,
      "y": 0,
      "w": 5,
      "h": 6
    },
    {
      "component": "cellSets",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 9,
      "y": 0,
      "w": 3,
      "h": 3
    },
    {
      "component": "genes",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 9,
      "y": 4,
      "w": 3,
      "h": 3
    },
    {
      "component": "heatmap",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 0,
      "y": 6,
      "w": 12,
      "h": 4
    }
  ],
  "initStrategy": "auto"
}

when fed through to_dict has a resulting Vitessce conf in the widget of

{
  "version": "1.0.1",
  "name": "277152f17b5a2f308820ab4d85c5a426",
  "description": "",
  "datasets": [
    {
      "uid": "A",
      "name": "277152f17b5a2f308820ab4d85c5a426",
      "files": [
        {
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "type": "cells",
          "fileType": "anndata-cells.zarr"
        },
        {
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "type": "cell-sets",
          "fileType": "anndata-cell-sets.zarr"
        },
        {
          "url": "https://assets.hubmapconsortium.org/277152f17b5a2f308820ab4d85c5a426/hubmap_ui/anndata-zarr/secondary_analysis.zarr",
          "type": "expression-matrix",
          "fileType": "anndata-expression-matrix.zarr"
        }
      ]
    }
  ],
  "coordinationSpace": {
    "dataset": {
      "A": "A"
    },
    "embeddingType": {
      "A": "UMAP"
    }
  },
  "layout": [
    {
      "component": "scatterplot",
      "coordinationScopes": {
        "dataset": "A",
        "embeddingType": "A"
      },
      "x": 0,
      "y": 0,
      "w": 4,
      "h": 6
    },
    {
      "component": "cellSetExpression",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 4,
      "y": 0,
      "w": 5,
      "h": 6
    },
    {
      "component": "cellSets",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 9,
      "y": 0,
      "w": 3,
      "h": 3
    },
    {
      "component": "genes",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 9,
      "y": 4,
      "w": 3,
      "h": 3
    },
    {
      "component": "heatmap",
      "coordinationScopes": {
        "dataset": "A"
      },
      "x": 0,
      "y": 6,
      "w": 12,
      "h": 4
    }
  ],
  "initStrategy": "auto"
}

which is missing all of the options

Adjust grid layout columns to prevent float values for x,y,w,h

In addition, the following code returns an error message.

vc.layout((umap | pca | cell_sets | genes) / heatmap);
Config validation failed
[
  {
    "keyword": "type",
    "dataPath": ".layout[0].w",
    "schemaPath": "#/definitions/components/items/properties/w/type",
    "params": {
      "type": "integer"
    },
    "message": "should be integer"
  }
]

I guess this is because the w value in react-grid-layout is accepting only integer values by a JSON schema, and adding many columns results the width less than 1.0. (1) I think the error message can be more understandable to users, or (2) allow double values (Although only the integer values are said to be officially allowed by react-grid-layout, I think double values work fine as well?).

Originally posted by @sehilyi in #34 (comment)

Vitessce JS dependency not upgrading on Google Colab

When I install the latest version of the vitessce Python package on Google Colab, the vitessce JS version is still 1.1.0. It should be 1.1.14. It is unclear to me why this is happening.

For example:

https://colab.research.google.com/github/vitessce/vitessce-python/blob/keller-mark/colab-notebook/docs/notebooks/widget_on_colab.ipynb

Run the cells in this notebook and then open the console. It says the vitessce version is 1.1.0 and the config validation fails because the Python code is generating a config with schema version 1.0.4 but that schema did not exist under the older vitessce JS package.

`data_export_s3.ipynb`: `FutureWarning: Moving element from .uns['neighbors']['distances'] to .obsp['distances'].`

os.makedirs("data", exist_ok=True)
adata_filepath = join("data", "habib17.processed.h5ad")
urlretrieve('https://covid19.cog.sanger.ac.uk/habib17.processed.h5ad', adata_filepath)
​
adata = read_h5ad(adata_filepath)
top_dispersion = adata.var["dispersions_norm"][
    sorted(
        range(len(adata.var["dispersions_norm"])),
        key=lambda k: adata.var["dispersions_norm"][k],
    )[-51:][0]
]
adata.var["top_highly_variable"] = (
    adata.var["dispersions_norm"] > top_dispersion
)
/opt/anaconda3/envs/vitessce-jupyter-examples/lib/python3.9/site-packages/anndata/compat/__init__.py:180: FutureWarning: Moving element from .uns['neighbors']['distances'] to .obsp['distances'].

This is where adjacency matrices should go now.
  warn(
/opt/anaconda3/envs/vitessce-jupyter-examples/lib/python3.9/site-packages/anndata/compat/__init__.py:180: FutureWarning: Moving element from .uns['neighbors']['connectivities'] to .obsp['connectivities'].

This is where adjacency matrices should go now.
  warn(

Either fix the warning... or add a note so the user isn't unduly alarmed by the warning?

Is there a suggested order to explore `docs/notebooks/`? Perhaps not...

https://github.com/vitessce/vitessce-python/blob/master/docs/widget_examples.rst determines the listing order for the notebooks in the generated documentation... but if there's a definite order a new user should look at these, should that be captured in the README.md? Or maybe the filenames could be numbered, so a jupyter user could just go down the list?

Somewhat related: How does the Jupyter user download the prerequisite data? #76

in `docs/notebooks/` factor out shared data download steps into separate notebook? Or `.py`?

Would it make sense to separate out the data directory creation and downloads to a separate notebook, if multiple notebooks use the same sample data? And then just have a link in the main notebooks to the prerequisite notebooks? Right now some notebooks are missing steps, and they can't just be run from the top to the bottom.

... Alternatively, the shared download code could be handled as just a plain .py file in this directory, and it could actually be run from each of the examples.

  • Plus: the notebooks could actually be run from top to bottom. (I know I don't always actually real the instructions.)
  • Minus: in terms of explaining the steps, we'd be hiding some of the details.

... but if the primary purpose of these notebooks is actually to support https://vitessce.github.io/vitessce-python/widget_examples.html, then we don't want references to files that only make sense when running the notebook: We really need everything to be right there, in each notebook.

There are ways to run one notebook from another, but what I think we'd really like is some kind of transclusion of the content, and I don't see that.

Should more functions be object oriented?

Rather than my_widget = VitessceWidget(config) do we want the API to be my_widget = config.widget()?

And likewise for uploading to AWS S3? Rather than upload_to_s3(config, s3, bucket_name), the API could be config.upload_to_s3(s3, bucket_name)

Add option to export data to local files

User may want a file export function if using a static web server other than S3 or others that are supported officially by this package.

Tricky thing: need to know the URL where the data will ultimately be served from before finalizing the view config. Could just ask the user for a "base URL"

`ContextualVersionConflict` when following getting started instructions ... but works on retry?

I've followed the README instructions up to here:

git clone https://github.com/vitessce/vitessce-python.git
cd vitessce-python/
conda env create -f environment.yml
conda activate vitessce-jupyter-dev
pip install -e .

... and then I get this error:

pip._vendor.pkg_resources.ContextualVersionConflict: (pandas 0.25.1 (/opt/anaconda3/lib/python3.7/site-packages), Requirement.parse('pandas>=1.1.2'), {'vitessce'})
$ pip install -e .
Obtaining file:///Users/chuck/github/hubmap/vitessce-python
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 407, in check_if_exists
    self.satisfied_by = pkg_resources.get_distribution(str(no_marker))
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 481, in get_distribution
    dist = get_provider(dist)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 357, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pip._vendor.pkg_resources.ContextualVersionConflict: (pandas 0.25.1 (/opt/anaconda3/lib/python3.7/site-packages), Requirement.parse('pandas>=1.1.2'), {'vitessce'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 345, in run
    resolver.resolve(requirement_set)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 196, in resolve
    self._resolve_one(requirement_set, req)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 359, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 291, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 255, in prepare_editable_requirement
    req.check_if_exists(use_user_site)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 418, in check_if_exists
    self.req.name
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 481, in get_distribution
    dist = get_provider(dist)
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 357, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pip._vendor.pkg_resources.ContextualVersionConflict: (pandas 0.25.1 (/opt/anaconda3/lib/python3.7/site-packages), Requirement.parse('pandas>=1.1.2'), {'vitessce'})

Poked around, didn't have any insights, tried again, and now it works:

$ pip install -e .
Obtaining file:///Users/chuck/github/hubmap/vitessce-python
Requirement already satisfied: ipywidgets>=7.0.0 in /opt/anaconda3/lib/python3.7/site-packages (from vitessce==0.1.0a10) (7.5.1)
Collecting hypercorn>=0.11.0 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/68/53/9ceb193c35ce828936cb522b8759c99422b3453d3563ef0f0054ba3f9792/Hypercorn-0.11.2-py3-none-any.whl
Collecting ujson>=4.0.1 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/32/49/84d979d75e4a01b92271b1451e31945eac0d5239bf481ba025af9ab271d7/ujson-4.0.2-cp37-cp37m-macosx_10_14_x86_64.whl
Collecting aiofiles>=0.6.0 (from vitessce==0.1.0a10)
  Downloading https://files.pythonhosted.org/packages/e7/61/007ac6f27fe1c2dc44d3a62f429a8440de1601428b4d0291eae1a3494d1f/aiofiles-0.7.0-py3-none-any.whl
Collecting starlette==0.14.0 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/3b/48/c305e580e6584d8dd0c2c58238dac973f484345d9de4bc1aa5b162c86a54/starlette-0.14.0-py3-none-any.whl
Collecting zarr>=2.5.0 (from vitessce==0.1.0a10)
  Downloading https://files.pythonhosted.org/packages/95/57/9d4833535d11e8e83629e0a0a78fa1115deadf59532e226082346ee3e677/zarr-2.8.3-py3-none-any.whl (140kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 143kB 4.4MB/s
Collecting numcodecs>=0.5.7 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/bf/e6/2b34ce6db3dbb13c5b2e6e41d4b5a5bc98e0fc2f8e039249179b0a2c706b/numcodecs-0.7.3-cp37-cp37m-macosx_10_9_x86_64.whl
Requirement already satisfied: scipy>=1.2.1 in /opt/anaconda3/lib/python3.7/site-packages (from vitessce==0.1.0a10) (1.3.1)
Collecting negspy>=0.2.24 (from vitessce==0.1.0a10)
Collecting generate-tiff-offsets>=0.1.7 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/a3/32/0b489c4d19e5b2cd06abbbdcbc0b0a330574d6d50fa024e188928e7a6f85/generate_tiff_offsets-0.1.7-py2.py3-none-any.whl
Collecting pandas>=1.1.2 (from vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/e2/01/d6ab319ffec641987d574ad2d1a9adee281389d5e24955f140d5e7c20283/pandas-1.2.4-cp37-cp37m-macosx_10_9_x86_64.whl
Requirement already satisfied: nbformat>=4.2.0 in /opt/anaconda3/lib/python3.7/site-packages (from ipywidgets>=7.0.0->vitessce==0.1.0a10) (4.4.0)
Requirement already satisfied: ipykernel>=4.5.1 in /opt/anaconda3/lib/python3.7/site-packages (from ipywidgets>=7.0.0->vitessce==0.1.0a10) (5.1.2)
Requirement already satisfied: widgetsnbextension~=3.5.0 in /opt/anaconda3/lib/python3.7/site-packages (from ipywidgets>=7.0.0->vitessce==0.1.0a10) (3.5.1)
Requirement already satisfied: traitlets>=4.3.1 in /opt/anaconda3/lib/python3.7/site-packages (from ipywidgets>=7.0.0->vitessce==0.1.0a10) (4.3.3)
Requirement already satisfied: ipython>=4.0.0; python_version >= "3.3" in /opt/anaconda3/lib/python3.7/site-packages (from ipywidgets>=7.0.0->vitessce==0.1.0a10) (7.8.0)
Collecting priority (from hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/de/96/2f4b8da7be255cd41e825c398efd11a6706ff86e66ae198f012204aa2a4f/priority-1.3.0-py2.py3-none-any.whl
Collecting typing-extensions; python_version < "3.8" (from hypercorn>=0.11.0->vitessce==0.1.0a10)
  Downloading https://files.pythonhosted.org/packages/2e/35/6c4fff5ab443b57116cb1aad46421fb719bed2825664e8fe77d66d99bcbc/typing_extensions-3.10.0.0-py3-none-any.whl
Requirement already satisfied: toml in /opt/anaconda3/lib/python3.7/site-packages (from hypercorn>=0.11.0->vitessce==0.1.0a10) (0.10.2)
Collecting wsproto>=0.14.0 (from hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/ea/25/0934b1d00f404d75335b144d4396e01998f25db8953bf54b4d6fe65b80ab/wsproto-1.0.0-py3-none-any.whl
Collecting h11 (from hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/60/0f/7a0eeea938eaf61074f29fed9717f2010e8d0e0905d36b38d3275a1e4622/h11-0.12.0-py3-none-any.whl
Collecting h2>=3.1.0 (from hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/bd/c2/5ffec707d0022208787908d9657f782ce35b653baa1e87abecf22a7cf513/h2-4.0.0-py3-none-any.whl
Requirement already satisfied: numpy>=1.7 in /opt/anaconda3/lib/python3.7/site-packages (from zarr>=2.5.0->vitessce==0.1.0a10) (1.17.2)
Collecting asciitree (from zarr>=2.5.0->vitessce==0.1.0a10)
Collecting fasteners (from zarr>=2.5.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/78/20/c862d765287e9e8b29f826749ebae8775bdca50b2cb2ca079346d5fbfd76/fasteners-0.16-py2.py3-none-any.whl
Collecting tifffile==2020.10.1 (from generate-tiff-offsets>=0.1.7->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/e8/8c/166c88fcbe3b3632dcf93a106f6d13892b1a2b822b61eb7cd9a5ab68b259/tifffile-2020.10.1-py3-none-any.whl
Requirement already satisfied: pytz>=2017.3 in /opt/anaconda3/lib/python3.7/site-packages (from pandas>=1.1.2->vitessce==0.1.0a10) (2019.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/anaconda3/lib/python3.7/site-packages (from pandas>=1.1.2->vitessce==0.1.0a10) (2.8.0)
Requirement already satisfied: ipython-genutils in /opt/anaconda3/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.2.0)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /opt/anaconda3/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (3.1.0)
Requirement already satisfied: jupyter-core in /opt/anaconda3/lib/python3.7/site-packages (from nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (4.5.0)
Requirement already satisfied: tornado>=4.2 in /opt/anaconda3/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets>=7.0.0->vitessce==0.1.0a10) (6.0.3)
Requirement already satisfied: jupyter-client in /opt/anaconda3/lib/python3.7/site-packages (from ipykernel>=4.5.1->ipywidgets>=7.0.0->vitessce==0.1.0a10) (5.3.3)
Requirement already satisfied: notebook>=4.4.1 in /opt/anaconda3/lib/python3.7/site-packages (from widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (6.0.1)
Requirement already satisfied: six in /opt/anaconda3/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.0.0->vitessce==0.1.0a10) (1.12.0)
Requirement already satisfied: decorator in /opt/anaconda3/lib/python3.7/site-packages (from traitlets>=4.3.1->ipywidgets>=7.0.0->vitessce==0.1.0a10) (4.4.0)
Requirement already satisfied: pickleshare in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.7.5)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (2.0.10)
Requirement already satisfied: backcall in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.1.0)
Requirement already satisfied: pygments in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (2.8.1)
Requirement already satisfied: setuptools>=18.5 in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (41.4.0)
Requirement already satisfied: appnope; sys_platform == "darwin" in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.1.0)
Requirement already satisfied: jedi>=0.10 in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.15.1)
Requirement already satisfied: pexpect; sys_platform != "win32" in /opt/anaconda3/lib/python3.7/site-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (4.7.0)
Collecting hyperframe<7,>=6.0 (from h2>=3.1.0->hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/d7/de/85a784bcc4a3779d1753a7ec2dee5de90e18c7bcf402e71b51fcf150b129/hyperframe-6.0.1-py3-none-any.whl
Collecting hpack<5,>=4.0 (from h2>=3.1.0->hypercorn>=0.11.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/d5/34/e8b383f35b77c402d28563d2b8f83159319b509bc5f760b15d60b0abf165/hpack-4.0.0-py3-none-any.whl
Requirement already satisfied: pyrsistent>=0.14.0 in /opt/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.15.4)
Requirement already satisfied: attrs>=17.4.0 in /opt/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (19.2.0)
Requirement already satisfied: importlib-metadata in /opt/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.23)
Requirement already satisfied: js-regex>=1.0.0 in /opt/anaconda3/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (1.0.1)
Requirement already satisfied: pyzmq>=13 in /opt/anaconda3/lib/python3.7/site-packages (from jupyter-client->ipykernel>=4.5.1->ipywidgets>=7.0.0->vitessce==0.1.0a10) (18.1.0)
Requirement already satisfied: nbconvert in /opt/anaconda3/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (5.6.0)
Requirement already satisfied: terminado>=0.8.1 in /opt/anaconda3/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.8.2)
Requirement already satisfied: jinja2 in /opt/anaconda3/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (2.10.3)
Requirement already satisfied: Send2Trash in /opt/anaconda3/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (1.5.0)
Requirement already satisfied: prometheus-client in /opt/anaconda3/lib/python3.7/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.7.1)
Requirement already satisfied: wcwidth in /opt/anaconda3/lib/python3.7/site-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.1.7)
Requirement already satisfied: parso>=0.5.0 in /opt/anaconda3/lib/python3.7/site-packages (from jedi>=0.10->ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.5.1)
Requirement already satisfied: ptyprocess>=0.5 in /opt/anaconda3/lib/python3.7/site-packages (from pexpect; sys_platform != "win32"->ipython>=4.0.0; python_version >= "3.3"->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.6.0)
Requirement already satisfied: zipp>=0.5 in /opt/anaconda3/lib/python3.7/site-packages (from importlib-metadata->jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.6.0)
Collecting mistune<2,>=0.8.1 (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10)
  Using cached https://files.pythonhosted.org/packages/09/ec/4b43dae793655b7d8a25f76119624350b4d65eb663459eb9603d7f1f0345/mistune-0.8.4-py2.py3-none-any.whl
Requirement already satisfied: bleach in /opt/anaconda3/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (3.1.0)
Requirement already satisfied: pandocfilters>=1.4.1 in /opt/anaconda3/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (1.4.2)
Requirement already satisfied: defusedxml in /opt/anaconda3/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.6.0)
Requirement already satisfied: entrypoints>=0.2.2 in /opt/anaconda3/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.3)
Requirement already satisfied: testpath in /opt/anaconda3/lib/python3.7/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.4.2)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/anaconda3/lib/python3.7/site-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (1.1.1)
Requirement already satisfied: more-itertools in /opt/anaconda3/lib/python3.7/site-packages (from zipp>=0.5->importlib-metadata->jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (7.2.0)
Requirement already satisfied: webencodings in /opt/anaconda3/lib/python3.7/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.0.0->vitessce==0.1.0a10) (0.5.1)
Installing collected packages: priority, typing-extensions, h11, wsproto, hyperframe, hpack, h2, hypercorn, ujson, aiofiles, starlette, numcodecs, asciitree, fasteners, zarr, negspy, tifffile, generate-tiff-offsets, pandas, vitessce, mistune
  Found existing installation: pandas 0.25.1
    Uninstalling pandas-0.25.1:
      Successfully uninstalled pandas-0.25.1
  Found existing installation: vitessce 0.1.0a10
    Can't uninstall 'vitessce'. No files were found to uninstall.
  Running setup.py develop for vitessce
Successfully installed aiofiles-0.7.0 asciitree-0.3.3 fasteners-0.16 generate-tiff-offsets-0.1.7 h11-0.12.0 h2-4.0.0 hpack-4.0.0 hypercorn-0.11.2 hyperframe-6.0.1 mistune-0.8.4 negspy-0.2.24 numcodecs-0.7.3 pandas-1.2.4 priority-1.3.0 starlette-0.14.0 tifffile-2020.10.1 typing-extensions-3.10.0.0 ujson-4.0.2 vitessce wsproto-1.0.0 zarr-2.8.3

Add a method to render the widget via iframe

Google colab still does not support custom widgets googlecolab/colabtools#498 but we may be able to work around this for now by rendering the config using vitessce.io/?url=data:,{config_json} in an <iframe/>.

However this would not support two-way data binding.

Note: the page in the iframe will need to be served over HTTPS, otherwise there is a mixed active content error since Colab uses HTTPS

Handle pandas Import Properly

Currently, installing this package requires pandas because of https://github.com/vitessce/vitessce-python/blob/master/vitessce/wrappers.py#L9

We should probably either move this import into the ATAC-seq pipeline or add it here:

vitessce-python/setup.py

Lines 147 to 158 in b785f6b

install_requires=[
'ipywidgets>=7.0.0',
'hypercorn>=0.11.0',
'ujson>=4.0.1',
'aiofiles>=0.6.0',
'starlette==0.14.0',
'zarr>=2.5.0',
'numcodecs>=0.5.7',
'scipy>=1.2.1',
'negspy>=0.2.24',
'generate-tiff-offsets>=0.1.7'
],

Feel Free to Use Discussions for General Questions!

Hello! We have a discussions section where you can ask for general help, advice, or help others. Of course, if you have a specific bug or feature you'd like (to report), then feel free to open an issue. Otherwise discussions are a great place to chat more freely and get help from maintainers (and the community) with your specific use-case.

Support Multi-Image Raster JSON

For multi-modal/image registration use cases, supporting more than one image in the images part of the raster json schema should be a feature.

Support Seurat data objects

Even though the seurat ecosystem is primarily based in R, I believe the h5Seurat file is just an HDF5 file which should be able to be loaded in Python as well

Check Centroid Condition for `AnnData` and `cells.json`

Right now, I believe the xy part of the cells.json (which is created from AnnData loaders and validated) needs to be part of the data because it is used for getting cell coordinates i.e the centroid needs to be included. We should thus throw an error on the AnnData loader if the polygons (i.e poly in the cells.json schema) are passed in but the centroids (xy in that lingo) are not:

https://github.com/vitessce/vitessce-python/blob/master/vitessce/wrappers.py#L365

Allow layout() to be used with explicit x/y/w/h values

Currently, calling vc.layout() will overwrite previous values for component position and dimension in the grid layout.

The layout() function could be updated to take into account existing x/y/w/h values and only arrange components in spaces of the grid which are not pre-occupied

How to convert labeled masks image to poly outlines?

Hi, I am trying to use vitessce to display cell phenotyping together with ome-tiff images. What I got for the cell segmentation masks is a tiff image labeled with different ID number for each cell. I wonder how to prepare this kind of mask into cells.json.
I also notice that the OmeTiffWrapper is able to take bitmask. Does bitmask work the same as the poly outline? If not, which one is preferred?

Thanks!

how to vc.widget() in my server

Hi,
I follow steps in:
https://github.com/vitessce/vitessce-python/blob/master/docs/notebooks/widget_brain.ipynb
when I run on university hpc server using

vw=vc.widget()
vw

I got Running on http://127.0.0.1:8002
and error in chrome:
This site can’t be reached127.0.0.1 refused to connect.

Any suggestion how to access the vw?

Thank you

BTW:
I created the following conda env

$ git clone https://github.com/vitessce/vitessce-python.git
cd docs/notebooks
conda env create -f environment.yml
conda activate vitessce-jupyter-examples
pip install -e ../..
jupyter labextension install @jupyter-widgets/jupyterlab-manager
jupyter labextension install ../../js

then
jupyter lab

`ValueError: "@jupyter-widgets/jupyterlab-manager" is not a valid extension`

Follow-up from #64 ... Continuing with the README, the next steps work, and then:

$ jupyter labextension install @jupyter-widgets/jupyterlab-manager
An error occured.
ValueError: "@jupyter-widgets/jupyterlab-manager" is not a valid extension:
schemaDir is empty: "./schema"
See the log file for details:  /var/folders/2f/yvyq4r852yxg3xf902p52w5h0000gn/T/jupyterlab-debug-8qzkz0cf.log
debug log
$ cat /var/folders/2f/yvyq4r852yxg3xf902p52w5h0000gn/T/jupyterlab-debug-8qzkz0cf.log
Node v14.1.0

> /Users/chuck/.nvm/versions/node/v14.1.0/bin/npm pack @jupyter-widgets/jupyterlab-manager
npm notice
npm notice πŸ“¦  @jupyter-widgets/[email protected]
npm notice === Tarball Contents ===
npm notice 14.5kB LICENSE
npm notice 380B   lib/index.js
npm notice 15.3kB lib/manager.js
npm notice 5.3kB  lib/output.js
npm notice 7.8kB  lib/plugin.js
npm notice 3.3kB  lib/renderer.js
npm notice 862B   lib/semvercache.js
npm notice 3.3kB  package.json
npm notice 375B   schema/plugin.json
npm notice 2.8kB  README.md
npm notice 277B   lib/index.d.ts
npm notice 5.9kB  lib/manager.d.ts
npm notice 1.9kB  lib/output.d.ts
npm notice 828B   lib/plugin.d.ts
npm notice 960B   lib/renderer.d.ts
npm notice 216B   lib/semvercache.d.ts
npm notice === Tarball Details ===
npm notice name:          @jupyter-widgets/jupyterlab-manager
npm notice version:       3.0.0
npm notice filename:      jupyter-widgets-jupyterlab-manager-3.0.0.tgz
npm notice package size:  15.6 kB
npm notice unpacked size: 64.0 kB
npm notice shasum:        fa47d03e2e72399ce3af4b86cc29aba2166d5781
npm notice integrity:     sha512-9diAvsHHiK/kY[...]cKQSYU/tiXxdQ==
npm notice total files:   16
npm notice
jupyter-widgets-jupyterlab-manager-3.0.0.tgz

Traceback (most recent call last):

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/debuglog.py", line 47, in debug_logging
    yield

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/labextensions.py", line 93, in start
    ans = self.run_task()

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/labextensions.py", line 147, in run_task
    for i, arg in enumerate(self.extra_args)

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/labextensions.py", line 147, in <listcomp>
    for i, arg in enumerate(self.extra_args)

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/commands.py", line 321, in install_extension
    return handler.install_extension(extension, pin=pin)

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/commands.py", line 525, in install_extension
    info = self._install_extension(extension, tempdir, pin=pin)

  File "/opt/anaconda3/lib/python3.7/site-packages/jupyterlab/commands.py", line 1415, in _install_extension
    raise ValueError(msg % (extension, '\n'.join(messages)))

ValueError: "@jupyter-widgets/jupyterlab-manager" is not a valid extension:
schemaDir is empty: "./schema"

Exiting application: jupyter

`widget_from_dict.ipynb`: Scatterplots not rendering

When I run widget_from_dict.ipynb, the two scatterplots are consistently not loading for me:

screenshot Screen Shot 2021-06-15 at 1 44 58 PM
stacktrace?

In the console, there is an error... but it seems to come before any data is fetched, so not sure whether it's related:

Uncaught TypeError: divRef.current is null
    VitessceWidget labplugin.js:277
    callCallback labplugin.js:6796
    invokeGuardedCallbackDev labplugin.js:6845
    invokeGuardedCallback labplugin.js:6907
    flushPassiveEffectsImpl labplugin.js:26389
    unstable_runWithPriority labplugin.js:29992
    runWithPriority$1 labplugin.js:14127
    flushPassiveEffects labplugin.js:26293
    enqueuePendingPassiveHookEffectUnmount labplugin.js:26325
    workLoop labplugin.js:29941
    flushWork labplugin.js:29914
    performWorkUntilDeadline labplugin.js:29681

labplugin.js:277 is the last line here:

   if (!divRef.current) {
      return function () {};
    }

    function handleMouseEnter() {
      var jpn = divRef.current.closest('.jp-Notebook');

      if (jpn) {
        jpn.style.overflow = "hidden";
      }
    }

    function handleMouseLeave(event) {
      var _event$relatedTarget$;

      if (event.relatedTarget === null || event.relatedTarget && ((_event$relatedTarget$ = event.relatedTarget.closest('.jp-Notebook')) === null || _event$relatedTarget$ === void 0 ? void 0 : _event$relatedTarget$.length)) return;
      var jpn = divRef.current.closest('.jp-Notebook');

      if (jpn) {
        jpn.style.overflow = "auto";
      }
    }

    divRef.current.addEventListener("mouseenter", handleMouseEnter);
    divRef.current.addEventListener("mouseleave", handleMouseLeave);
    return function () {
      divRef.current.removeEventListener("mouseenter", handleMouseEnter);

Confused about multiple `environment.yaml`: Are both needed?

The README currently says to install the top-level one, but when @keller-mark set me on the right path he pointed me at docs/notebooks/environment.yml... Should there be two? Either more explanation, or deleting one, may be helpful.

diff environment.yml docs/notebooks/environment.yml
1c1
< name: vitessce-jupyter-dev
---
> name: vitessce-jupyter-examples
6c6
<  - python==3.8
---
>  - python>=3.7
9a10,12
>  - numba>=0.53.0
>  - scanpy>=1.6.0
>  - jupyterlab>=3
11,14c14
<  - numcodecs>=0.7.2
<  - scipy>=1.0.0
<  - nbsphinx>=0.7.1
<  - pandoc>=2.11.3
---
>  - boto3>=1.16.30
17d16
<    - build==0.1.0
19,22c18
<    - sphinx
<    - sphinx-rtd-theme
<    - nbclean>=0.3.2
<    - generate-tiff-offsets>=0.1.7
---
>    - scikit-misc>=0.1.3

fromDict Should Handle Data Types with no URL

# Add each dataset from the incoming config.
for d in config["datasets"]:
new_dataset = vc.add_dataset(uid=d["uid"], name=d["name"])
for f in d["files"]:
new_file = new_dataset.add_file(
url=f["url"],
data_type=f["type"],
file_type=f["fileType"]
)

The above requires that the file object contains a url which is not necessarily true, for example, for raster data which have options not `url.

Add `set_props` Options to Docs

Right now it's not super clear from the docs that you can change titles etc. by setting the props of the component. We should have some sort of list or link to show users what components have what options, like observationsLabelOverride for Heatmap and Spatial etc.

Link from docs to github?

For example, https://vitessce.github.io/vitessce-python/notebooks/widget_brain.html is rendered from https://github.com/vitessce/vitessce-python/blob/master/docs/notebooks/widget_brain.ipynb, but there's no pointer from the docs page to the notebook, which would probably be useful for someone trying to run the examples.

There is a link at the top to the notebook JSON which I doubt is much use to anyone.

Proposal: Change link text from "View page source" to "Jupiter notebook", and update href accordingly.

Handling "Non-Standard" AnnData Zarr Data

Related to vitessce/vitessce#713, but this line for example

cluster_ids = adata.obs['CellType'].unique().tolist()
cell_cluster_ids = adata.obs['CellType'].values.tolist()

is for a custom part of the AnnData so we should probably not have it as a standard loadable object. Am I wrong? I can't find any documentation for CellType on the website. This part of the documentation lays out how to use __categories/MY_CATEGORY (which is part of the zarr store) but also has a reference to cell_type.

I am coming across this stuff as I write the JSON API for declaring parts of AnnData store for usage in Vitessce. It's tricky because it's not clear what we should have - for example, for spatial you might have something like

        {
          "type": "cells",
          "fileType": "anndata-cells.zarr",
          "url": "http://127.0.0.1:8081/habib.zarr",
          "options": {
            "obsm.spatial": "xy",
            "obsm.poly": "poly"
          }
        },

Where the correspondence between our JSON schema terminology and this config is one to one but then for something like

        {
          "type": "cell-sets",
          "fileType": "anndata-cell-sets.zarr",
          "url": "http://127.0.0.1:8081/habib.zarr",
          "options": {
            "obs.CellType": "sets"
          }

you don't have such a nice correspondence at which point you are dealing with "magic" strings.

I think the reason I am mentioning this hear is that we probably want to harmonize the terminology we use for custom parts of the store across languages. Thoughts?

correct way to store polygon coordinates in spatial_polygon_obsm?

Hi vitessce team,

I've got a dataset in AnnData format but the polygon vertices stored in obsm are giving me an error.

data

glom_signals : pd.DataFrame where columns are signals and rows are the associated segmentations, put into np.ndarray to represent expression matrix
glom_polys : list of np.ndarray where each row is a vertex of the polygon

what I attempted with vitessce-python

# initialize data.frames
obsm = pd.DataFrame()
obs = pd.DataFrame()
var = pd.DataFrame(index=glom_signals.columns)

# add data
obsm["Glomeruli Coordinates"] = glom_polys
obsm.index = obsm.index.astype(str)
obs["clustering"] = np.random.randint(0,5, len(glom_signals)) # just testing

# to AnnData
adata = AnnData(X = np.asarray(glom_signals), obsm=obsm)
adata.obsm["Glomeruli Coordinates"] = obsm["Glomeruli Coordinates"]
adata.obs["clustering"] = np.random.randint(0,5, len(glom_signals))

# vitessce configuration
vc = VitessceConfig(name='glom test', description='test glom')
dataset = vc.add_dataset(name='gloms').add_object(AnnDataWrapper(
        adata,
        cell_set_obs=["clustering"],
        expression_matrix="X",
        spatial_polygon_obsm="Glomeruli Coordinates"
    )
)

the error:

This appears to be related to writing the zarr store for the dataset and it not handling well the polygon data. I couldn't find much on how these should be arranged and none of the examples using AnnData contain such data unless I missed something.
The obsm data.frame contains the each polygon's vertices as a 2d np.ndarray and the dtype : object

Traceback (most recent call last):
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\utils.py", line 209, in func_wrapper
    return func(elem, key, val, *args, **kwargs)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\zarr.py", line 165, in write_array
    g[key][:] = value
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1122, in __setitem__
    self.set_basic_selection(selection, value, fields=fields)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1217, in set_basic_selection
    return self._set_basic_selection_nd(selection, value, fields=fields)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1508, in _set_basic_selection_nd
    self._set_selection(indexer, value, fields=fields)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1560, in _set_selection
    self._chunk_setitem(chunk_coords, chunk_selection, chunk_value, fields=fields)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1734, in _chunk_setitem
    self._chunk_setitem_nosync(chunk_coords, chunk_selection, value,
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1739, in _chunk_setitem_nosync
    cdata = self._process_for_setitem(ckey, chunk_selection, value, fields=fields)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1798, in _process_for_setitem
    return self._encode_chunk(chunk)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\zarr\core.py", line 1841, in _encode_chunk
    chunk = f.encode(chunk)
  File "numcodecs/vlen.pyx", line 103, in numcodecs.vlen.VLenUTF8.encode
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "C:\miniconda3\envs\writkelx\lib\site-packages\IPython\core\interactiveshell.py", line 3437, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-324-8645ab5c3736>", line 1, in <module>
    dataset = vc.add_dataset(name='gloms').add_object(AnnDataWrapper(
  File "C:\miniconda3\envs\writkelx\lib\site-packages\vitessce\config.py", line 151, in add_object
    obj.convert_and_save(self.dataset["uid"], len(self.objs))
  File "C:\miniconda3\envs\writkelx\lib\site-packages\vitessce\wrappers.py", line 417, in convert_and_save
    self._adata.write_zarr(zarr_filepath, chunks=[self._adata.shape[0], VAR_CHUNK_SIZE])
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_core\anndata.py", line 1969, in write_zarr
    write_zarr(store, self, chunks=chunks)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\zarr.py", line 54, in write_zarr
    write_attribute(f, "obsm", adata.obsm, dataset_kwargs)
  File "C:\miniconda3\envs\writkelx\lib\functools.py", line 875, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\zarr.py", line 71, in write_attribute_zarr
    _write_method(type(value))(f, key, value, dataset_kwargs)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\zarr.py", line 82, in write_mapping
    write_attribute(f, f"{key}/{sub_k}", sub_v, dataset_kwargs)
  File "C:\miniconda3\envs\writkelx\lib\functools.py", line 875, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\zarr.py", line 71, in write_attribute_zarr
    _write_method(type(value))(f, key, value, dataset_kwargs)
  File "C:\miniconda3\envs\writkelx\lib\site-packages\anndata\_io\utils.py", line 212, in func_wrapper
    raise type(e)(
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Above error raised while writing key 'obsm/Glomeruli Coordinates' of <class 'zarr.hierarchy.Group'> from <zarr.storage.DirectoryStore object at 0x000002521328C490>.

in `docs/notebooks/` separate data directories for separate examples?

Different notebooks have different prerequisites: pbmc3k_final.h5ad, habib17.processed.h5ad, snapatac/filtered_cell_by_bin.mtx, ... . Would it be useful to establish a subdirectory for each example? Or maybe that's overkill -- Is there already a convention that if it's a single file it goes on the top level, and if there are multiple files, then it gets a subdirectory?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.