Giter VIP home page Giter VIP logo

q2-coordinates's Introduction

q2-coordinates

CI DOI

A qiime2 plugin supporting methods for geographic mapping of qiime2 artifact data or metadata.

q2-coordinates makes it easy to plot geographic coordinates and associated (meta)data on beautiful topographic, street maps or interactive geographical maps. Map tiling, resolution calculation, and coordinate projection occur automatically. All the user needs to do is input a two-dimensional list of geocoordinates to plot, in decimal degrees, as shown in the examples below.

Currently, StamenTerrain, Open Street Maps, Google Maps and OpenLayers are supported, producing high-quality maps from anywhere on planet Earth. Map projections are quick for small maps, but may take several minutes for very large maps (e.g., trans-continental). Additionally, quadtree functionality allows the user to objectively partition a dataset based on x and y coordinate information (for example, longitude and latitude).

Install

We recommend using the functionalities in a conda environment with the required dependencies installed within:

conda create -y -n q2coord
conda activate q2coord

conda install -y \
  -c conda-forge -c bioconda -c qiime2 -c udst -c defaults \
  qiime2 q2cli q2templates q2-types q2-diversity \
  pysal==2.1.0 cartopy==0.19 matplotlib geopy dill geopandas \
  pandana urbanaccess tzlocal==2.1

pip install git+https://github.com/bokulich-lab/q2-coordinates.git

DEV-only note:

Until QIIME 2 2021.8 is officially released, replace -c qiime2 in the command above with -c https://packages.qiime2.org/qiime2/2021.8/staged to fetch the latest dev version instead.

Examples

In the examples below we will use some bacterial 16S rRNA gene amplicon sequence data collected from Californian vineyards, as described by Bokulich et al. 2016. ๐Ÿ˜Ž๐Ÿท

Plotting geocoordinates colored by alpha diversity values

This visualizer takes a SampleData[AlphaDiversity] artifact and sample metadata TSV as input, and plots sample coordinates on the built-in maps. Sample points are colored as a function of alpha diversity values.

Clone into repository and get access to the test data:

git clone https://github.com/bokulich-lab/q2-coordinates.git

cd q2-coordinates/q2_coordinates/tests/data/

Draw map of alpha diversity values:

qiime diversity alpha \
    --i-table even_table.qza \
    --p-metric observed_features \
    --o-alpha-diversity alpha_diversity_sample.qza

qiime coordinates draw-map \
    --m-metadata-file alpha_diversity_sample.qza \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-latitude latitude \
    --p-longitude longitude \
    --p-column observed_features \
    --o-visualization diversity-map.qzv

Alt text

Draw interactive map of alpha diversity values:

qiime coordinates draw-interactive-map \
    --m-metadata-file alpha_diversity_sample.qza \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-latitude latitude \
    --p-longitude longitude \
    --p-column observed_features \
    --o-visualization diversity-interactive-map.qzv

Note that any metadata-transformable artifacts can be used as a metadata-file input to this command, so this opens the door to many other data types, e.g., PCoA results, predictions, etc.

HINT: If you are not sure what the column name is for your artifact of interest, use qiime tools inspect-metadata to see the available column names.

Plotting geocoordinates colored by metadata category values

We can use the same visualizer action for plotting alpha diversity values to color sample points as a function of continuous or categorical sample metadata. To do this, we simply add the "category" parameter to use that category from the sample metadata instead of alpha diversity values. The plot below shows the various vineyard sites (indicated by anonymous numbers) where samples were collected.

qiime coordinates draw-map \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-column vineyard \
    --p-latitude latitude \
    --p-longitude longitude \
    --p-discrete \
    --o-visualization vineyard-map.qzv

Alt text

Converting (geo)coordinates to a distance matrix

This is a cinch with q2-coordinates! To calculate geodesic distance from geocoordinate data, use the geodesic-distance method:

qiime coordinates geodesic-distance \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-latitude latitude \
    --p-longitude longitude \
    --o-distance-matrix geodesic_distance_matrix_sample.qza

This computes geodesic distance (in meters) between each point. Note that samples with missing values are ignored.

We can also construct a distance matrix from 2D or 3D spatial coordinates using the euclidean-distance method:

qiime coordinates euclidean-distance \
    --m-metadata-file xyz-coordinates.tsv \
    --p-x x \
    --p-y y \
    --p-z z \
    --o-distance-matrix xyz_distance_matrix_sample.qza

We can use these distance matrices for other useful QIIME 2 methods, e.g., to compute a mantel test comparing two different distance matrices. For example, we can compare Bray-Curtis dissimilarities between microbial communities to geospatial distances between our vineyard samples:

qiime diversity beta \
    --i-table even_table.qza \
    --p-metric braycurtis \
    --o-distance-matrix bray_curtis_distance_sample.qza

qiime diversity mantel \
    --i-dm1 geodesic_distance_matrix_sample.qza \
    --i-dm2 bray_curtis_distance_sample.qza \
    --p-intersect-ids \
    --o-visualization mantel_sample.qzv

Computing autocorrelation statistics

Spatial autocorrelation measures the similarity of a measurement taken across space. Correlations can be either positive, which indicates similar values in adjacent spaces, or negative, which indicates that dissimilar measurements are evenly arranged across space. In both cases, autocorrelation indicates that the observed pattern is non-random. We can compute Moran's I and Geary's C autocorrelation tests based on a spatial distance matrix using the autocorr visualizer.

qiime coordinates autocorr \
    --i-distance-matrix geodesic_distance_matrix_sample.qza \
    --m-metadata-file alpha_diversity_sample.qza \
    --m-metadata-column observed_features \
    --p-intersect-ids \
    --o-visualization autocorrelation_sample.qzv

Moran's I ranges from -1 (negative spatial autocorrelation) to 1 (positive spatial autocorrelation); values near 0 or the expected I (EI, which approaches 0 with large sample sizes) indicate a random spatial distribution. Geary's C ranges from 0 (positive spatial autocorrelation) to some unspecified value greater than 1 (negative spatial autocorrelation); values near 1 indicate a random distribution. Both are global autocorrelation tests, though Geary's C is much more sensitive to local autocorrelation processes. The accompanying Moran plot shows the relationship between the variable of interest and its own spatial lag (i.e., the degree to which neighboring observations are autocorrelated).

Using quadtrees

The quadtree function splits the region of points into bins recursively based on a threshold. If the number of samples in a section is larger than the threshold they will split into four, allowing detailed use, description, and partitioning of space based on sample density.

qiime coordinates quadtree \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-y-coord latitude \
    --p-x-coord longitude \
    --p-threshold 20 \
    --output-dir test_quadtree

Visualizing quadtrees

Quadtrees (of qiime2 type SampleData[QuadTree]) can easily be integrated into downstream analyses with q2-coordinates and other QIIME 2 plugins, and visualized using other qiime2 plugins (e.g., q2-empress).

  1. To display the sample positions and color-code them by respective quadrant with a split-depth of 1 you can use q2-coordinates draw-map (or draw-interactive-map) as:
qiime coordinates draw-map \
    --m-metadata-file test_quadtree/output_table.qza \
    --m-metadata-file chardonnay_sample_metadata.txt \
    --p-latitude latitude \
    --p-longitude longitude \
    --p-column split-depth-1 \
    --p-discrete \
    --o-visualization test_quadtree/quadtree-map-depth1.qzv

Alt text

  1. To view and navigate the quadtree use q2-empress that allows you to see the number and size of splits. Beware, that you should have the qiime2 empress plugin installed as described here.
    qiime empress tree-plot \
    --i-tree test_quadtree/output_tree.qza \
    --m-feature-metadata-file chardonnay_sample_metadata.txt \
    --output-dir test_quadtree/empress

License

q2-coordinates is released under a BSD-3-Clause license. See LICENSE for more details.

q2-coordinates's People

Contributors

adamovanja avatar antgonza avatar gregcaporaso avatar lenafloerl avatar misialq avatar nbokulich avatar zoechallacombe avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

q2-coordinates's Issues

add distance scale to maps

automatically adjust scale to sensible unit, size, and intervals (e.g., 1 km intervals)

automatically locate to unused corner or outside of plot beneath legend.

BUG: `draw-*map`: `column` should be optional

column should be optional in draw-interactive-map and draw-map, i.e., if no column is selected, all points should be colored uniformly. Currently, it is optional but omitting a column raises an error:

Test data: metadata.txt

Command:

qiime coordinates draw-interactive-map \
    --m-metadata-file metadata.txt \
    --p-latitude lat \
    --p-longitude lon \
    --o-visualization map.qzv

Full traceback:

Traceback (most recent call last):
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/qiime2/sdk/action.py", line 452, in _callable_executor_
    ret_val = self._callable(output_dir=temp_dir, **view_args)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/q2_coordinates/mapper.py", line 129, in draw_interactive_map
    ['column', 'latitude', 'longitude'], missing_data)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/q2_coordinates/_utilities.py", line 27, in _load_and_validate
    metadata = metadata[columns]
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/pandas/core/frame.py", line 2912, in __getitem__
    indexer = self.loc._get_listlike_indexer(key, axis=1, raise_missing=True)[1]
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/pandas/core/indexing.py", line 1254, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "/Users/nbokulich/miniconda3/envs/q2coord/lib/python3.6/site-packages/pandas/core/indexing.py", line 1304, in _validate_read_indexer
    raise KeyError(f"{not_found} not in index")
KeyError: '[None] not in index'

However, selecting a column causes this to work. E.g., command:

qiime coordinates draw-interactive-map \
    --m-metadata-file metadata.txt \
    --p-latitude lat \
    --p-longitude lon \
    --p-column language \
    --o-visualization map.qzv

Looks like draw-map raises the same error as draw-interactive-map so column is not being handled appropriately by either...

Interactive map is sometimes missing

Steps to reproduce:

  • use the attached metadata file to draw an interactive map as follows:
    qiime coordinates draw-interactive-map \ 
       --m-metadata-file meta_sample.txt \           
       --p-latitude Latitude \
       --p-longitude Longitude \
       --p-column Depth \            
       --o-visualization test-viz.qzv 
    
  • open the visualization

Expected behaviour:
Map is displayed correctly.

Actual behaviour:
Map is missing - see the screenshot.
Screen Shot 2022-01-25 at 13 33 26

Do quadtrees need a new transformer?

This is a followup question/issue related to #10

I note that the quadtree is output as a Phylogeny[Rooted], but the README shows some provenance-breaking reformatting necessary to use with q2-empress. Is a new transformer needed? Or why is this reformatting not just done in the first place?

If reformatting is necessary for this to be used as a normal Phylogeny[Rooted], then a quadtree should have its own special type (maybe Phylogeny[quadtree]), and we can add a transformer or an action to perform this conversion.

Any thoughts @zoechallacombe @wasade ? Thanks!

refactor readme

The readme is quite outdated. Should be updated to:

  1. patch up the text and visuals
  2. include links to view/download example data/qzvs
  3. some methods need a little more description (or links to better description), e.g., for quadtrees.

choropleth maps

Choropleth maps would be a useful feature, e.g., for plotting average values within a region, or even density of sampling within a region.

input: metadata
required params: latitude, longitude
output: interactive chloropleth map

optional inputs: metadata column to average per area [otherwise plot density by default]

Some examples and relevant projects:
https://plotly.com/python/choropleth-maps/
https://github.com/pysal/mapclassify
https://geopandas.org/en/stable/community/ecosystem.html

update installation instructions

they point to nbokulich/q2-coordinates instead of bokulich-lab/q2-coordinates

also the dependencies, etc might need updating.

adjust map size

they should be larger, and dimensions should be fixed, or at least fixed height? Currently fixed width (I think), which results in lots of white space if maps are longer than wide.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.