navis-org / pymaid Goto Github PK

See here: https://marcobonzanini.com/2015/02/25/fuzzy-string-matching-in-python/

Use app.use(app='PyQt5') to set default - otherwise will throw warning that other backends have not been detected

Should be ~10x speed up

Also: test the tests (it's been a while)

Kaleido is an algorithm to optimise assignment of random colours using the neurons' average distance to each other. See "Kaleido: Visualizing Big Brain Data with Automatic Color Assignment for Single-Neuron Images" by Ting-Yuan Wang et al. (https://link.springer.com/article/10.1007%2Fs12021-018-9363-3).

• would need wrappers to dump neuron morphology, connectors, volumes and dotprops in Blender ( .plot_b3d() ?)
• could keep objects/collections as classes with wrappers to change basic parameters (color, bevel, size (for connectors), layers, visibility, transparency), select or unselect, and remove objects
• wrappers for clustering

E.g. to quickly resize a volume to be a bit more generous.

On-demand attributes

1. reconstruction_time
2. review_time
3. reconstruction_by_users
4. review_by_users

Currently all user-assigned properties are purged when using .copy on CatmaidNeurons/Lists. Could easily fix this by copying the whole __dict__ instead of only selected attributes.

Uses scipy to test if points are within a given CATMAID volume. The idea is to test if adding the point to the cloud would change the convex hull.

Doesn't this implicitly assume that the volume is a convey hull? Think about the convex hull of the mushroom body which looks a bit like a pyramid. It seems to me that a point could be inside the pyramid but far outside the alpha shape defining the mushroom body and still pass this test.

1. For each connector get the distance to its treenode. Use this as radius.
2. Interpolate radius for treenodes without connectors.
3. Maybe incorporate neuron morphology (e.g. radius should not decrease with strahler number).

Should work like a charm for parts of the neuron that have high density of connectors (see e.g. GF1 DB GMC SM #4947529).

Would have implement a volume plot mode.

1. Calculate connectivity distance matrix
2. Cluster into simplices [see Giusti et al., Journal of Computational Neuroscience (2016)]
3. Collapse network

This requires not only a thin wrapper for morpho.in_volume but also removal of connectors and more importantly closing of branches that don't have a parent anymore. There may be quite a few functions that act up when there is more than one root node.

Examples:

1. Pulling neurons with annotation X, then adding that annotation to another neuron (in CATMAID) will not change subsequent queries from pymaid as now outdated cached response is used
   2.CatmaidNeuron.reload() also uses cached data. Is this working as intended?
2. Querying for non-existent annotations throws an error. Even if that annotation is added, will still fail due to cached response.

Possible solutions:

• Exclude some urls (annotations, tags, etc.) from caching. Most definitely for everything that writes to the CATMAID database (Thanks for flagging, Greg!). Could use a @no_cache decorator.
• Check if error is raised before caching server response (e.g. if annotation is not found)
• Allow relevant functions (e.g. .reload) to overwrite caching=True decorator
• Allow relevant functions to force cache update
• Add clear_cached_response() analogous to get_cached_response()
• Raise Exception/Warning if data might be outdated.
• Add pymaid.clear_cache() function and refer to it in Cached data used message.

Make get_partners and adjacency_matrix use actual connectors of (e.g. pruned or split) neurons passed as sources/targets.

Could simply implement a low level function that extracts adjacency matrix from get_connector_details.

It appears as if the n not in exc_sel query in selection = [n for n in inc_sel if n not in exc_sel] is now taking forever.

One problem appears to be that __eq__ asks for soma and root which will trigger fetching skeleton data if not available. But even with loaded data, it takes forever.

• mapping between CATMAID and tile coordinates
• basic cutting and stitching (PIL?)

Need some testing but TriMesh is pure Python (unlike PyOctree which causes problems when compiling on some systems).

TriMesh.contains() to check if points are contained in volume.
TriMesh.volume to return volume

Not just CatmaidNeuron/Lists

Should in these cases simply assign the successor branch's strahler order to the "not a branch" branch.

Add plot3d function to cluster.clust_results class. Could take k_clusters or threshold parameter and colour neurons accordingly.

Confidence:
Confidence along the arbor: go from root to all leafs, multiply low confidences on along the way. Make this part of .plot2d and .plot3d.

Edge risk:
In CATMAID this is calculated by (a) taking the lowest common ancestor for all synapses making up a connection and then (b) checking the fraction of total the synapses that would be "lost" if that branch was pruned off -> if all relevant synapses are clustered close together at a distal branch, cutting them off has little impact on the total number of synapses -> high risk.

risk = 1 - lost_inputs / total_inputs

Could somehow combine this with confidence.

Add new function prune_by_strahler() to pymaid.morpho and link to core.CatmaidNeuron

Also thinking about: prune_distal_to() and prune_proximal_to() for neuron objects. Would just be a thin wrapper for morpho.cut_neuron()

find_skid is essentially a visualisation of a neuron's history based on logs. Could add this functionality as part of pymaid.user_stats module?

Calculate overlap based on cable within X um distance for sets of neurons.

Currently, neurons are processed sequentially. Querying intersection for all points at once might could potentially speed up the whole process.

E.g. flow centrality, axon-dendrite split, etc.

• make .plot3d return custom class that allows adding/removing of neurons
• make .plot3d search for existing vispy canvases

Also: rotate object by default

Installation of rpy2 fails if R ist not already installed

Would show annotations used by given user(s) sorted by number of neurons annotated. Also integrate this with catbot (https://github.com/flyconnectome/catbot)

Thanks for this great library for interfacing with CATMAID.

It would be great if I could include a single module (e.g. pymaid.fetch) without the other modules. This is currently not possible due to pymaid/__init__.py automatically importing all of the submodules.

The full set of modules can take a while to load on my machine, and the initialization of VisPy leads to the spawning of a GUI application even if no visualization routine is called.

Just copy paste code from get_urls_threaded

• allow resample to a node every N microns
• add .simple attribute to CatmaidNeuron/List that generates & stores a simplified version of the neuron(s)

A minimal example:
nl = pymaid.get_neuron('annotation:glomerulus DA1')
nl.resample(1000, inplace=True)
res = rmaid.nblast_allbyall( nl )

Additionally, my understanding is, since the nblast algorithm looks for cloud vectors in partner neurons, resample is alway preferred for nblasting since otherwise nodes with uneven intervals could mess up interpretation of distance between pairs.

See here for an example.

This would cut decoding time e.g. for an average neuron from 22ms to 8ms.

ujson library seems to compile just fine on OSX.