Giter VIP home page Giter VIP logo

fafbseg-py's People

Contributors

perlman avatar schlegelp avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

braincircuitsio tomka bjm5164 bocklab

fafbseg-py's Issues

Attribute error in `flywire.fetch_connectivity`

python version = 3.9
fafbseg version = 2.0.2

Code:

from fafbseg import flywire

a = flywire.fetch_connectivity(720575940623472716, clean = True,
                               min_score = 50, upstream = True, 
                               downstream = False, neuropils = True,
                              )

Error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[88], line 1
----> 1 a = flywire.fetch_connectivity(720575940623472716, clean = True,
      2                                min_score = 50, upstream = True, 
      3                                downstream = False, neuropils = True,
      4                               )

File ~/opt/miniconda3/envs/py39/lib/python3.9/site-packages/fafbseg/flywire/utils.py:120, in inject_dataset.<locals>.outer.<locals>.inner(*args, **kwargs)
    118 if disallowed and ds in disallowed:
    119     raise ValueError(f'Dataset "{ds}" not allowed for function {func}.')
--> 120 return func(*args, **kwargs)

File ~/opt/miniconda3/envs/py39/lib/python3.9/site-packages/fafbseg/flywire/synapses.py:1083, in fetch_connectivity(x, clean, style, upstream, downstream, proofread_only, transmitters, neuropils, filtered, min_score, batch_size, mat, progress, dataset)
   1080 neuropils = make_iterable(neuropils)
   1082 if len(neuropils):
-> 1083     filter_in = [n for n in neuropils if not n.startswith("~")]
   1084     filter_out = [n[1:] for n in neuropils if n.startswith("~")]
   1086     syn["neuropil"] = get_synapse_areas(syn["id"].values)

File ~/opt/miniconda3/envs/py39/lib/python3.9/site-packages/fafbseg/flywire/synapses.py:1083, in <listcomp>(.0)
   1080 neuropils = make_iterable(neuropils)
   1082 if len(neuropils):
-> 1083     filter_in = [n for n in neuropils if not n.startswith("~")]
   1084     filter_out = [n[1:] for n in neuropils if n.startswith("~")]
   1086     syn["neuropil"] = get_synapse_areas(syn["id"].values)

AttributeError: 'numpy.bool_' object has no attribute 'startswith'

module 'fafbseg.flywire' has no attribute 'set_default_dataset'

According to the documentation, setting the dataset to public requires fafbseg.set_default_dataset("public"). However, when I run it, I receive the error: module 'fafbseg.flywire' has no attribute 'set_default_dataset'. Here is my code:

from fafbseg import flywire
flywire.set_default_dataset("public")

Ideas for new features

Just to keep a running list of potential new features:

  • annotations:
    • pre-populate a type column (cell_type back-filled with hemibrain_type)
  • connectivity queries:
    • optionally add annotations (type, side, etc)
    • optionally collapse neurons into types
    • add a get_graph function (this could include a grow parameter to grow the graph by n hops from the query IDs)
    • add a function to download the entire graph - I never checked how long that would even take these days
    • for large queries add a sparse parameter to get_adjacencies which would return either the edge list or a sparse matrix

graceful failures in l2_skeleton

Via R but I guess it will be the same in python. 720575940452114071 is a very small body (2 supervoxels). This is annoying when you are skeletonising 200 neurons in a single call and one fails ...

> fp$flywire$l2_skeleton("720575940452114071")
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  AxisError: axis 1 is out of bounds for array of dimension 1

Detailed traceback:
  File "/Users/jefferis/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/fafbseg/flywire/l2.py", line 237, in l2_skeleton
    l2_eg = np.unique(np.sort(l2_eg, axis=1), axis=0)
  File "<__array_function__ internals>", line 180, in sort
  File "/Users/jefferis/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 1004, in sort
    a.sort(axis=axis, kind=kind, order=order)

l2 dotprops seems to fail when pca attributes missing for some points

From R, but I think this is general now that not all PCAs are calculate https://flywire-forum.slack.com/archives/C01M4LP2Y2D/p1646955614385879

> read_l2dp('720575940619695860')
Fetching L2 vectors:   0%|          | 0/233 [00:00<?, ?it/s]/Users/jefferis/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/caveclient/l2cache.py:78: UserWarning: L2Cache is in an experimental stage
  warnings.warn("L2Cache is in an experimental stage", UserWarning)
                                                                       Error in py_call_impl(callable, dots$args, dots$keywords) : 
  KeyError: 'pca'

Detailed traceback:
  File "/Users/jefferis/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/fafbseg/flywire/l2.py", line 494, in l2_dotprops
    vec = np.vstack([i['pca'][0] for i in this_info])
  File "/Users/jefferis/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/fafbseg/flywire/l2.py", line 494, in <listcomp>
    vec = np.vstack([i['pca'][0] for i in this_info])

Possible mismatch between Codex and fafbseg.flywire

Hi! First of all thanks for making this, it's been extremely useful and the docs are πŸ’―

I've been having an issue with a subset of my queries by cell type returning errors/being empty. What's surprised me is that for several of these, I can find them through Codex. I'm using materialization 783 for both as far as I can tell. I've attached a list of cell types for which I'm getting this problem.
issues.txt

Here's what I've been using to query (excuse the giant set of if/try/excepts at the end, I'm trying to make sure I don't miss anything obvious):

from fafbseg import flywire
from fafbseg.flywire import NeuronCriteria as NC
import logging
from contextlib import redirect_stdout


def get_from_codex(cell_type, regex=False):
    with redirect_stdout(None):
        filter_cell_type = NC(
            cell_type=cell_type,
            regex=regex,
        )
        result_cell_type = flywire.search_annotations(filter_cell_type)
    return result_cell_type


def get_from_hemibrain(cell_type, regex=False):
    with redirect_stdout(None):
        filter_hemibrain = NC(
            hemibrain_type=cell_type,
            regex=regex,
        )
        result_hemibrain = flywire.search_annotations(filter_hemibrain)
    return result_hemibrain


def get_neurons(cell_type):
    result_cell_type = None
    try:
        result_cell_type = get_from_codex(cell_type)
    except:
        logging.info(f"Could not find {cell_type} in Codex cell types")
    if result_cell_type is None:
        try:
            result_cell_type = get_from_hemibrain(cell_type)
        except:
            logging.info(f"Could not find {cell_type} in Hemibrain cell types")
    if result_cell_type is None:
        try:
            result_cell_type = get_from_codex(cell_type, regex=True)
        except:
            logging.info(f"Could not find {cell_type} in Codex cell types using regex")
    if result_cell_type is None:
        try:
            result_cell_type = get_from_hemibrain(cell_type, regex=True)
        except:
            logging.info(
                f"Could not find {cell_type} in Hemibrain cell types using regex"
            )
    return result_cell_type

Possibly relevant details:

>>> fafbseg.__version__
'3.0.8'
>>> navis.__version__
'1.6.0'

Grateful for any help!

Issue with get_skeletons Function

Hi!

First of all, thank you for creating this library. It has been extremely useful, and the documentation is excellent.

I am encountering an issue with the get_skeletons function. According to the documentation, this function should work for proofread root IDs 630 and 783. To ensure the neuron IDs are up-to-date, I use the is_latest_root function. Here is the relevant code snippet:

print(fafbseg.__version__)
flywire.is_latest_root([720575940627895056, 720575940507344965], timestamp='mat_783')

This returns:

3.0.9
array([ True,  True])

Based on this, I assumed I could use the get_skeletons function for both IDs. However, while the function works for the first ID, it fails for the second one. Here is the code and error message:

flywire.get_skeletons(720575940507344965, dataset=783)

HTTPError                                 Traceback (most recent call last)
Cell In[67], [line 1](vscode-notebook-cell:?execution_count=67&line=1)
----> [1](vscode-notebook-cell:?execution_count=67&line=1) flywire.get_skeletons(720575940507344965, dataset=783)

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:133](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:133), in parse_neuroncriteria.<locals>.outer.<locals>.inner(*args, **kwargs)
    [131](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:131)         kwargs[key] = nc.get_roots()
    [132](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:132) try:
--> [133](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:133)     return func(*args, **kwargs)
    [134](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:134) except NoMatchesError as e:
    [135](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/annotations.py:135)     if allow_empty:

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:675](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:675), in get_skeletons(root_id, threads, omit_failures, max_threads, dataset, progress)
    [672](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:672) root_id = np.int64(root_id)
    [674](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:674) try:
--> [675](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:675)     tn = navis.read_precomputed(f'{SKELETON_BASE_URL[str(dataset)]}[/](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/){root_id}',
    [676](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:676)                                 datatype='skeleton',
    [677](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:677)                                 info=SKELETON_INFO)
    [678](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:678)     # Force integer (navis.read_precomputed will turn Id into string)
    [679](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/fafbseg/flywire/skeletonize.py:679)     tn.id = root_id

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/precomputed_io.py:344](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/precomputed_io.py:344), in read_precomputed(f, datatype, include_subdirs, fmt, info, limit, parallel, **kwargs)
    [341](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/precomputed_io.py:341) else:
    [342](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/precomputed_io.py:342)     reader = PrecomputedMeshReader(fmt=fmt, attrs=kwargs)
--> [344](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/precomputed_io.py:344) return reader.read_any(f, include_subdirs, parallel, limit=limit)

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:771](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:771), in BaseReader.read_any(self, obj, include_subdirs, parallel, limit, attrs)
    [769](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:769) except TypeError:
    [770](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:770)     pass
--> [771](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:771) return self.read_any_single(obj, attrs)

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:668](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:668), in BaseReader.read_any_single(self, obj, attrs)
    [666](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:666)         return self.read_file_path(p, attrs)
    [667](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:667)     if obj.startswith("http://") or obj.startswith("https://"):
--> [668](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:668)         return self.read_url(obj, attrs)
    [669](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:669)     return self.read_string(obj, attrs)
    [670](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:670) if isinstance(obj, bytes):

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:563](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:563), in BaseReader.read_url(self, url, attrs)
    [553](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:553) # Note: originally, we used stream=True and passed `r.raw` to the
    [554](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:554) # read_buffer function but that caused issue when there was more
    [555](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:555) # than one chunk which would require us to concatenate the chunks
   (...)
    [560](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:560) # may have raised an exception earlier if the file was corrupted or
    [561](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:561) # the wrong format.
    [562](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:562) with requests.get(url, stream=False) as r:
--> [563](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:563)     r.raise_for_status()
    [564](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:564)     props = self.parse_filename(url.split('[/](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/)')[-1])
    [565](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/navis/io/base.py:565)     props['origin'] = url

File [~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1021](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1021), in Response.raise_for_status(self)
   [1016](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1016)     http_error_msg = (
   [1017](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1017)         f"{self.status_code} Server Error: {reason} for url: {self.url}"
   [1018](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1018)     )
   [1020](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1020) if http_error_msg:
-> [1021](https://vscode-remote+ssh-002dremote-002b10-002e243-002e83-002e35.vscode-resource.vscode-cdn.net/home/michaelshewarega/Desktop/test/~/miniconda3/envs/arkouda/lib/python3.11/site-packages/requests/models.py:1021)     raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://flyem.mrc-lmb.cam.ac.uk/flyconnectome/flywire_skeletons_783/720575940507344965

Btw I retrieved both IDs from the Codex Flywire website.

Does anyone know why this might be happening? Is it possible that the get_skeletons function is accessing an outdated dataset?

Thank you in advance for your help!

fafbseg.flywire.synapses.fetch_adjacency doesn't recognize lists as inputs

In fafbseg/utils.py, we have the following function:
def make_iterable(x, force_type = None)And when passing a list, utils.make_iterable is called in `fafbseg/flywire/utils.py:


 if isinstance(x, navis.BaseNeuron):
    ids = [x.id]
 elif isinstance(x, navis.NeuronList):
    ids = x.id
 elif isinstance(x, (int, [np.int](http://np.int/))):
    ids = [x]
 else:
    ids = utils.make_iterable(x, dtype=np.int64)

But an error occurs, because the parameters do not match the function declaration.

Outsource some of the FlyWire functions to a separate package

Maybe navis-graphene? That way we can more easily reuse code for FANC.

Alternatively, we could widen the scope of fafbseg and rename it to e.g. navis-emdatasets with modules for flywire, fafb, fanc and hemibrain.

navis-flyem, navis-fly, navis-flydata

Failure to complete significant API calls, e.g. update 3+ neuron ID at same time or flywire.get_synapses()

Background
Hi. I have been at this for weeks now, without resolving the issue. I have tried multiple python versions, IDEs, networks/ethernet, generated new flywire tokens, conda environments... everything I can think of. I am on Mac OS X 14.2.1 (Sonoma) on a MacBook Pro M3 Max, using Pycharm to develop in a Python 3.10 virtual miniconda environment, with fafbseg 3.0.2, CAVE client 5.17.2, When using the newest python/fafbseg, I'd had some quirky issues with installation dependencies due to using a Mac with a Silicon chip, and so I downgraded to the versions my lab colleague uses successfully for flywire. I have an account with flywire & the codex, but do not seem to have production dataset access, but I just set the default database to public for now.

Goal
Anyway, I have been attempting to write a script which gets a synapse list for a specific neuron ID, and then compares it to lists of neurons I have by cell type, in order to produce a basic graph of what cell types this neuron projects to. Eventually I will do a ton of other stuff, but I have been stuck at just getting the synapse list.

What works and what doesn't
I am able to successfully update the ID for 1-4/5 neurons at one time with the update_ids() function. However, if I try to update more than 5 at once using a list, I get the attached timeout error:
ERROR_update_ids.txt

I am also able to use the. get_mesh_neuron() function successfully, though I don't actually need the function.

I am entirely unable to get a response for the get_synapses(), get_connectivity/synapses.fetch_connectivity(), get_transmitter_predictions(), or get_synapse_counts() functions. I have attached the typical error I get when attempting to run the following line (regardless of any combination of parameters):
ERROR_get_synapses.txt

fafbseg.flywire.get_synapses($neuronID, downstream=False, min_score=30)

Since the two errors are extremely similar, I'm thinking the error has something to do with connectivity..? Please let me know if anyone has seen this error before and/or what I could do about it. I have also attached a trimmed down version of my script below. Thanks!

script_for_help.txt

-- Mason Weinstock

flywire/skeletonize.py possibly outdated

The use of the skeletonize_neuron() function in flywire/merge.py does not match its definition in flywire/skeletonize.py . The function definition has the input variable remove_soma_hairball and returns one variable, while the function usage in merge.py (line 88) has the input variable drop_soma_hairball and expects 3 variables returned.

OverflowError: Python int too large to convert to C long

when I read the tutorials to fetche the neuron as mesh and then skeletonizes it in https://fafbseg-py.readthedocs.io/en/latest/source/tutorials/flywire_neurons.html, but the program reported an error OverflowError: Python int too large to convert to C long ，The code is all from the official website tutorial. I remember a few months ago when I wanted to implement the same code to obtain the skeleton, it was very smooth. I don't know what happened now, and I sincerely request your help

import navis
from fafbseg import flywire
sk = flywire.skeletonize_neuron(720575940617774213, progress=False)
Skeletonizing: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 232440/232440 [00:01<00:00, 214595.25it/s]
Traceback (most recent call last):
  File "D:/Codes/230308-point-visualize/0417_get_flywire_swc.py", line 14, in <module>
    sk = flywire.skeletonize_neuron(720575940634710629)
  File "C:\Users\Administrator\miniconda3\lib\site-packages\fafbseg\flywire\skeletonize.py", line 204, in skeletonize_neuron
    if is_materialized_root(id):
  File "C:\Users\Administrator\miniconda3\lib\site-packages\fafbseg\flywire\utils.py", line 388, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\Administrator\miniconda3\lib\site-packages\fafbseg\flywire\annotations.py", line 140, in is_materialized_root
    id = make_iterable(id, force_type=int)
  File "C:\Users\Administrator\miniconda3\lib\site-packages\fafbseg\utils.py", line 124, in make_iterable
    return np.asarray(x, dtype=force_type)
OverflowError: Python int too large to convert to C long

The same ID was okay when I obtained the mesh

n = flywire.get_mesh_neuron(720575940617774213)
print(n)

it succeeded

type            navis.MeshNeuron
name                        None
id            720575940617774213
units                1 nanometer
n_vertices                265599
n_faces                   532059
dtype: object

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.