flybrainlab / neuromynerva Goto Github PK

For hemibrain, only the one with adult(hemibrain) works anything else does not seem to work.
This prohibits the use of two hemibrain settings each connecting to a different processor.

• GFX window should have a button that allows user to specify iframe address.
• The front-end should work even if FBLClient is not installed.

Prior to start of Spring 2019 semester.

URGENT

• Docmanager
  • Integrate current save/load prototype into JupyterLab docmanager
• Model
  • Revisit signal logic especially as to model propagation without Neu3D widget.
• Multiwidget
  • Add support for multiple child widgets of same type with given master widget (and maybe even multiple master widgets?)
• Build Error
  • Fix error with jupyterlab --watch where package isnt found as well as build order for npm

Following

• DevOps
  • Make packages and updates with JupyterLab more automatic
• JupyterLab Update
  • Update and stress test JupyterLab 1.0.0 upon release, should have faster build times (hopefully)
• Comms
  • Comms still has some issues, look into this.
  • Also zmq issues specifically appearing post load, look into this...
• Neu3D
  • Neu3D Promise propagation is wrong and needs to be fixed to allow for saving/loading of camera state.
    • Can be done for now with a manual button (?)
• Save API
  • Make the Saving JSON extendable through an API for future child widgets similar to command registry maybe
• Warnings
  • Add more support for warnings during navigation to prevent loss of work

Adding capability to abort a large query to FFBOProcessor. Requires changes to crossbar settings.

In addition to removing all visualized neurons and synapses in the window, it should also fire a query to the backend to clear the state.

As title.

KernelChanged callback causes init client call to be fired for each widget. The client should be re-initialized only for each client, not all widgets.

Client initialization seems fine when creating the FBL Workspace for the first time.
However, when restarting the kernel using the reload button on the top of the Neu3D window, client initialization seems to repeat for multiple times. Here is a trace of what's executed from ipython history:

if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'LE', 'FBL-Comm:Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'vE', 'FBL-Comm:NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'LE', 'FBL-Comm:Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'vE', 'FBL-Comm:NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'LE', 'FBL-Comm:Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'vE', 'FBL-Comm:NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'LE', 'FBL-Comm:Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
fbl.widget_manager.add_widget('NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502', 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', 'vE', 'FBL-Comm:NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502')
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm,user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531'])
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm,user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502'])
if 'fbl' not in globals():    
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm,user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531'])
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm, user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502'])
if 'fbl' not in globals():
    import flybrainlab as fbl
   fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm, user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531'])
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm, user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502'])
if 'fbl' not in globals():
    import flybrainlab as fbl
   fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm, user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531'])
if 'fbl' not in globals():
    import flybrainlab as fbl
    fbl.init()
# if 'client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531' not in fbl.client_manager.clients:
_comm = fbl.MetaComm('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', fbl)
_client = fbl.Client(FFBOLabcomm=_comm, user='guest',secret='guestpass',ca_cert_file="isrgrootx1.pem",debug=False,url=u"ws://localhost:8081/ws",dataset="flycircuit",realm=u"realm1",)
fbl.client_manager.add_client('client-Neu3D-3-255ccf72-9ac8-4dd9-b1d6-be386dd46531', _client, client_widgets=['NeuGFX-3-a086cc16-f2b8-426f-a059-4056ed862502'])

This is confirmed on the processor side, I see 16 new sessions attached to the processor.

The text in the query bar is a bit difficult to read at times. Make it darker?

History:

@mkturkcan We should provide command palette and launcher commands for automatic creation and layouting of multiple widgets for each of the available datasets.

@TK-21st Initial working example in workspace branch, tracked in https://github.com/TK-21st/FBL-Wrapper/pull/16

Neu3D does not get disposed of correctly even after refresh, causing memory to build up in browser.

NeuroMynerva V2 aims at supporting connection to multiple servers and switching between datasets on a given server.

Nomenclature

We need to standardize nomenclature

• Crossbar Server: FBLServer ?
• NeuroArch Database: dataset?

Use Case 1

Use case 1 is focused on browser entry point, where user experience mimicks logging on to a data server with options of which available sessions to connect to.

1. After launching JupyterLab, the user should be able to click on a button to view currently available server backends, with metadata showing information such as ip address, and a list of available datasets
2. Ideally, there should be a way to test to see if a connection can/cannot be established with one of the servers at any given time.
3. In addition to making edits in a popup dialog, user should able to add/remove available dataset settings.
4. User should be able to easily change settings of the currently available server settings, primarily with regards to secrets, url, ssl, etc.
5. User should be able to export/import settings to/from JSON or INI file formats.

Also finished fetching data message from Neuroarchh server is not correctly shown.

@TK-21st The mouse event block does not seem to be working in the current implementation when resizing the window. Consider re-writing widget to not use IFrame.

@mkturkcan Caused by stopPropagation calls stopping global mouse events from firing in JupyterLab and missing css. Issue fixed in the short term, working on rewrite.

This issue is for tracking issues encountered while using build tools.

As title.

fblTracker tracks the correct number of widgets but on page refresh only one gets restored.

Can the scene.background be set by a parameter in the settings so it does not have to stay black?

Large queries hang the browser. could be a debounce issue.

This issue is meant for tracking issues encountered while running XKCD example.

At least in a setting where a processor has multiple datasets.
If I switch processor from hemibrain to flycircuit, the NLP query is still going to the hemibrain NLP server.

When using state restoration, I am getting this following error:

Uncaught DOMException: Failed to execute 'inverse' on 'SVGMatrix': The matrix is not invertible.
    at SvgPanZoom.handleMouseDown (webpack://NeuGFX/./node_modules/svg-pan-zoom/src/svg-pan-zoom.js?:444:97)
    at SVGSVGElement.mousedown (webpack://NeuGFX/./node_modules/svg-pan-zoom/src/svg-pan-zoom.js?:110:25)

The iframe shows up by there is no circuit diagram.

as title.

can use sessionContext._setBusy's API.

The compactness of the table (as compared to the long list in NeuroNLP) works well for me. But I think it would take less effort to navigation with scrolls than moving back and forth to click on the page buttons. What do you think?

FBL Widgets are associated with sessions (with or without kernel).

Current Approach

Currently the sessions are spawned and managed by the global (JupyterLab Application's) sessionManager instance, which allows for keeping track of FBL sessions in the running sessions SideBar tab.

Known Issues

However, there are a few issues:

1. Custom session type does not work well with RunningSessionManager. RunningSession tab only recognizes console and notebook types and there is no way to extend that.
2. It's currently rather difficult to check if a given session is FBL compatible. In the isFBLKernel function, we are relying on checking for comm target to see if a kernel is FBL compatible.

Desired Behavior

User should be able to know which sessionContext is associated with any given widget in an intuitive way.

Potential Remedies

1. Have a custom specs/session-Manager that is FBL specific. This way we can keep track of FBL compatible sessions separately from the rest of the console/notebooks.
   • What happens if we switch from FBL session to a non-FBL session? Would it be taken out of FBL's RunningSession manager?
   • What happens if we switch from non-FBL session to FBL session? Would it be taken out of the RunningSession manager?
2. Custom kernelSpec.
   • Issue could arise when user is changing current kernel/refreshing kernel etc.

Current Approach

    sessionContext = this.sessionContext =
      sessionContext ||
      new SessionContext({
        sessionManager: manager.sessions,   // <= manager = app.serviceManager
        specsManager: manager.kernelspecs,   // <= manager = app.serviceManager
        path,
        name: this.name,
        type: 'console',
        kernelPreference: {
          shouldStart: true,
          canStart: true,
          name: "python"
        },
        setBusy: options.setBusy
      });

Displaying some additional information (when they are available) would be useful, perhaps with a collapse button. Let's just standardize the summary block.

For a neuron:

1. Unique Name: uname,
2. Class: Neuron,
3. Type: name,
4. Datasource: data_source,
5. ID/Name in original dataset: referenceId,
6. Neurotransmitter info: 'Transmitters'
7. Additional info: info field which is a dict for additional annotations. OSN in hemibrain can be used for testing, which contains a dict like {target_glomerulus=DM3, receptors=[OR47a, OR33b], name=ab5B, co-receptors=[Orco], sensillum=basiconica, sensillum location=antenna}. This could include also publications and other links so these should be displayed as clickable items.
8. Morphometric info: extract from MorphologyData node.

For a synapse:

1. Unique Name: uname,
2. Class: Synapse,
3. Type: name,
4. Datasource: data_source,
5. presynaptic neuron: uname.split('--')[0]
6. postsynaptic neuorn: uname.split('--')[1]

As title.

Frontend should be aware of FFBOProcessor status. Possibly by polling.

After changing the processor, let's say from dataset A to dataset B, the neuropil meshes from dataset A is loaded (in the camera position specified by the processor of dataset B).

1. Neuron's name is displayed in info panel's pre/post synaptic partners, should be uname.
2. The textbox for filtering neuron names and number of synapse seems to be connected to the button for sorting the column. Once I click in the textbox, it changes sort order.
3. A button for adding/removing all listed neurons or synapses would be useful, to mirror the latest NeuroNLP webpages.
4. Use rid to add/remove neuron/synapse, rather than uname. All the servers have been updated to take rid queries. This is much faster, see hemibrain NeuroNLP webpage for comparison.
5. delete button in Neu3D window does not change +/- sign in the pre/post list.
6. Clicking on a synapse does not show anything in info panel. Should probably mirror what's in the hemibrain NeuroNLP webpage.

In order to access current ffboProcessor using the name of the processor before workspace is fully resolved, this._processor in NeuGFXWidget should be set directly by using the options.processor passed in.

The use case is to specify a custom URL from the processor settings for the NeuGFX iframe. Here is a hack in 14b5850. Might have better ways to go about it.

When switching kernel session, or starting widget with known kernel session, a common error is as follows:

default.js:987 Exception opening new comm
_handleCommOpen @ default.js:987
async function (async)
_handleCommOpen @ default.js:980
_handleMessage @ default.js:1123
(anonymous) @ default.js:94
Promise.then (async)
KernelConnection._onWSMessage @ default.js:91
default.js:99 Error: Object 'FBL-Comm:NeuAny-3-8fbc890b-fffc-4224-82eb-65afd8d8e418' not found in registry
    at default.js:1243
    at new Promise (<anonymous>)
    at Object.loadObject (default.js:1221)
    at KernelConnection._handleCommOpen (default.js:980)
    at KernelConnection._handleMessage (default.js:1123)
    at default.js:94

Reproducing

1. Create NeuAny Instance (NeuAny-12)

>> get_ipython().kernel.comm_manager.comms
{'d11b5847215348db8fed3caf076ca8d6': <ipykernel.comm.comm.Comm at 0x7fcac80ddd50>}
>> comm.comm_id
'd11b5847215348db8fed3caf076ca8d6'
>> comm.target_name
'FBL-Comm:NeuAny-12-e9f084d7-52eb-4a7e-9217-f24be2ee1d60'

2. Create another NeuAny Instance (NeuAny-13)

>> get_ipython().kernel.comm_manager.comms
{'9de245123b62460d93ad8bac68a5f22a': <ipykernel.comm.comm.Comm at 0x7fbd98713f90>}
>> comm.comm_id
'9de245123b62460d93ad8bac68a5f22a'
>> comm.target_name
'FBL-Comm:NeuAny-13-04d84a89-2394-4be8-b4a5-7660a4b225c1'

3. Switch NeuAny-12 to use NeuAny-13's sessionContext

default.js:987 Exception opening new comm
_handleCommOpen @ default.js:987
async function (async)
_handleCommOpen @ default.js:980
_handleMessage @ default.js:1123
(anonymous) @ default.js:94
Promise.then (async)
KernelConnection._onWSMessage @ default.js:91
default.js:99 Error: Object 'FBL-Comm:NeuAny-12-e9f084d7-52eb-4a7e-9217-f24be2ee1d60' not found in registry
    at default.js:1243
    at new Promise (<anonymous>)
    at Object.loadObject (default.js:1221)
    at KernelConnection._handleCommOpen (default.js:980)
    at KernelConnection._handleMessage (default.js:1123)
    at default.js:94

>> get_ipython().kernel.comm_manager.comms
{'9de245123b62460d93ad8bac68a5f22a': <ipykernel.comm.comm.Comm at 0x7fbd98713f90>}
>> [c.comm_id for c in comms.values()]
['9de245123b62460d93ad8bac68a5f22a', '3c9ae041be344ae49ca5f3ec7870361e']
>> [c.target_name for c in comms.values()]
['FBL-Comm:NeuAny-13-04d84a89-2394-4be8-b4a5-7660a4b225c1',
 'FBL-Comm:NeuAny-12-e9f084d7-52eb-4a7e-9217-f24be2ee1d60']

As title.

Otherwise, it is likely to overwrite the result in the notebook, the that previous query also return to res.

Widget session is not saved on restoration making double refresh lose the session.

The current practice follows private.count++, which is the borrowed from JLab v2 source code, and shares the same issue with JLab core.

JLab3 seems to have fixed the issue and should be looked at.

In JLab 3.4 the settings editor is very nice and seems like there's no need to use custom JSONEditor anymore.
The config button in master widget should just open up the default settings editor:

Error in the Client call most likely indicate a connection failure, which will yield subsequent queries fail in silence.
It would be helpful if exceptions raised in the code that execute in the background can be captured in the frontend, either as a pop-up message or in the notebook.

Some important considerations to address this week:

• JSONEditor requires a major overhaul. Possibly requires removal or redesign. In its current incarnation, it not interactive enough.
• Workspace saving/loading needs to be turned into buttons in the user interface.
• Try to integrate the upcoming version of Neu3D with performance improvements for low end machines.

From meeting of @mkturkcan @TK-21st and @MarkMazel on September 27th:

URGENT

• Model + Model Update
  • Currently seems to be duplicating large data and causing crashes, need to pass by ref
• Larva
  • Get ready for other species by adding Larva, requires new mesh json and camera/render settings
• InfoPanel
  • Fix info panel to only redraw when visible (find general way to do this for all widgets not just sidepanel)
• Memory Leak(?)
  • Use Chrome front-end memory management to search for possible mem-leak
• Build Error
  • Fix error with jupyterlab --watch where package isnt found as well as build order for npm

Following

• Floating Windows
  • Add floating window support specifically for Figures
• Loading ipynb
  • Ability to load existing notebook into fly brain lab (probably in launcher and maybe with .fbl extension)
• Multiwidget
  • Add support for multiple child widgets of same type with given master widget (and maybe even multiple master widgets?)

If initClient is called before a comm target registration callback is complete, any callback in the initialization will have no effect since comm is not found on kernel yet.

The following query should directly send Neuropil mesh data to Neu3D widget.

res = client.executeNAquery('query: [{'action': {'method': {'query': {}}}, 
                                                 'object': {'class': 'Neuropil'}}
                                    ],
                            'format': 'morphology'})

can be use that to show Neuropil meshes so Neuromynerva no longer needs to depend on separate, static files.

As title.

JLab V3 supports, among other things,

• Much faster build time with federated webpack
• Distribution as PyPI packages.

They sometimes appears and then disappears after resizing/closing the widget.

For development installation, add NPM script to force reinstallation
For NPM installation, add bash script to update package
Support updating of upstream packages like neu3d as well.

Very handy in NeuroNLP to sort through neurons/synapses but is missing in the NeuroMynerva.

Original Comment

It looks like fbl.widget_manager.last_active.model.states does not have updated info the second time I click on it. Like if I cleared the workspace, the neurons are still in the states.

I meant when I click on the button for the second time, the states are still the same as I click on it for the first time.

Problem Source

The callback fired when a mesh is added to a workspace is only updating the model.data field.

model.state is only updated after mouse interaction via this callback

This issue keeps track of documentations required for V2.

• Explain FFBO Processor concept
• More detailed explanation for data flow between widgets, kernel and processor
• Detailed examples of widget usage with gifs.