apartresearch / deepdecipher Goto Github PK

It is not necessary to explicitly store which services are available for which models.
Instead, this can be calculated from which data objects a model has available.
To do this, each service provider must provide a list of necessary data objects.
When receiving a service API call, first check that the model has the necessary data objects, and if not, respond with an error message saying which are missing.
When receiving an all API call, the response JSON should contain an object for all services in the database, with the before mentioned error message for the unavailable ones.

This should at least consist of metadata and neuron suggestions based on some interestingness measure.
Depends on:

• #60

This is the object through which the data directory is manipulated

Deadline is 29/08/2023

• Get the server up and running again.
• Case studies.
• Add Neuroscope data for all 25 neuroscope.io models.
• Add Neuron2Graph data for as many models as possible.
• Measure response time for single requests.
• Stress test server against high traffic.
• Have someone reproduce some of our data and compare it with what DeepDecipher shows.
• Write deeper methodology section describing what we did and not only how to use it.
• Stitch together rebuttal.
• Submit rebuttal!

Make the gifs render for longer and show one full example per method

Actix web has a great guide on testing in their frame work here.

• #57
• #60
• #66
• #58
• #59
• #9
• #67
• #68
• #64
• #72
• #73
• #74
• #75

Should cover everything necessary to build package and start server

Probably using Sphinx

Should show model metadata and available services

Get people with some mech-interp experience to take a look around the website and say what the like/don't like.
Show some people the index site and ask them to do specific things and think aloud while doing so.

Server needs logging.

Should use log, env_logger, and multi_log crates.

Currently, the backend can't take more than 20 requests a second, even for neuron pages without N2G. This is not nearly good enough.

Depends on:

• #13

Depends on:

• #34

Current frontend is very basic. One of our main selling points over e.g. Neuroscope is that our site is supposed to be nicer to look at and easier to use. Lets actually deliver on that.

Not much point in working on this before all (MVP) parts of the frontend are done, so depends on:

• #66
• #58

Possibly use utoipa

Should we ask for permission to use their data in this way? What is the license?

E.g. activation kurtosis

Create a script that takes a model (and probably some data) and calculates an interestingness measure for each neuron in the model and stores it as an array.

Neuroscope service can of course only use data objects of type neuroscope, but the name shouldn't be forced to be neuroscope.

A data object requires certain rows to exist in tables model_data, layer_data, and neuron_data. This is not ensured through transactions, so it should at least be possible to validate after the fact.

Strings were used to make initial implementation simpler, but this can cause issues in the long run.

Should be possible for

• Neuroscope neuron pages

This may not be worth it, but worth looking into if storage space becomes a major issue