allenneuraldynamics / aind-bci-analysis Goto Github PK
View Code? Open in Web Editor NEWTools to analyze BCI data
aind-bci-analysis's People
Contributors
Watchers
aind-bci-analysis's Issues
Data issues collected
Folder: https://drive.google.com/drive/folders/1scasz384Ma7cAEtlOQLPBVPnHfRTINqU
-
roiX and roiY are not of the same size. roiX is (x,) and roiY is (x+-1, 2). The second axis is just 0, its just a placeholder. Maybe its best of combine them to have an roi column.
https://github.com/kpdaie/BCI_analysis/issues/4 -
BCI11 data, conditioned neuron index go upto 240++, however, there are only 98 neurons, there is some issue with the numbering and indexing.
-
Sometimes the data has conditioned_neuron_name as 'ROI,2' and not 'ROI 2', this issue occurred while looking at BCI_26 040522 and using martons code to find cn, A fix is added in
https://github.com/kpdaie/BCI_analysis/pull/11 -
weird trial_num when a day has >1 sessions, the trial_num variables are stacked together. look at bpod_zaber_data['scanimage_file_names"] and don’t take files that have no scanimage equivalent.
-
DLC_data BCI_26 not there for all sessions, and some sessions have multiple DLC's however excel sheet shows only one session
Updaate 062922
- frame_times_rel0 need not be continuous. For eg in 041822, it1 runs from 12 to 23 seconds and it2 from 14072 to 14083. Not sure how to fix it right now.
Fix docstring conventions
There are multiple warnings about the docstrings when using sphinx to automatically create documentation:
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.suite2p_to_npy:5: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.align_trace_to_event:1: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.suite2p_to_npy:36: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.suite2p_to_npy:5: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.suite2p_to_npy:39: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.trial_times_to_session_indices:21: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.sessionwise_to_trialwise:1: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/io/io_suite2p.py:docstring of bci_analysis.io.io_suite2p.trial_times_to_session_indices:29: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_align.py:docstring of bci_analysis.pipeline.pipeline_align.get_aligned_data:6: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_align.py:docstring of bci_analysis.pipeline.pipeline_align.collapse_dlc_data:1: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/pipeline/pipeline_align.py:docstring of bci_analysis.pipeline.pipeline_align.get_aligned_data:22: WARNING: Definition list ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/pipeline/pipeline_align.py:docstring of bci_analysis.pipeline.pipeline_align.get_aligned_data:38: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_pybpod_files:22: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_pybpod_files:17: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_pybpod_files:26: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_single_pybpod_session:25: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_single_pybpod_session:28: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_pybpod_files:23: WARNING: Block quote ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/pipeline/pipeline_bpod.py:docstring of bci_analysis.pipeline.pipeline_bpod.export_pybpod_files:25: WARNING: Definition list ends without a blank line; unexpected unindent.
BCI_analysis/src/bci_analysis/pipeline/pipeline_imaging.py:docstring of bci_analysis.pipeline.pipeline_imaging.find_conditioned_neuron_idx:9: ERROR: Unexpected indentation.
BCI_analysis/src/bci_analysis/pipeline/pipeline_imaging.py:docstring of bci_analysis.pipeline.pipeline_imaging.find_conditioned_neuron_idx:6: WARNING: Block quote ends without a blank line; unexpected unindent.
add new entries (f0 scalar, photon counts etc)
[BCI_analysis/BCI_analysis/io/io_python.py](https://github.com/kpdaie/BCI_analysis/blob/420c5b4b14b39270f2c1cd6db076fff548c0b6b5/BCI_analysis/io/io_python.py#L4)
Line 4 in [420c5b4](https://github.com/kpdaie/BCI_analysis/commit/420c5b4b14b39270f2c1cd6db076fff548c0b6b5)
def read_sessionwise_npy(file_path):
multi-session .mat files ROI information problem (v8)
Currently the .mat file has the following fields:
['all_si_filenames', 'closed_loop_filenames', 'cn', 'dff_sessionwise_all_epochs', 'dff_sessionwise_closed_loop', 'dff_trialwise_closded_loop', 'dist', 'epoch_name', 'epoch_number', 'f_sessionwise_closed_loop', 'f_trialwise', 'file', 'mean_image', 'mouse', 'mouse_name', 'roiX', 'roiY', 'sessionDate', 'time_from_trial_start']
The dimensions of 'roiX' and 'roiY' are not consistent:
data['roiX'][0].shape = (215,)
array([428. , 512.88888889, 534. , 586.70833333, .....])
data['roiY'][0].shape = (214,2)
array([[ 0. , 580. ],
[0. , 526.5],
....... ]])
The number of cells is 215 (len(data['dist'])), so neither of them is correct.
dff_trialwise_closed_loop TypeError due to typo in mat field #9
The field "dff_trialwise_closed_loop" has a typo in it and instead says "dff_trialwise_closded_loop" -- I am able to temporarily remedy this error on my end by just changing the dictionary key in the io_matlab file to match the typo
Fixed in the new python structure. Commenting here for to make sure to change our scripts later when we use python data
Missing Fields in mat files and int() type error with reading BCI12
-
BCI11 is missing the field " dff_trialwise_closed_loop - float [sessions x time points x neurons x trials] "
-
When running BCI12_030222v8.mat through io_matlab.read_multisession_mat(), a type error pops up:
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
This error occurs at line 79 in io_matlab.read_multisession_mat() when defining a key in the data dicitonary
'session_dates':[datetime.strptime(str(int(i)).zfill(6),'%M%d%Y').date() for i in data['sessionDate']],
Refactor Project structure
User story
As a developer, I want a standardized project structure, so I can maintain and deploy the code base more easily.
Acceptance criteria
- Add pyproject.toml file with dev dependencies
- Add Issue templates
- Add version to init file
- Add src and test directories
- Add example unit test
- Add configuration loader
- Add linters and code coverage
- Add github action to run coverage tests and linting checks
- Add github action to check for version bump
- Add github action to create tags
- Add License file (MIT)
Sprint Ready Checklist
- 1. Acceptance criteria defined
- 2. Team understands acceptance criteria
- 3. Team has defined solution / steps to satisfy acceptance criteria
- 4. Acceptance criteria is verifiable / testable
- 5. External / 3rd Party dependencies identified
- 6. Ticket is prioritized and sized
Helper IO script with paths and default parameters
Our scripts are reading and writing files and their locations have always have to be specified. The location depends on where you mounted the bucket.
We should have a helper script where we hard-code these paths, and all scripts can grab the path from there.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.