Giter VIP home page Giter VIP logo

iltis's Introduction

ILTIS - an Interactive Labeled-Trial Image-stack Slicer

screenshot

General description

This program was designed to interactively and flexibly slice datasets from functional imaging experiments along the time axis. A classical imaging experiment generates data sets of high dimensionality: The individual imaging trials, e.g. the response of a certain area to a stimulus is usually imaged as (x,y) images over time (t), resulting in 3d image stacks. Additionally, different stimuli (S) are given with a certain number of repetitions (R), resulting in 5 dimensional data (x,y,t,S,R).

This program serves the following purposes:

  1. It is a flexible data visualizer and inspector by offering interactive intensity scaling, color maps, overlays and data subselection to really see how your dataset looks like.
  2. It allows to select regions-of-interest (ROI) at which the data set is sliced along the temporal axis. The resulting time traces are directly displayed and updated upon any change.
  3. The traces can be extracted in .csv format, sorted to stimulus class S and repetition R for subsequent data analysis.

Usage

Loading data

The program is still in a very early developmental stage, but already usable. Load your data from the load menu, trial labels can be added with the load labels function. Label files are expected to consist of a label in each line, with the index of each line corresponding to the respective data set. For example, if you want to load the files

  • my_trial_1.tif
  • my_trial_2.tif
  • my_trial_3.tif
  • my_trial_4.tif

and in your experiment trial 1 and 3 were stimulus A, 2 and 4 were stimulus B, then the label file has to look like

stimulus A
stimulus B
stimulus A
stimulus B

File format support

Fully supported are currently any .tif files, Zeiss .lsm files generated by 'pre-Zen' LSM 510 software (later ones might work) and .pst files from TILL Vision imaging rigs. Imports for other formats can be developed, please post a request and send me an example file.

Frame visualization

To subset the currently displayed data set use the Data Selector on the top right. It supports intuitive ctrl+click, shift+click, ctrl+a etc. to select or deselect individual trials to display. The mode of display can be set with the centrally position icons, for example switch between dF/F raw display, average over frames, or a monochrome mode for individual trials with the raw data as the background optionally overlaid with the dF/F signal in a glow colormap.

Preprocessing

Currently the datasets can be gaussian filtered along image and time axes, by convolving the x,y,t image stack with a gaussian kernel of size set in the options window. Frames that go into the background image for dF/F calculation can also be specified in the options window.

ROI functionality

ROIs are added by simply clicking on the image, the type of ROI (circular or polygonal) can be selected in the extra Options window, opened by clicking on the icon in the toolbar. ROIs can be interactively resized and dragged to different regions, all traces will be updated interactively.

The ROI Manager supports the same multiple selection of ROIs just as the Data Selector. If multiple ROIs are selected, the coloring and traces display mode changes: The traces of all selected ROIs are now color coded to the ROI instead of the trial. This is useful if instead of comparing the response in two different trials, one wants to compare the response in two different areas which are within the field of view.

Traces visualization

Depending on the selected datasets and the activated ROIs, the displayed data is sliced along the time domain under the area covered by the ROIs, each frames values are averaged. The sliced traces are color coded and can be displayed in two ways:

  1. All traces with a common time base
  2. sorted to stimulus identity

Both display modes can be accessed by selecting the corresponding tab of the traces visualizer. Both can also be detached from the main window by double-clicking on the tab, for example to move it on a separate screen. Traces are interactively updated upon any change to the ROIs or data selection.

Traces export

After setting up all ROIs, the dataset can be sliced according to the pixels covered by the ROIs. The average value of those pixels for each frame is calculated and the resulting vectors can be written in different ways into .csv files:

  • .csv - normal writes one file per loaded dataset, with the shape (t,ROI).
  • .csv - sorted writes one file per ROI and stimulus combination, with the individual columns representing the repetitions.

If a dt value is specified in the Options, the corresponding time vector is calculated and added to the first column of the .csv, with the stimulus start being set to 0, pre stimulus times are thus negative.

Installation

  1. Install the python 3.7 version of Anaconda for your operating system.
  2. Download the repostory ILTIS
  3. Open a terminal (Linux and Mac) or Powershell (Windows) and create an environment containing Python 3.7:
    conda create -n ILTIS python=3.7 (More on managing conda environments here)
  4. Activate the conda environment created
    conda activate ILTIS
  5. Navigate into the repository containing ILTIS.
    cd <path of the repository ILTIS>
  6. Make sure that the file setup.py in your current path by looking at the output of the command ls (Linux/Mac) or dir (Windows).
  7. Install ILTIS into the environment
    pip install ./

Starting ILTIS

  1. Open a terminal (Linux and Mac) or Powershell (Windows) and activate the conda environment created
    conda activate ILTIS
  2. Start ILTIS using the command "iltis"
    iltis

Upcoming functionality implementations

  • automatic calculation of ROIs (NNMF)
  • movement corrections: rigid and nonlinear transformation based movement correction

iltis's People

Contributors

grg2rsr avatar dahaniel avatar wolfhuette avatar

Stargazers

Santi Villalba avatar Andrew Straw avatar  avatar

Watchers

James Cloos avatar  avatar  avatar

iltis's Issues

Installation error

(ILTIS) Yunusas-MacBook-Air:ILTIS_master_2 yunusa$ pip install ./
Processing /Users/yunusa/Documents/ILTIS_master_2
ERROR: Command errored out with exit status 1:
command: /Users/yunusa/anaconda3/envs/ILTIS/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/setup.py'"'"'; file='"'"'/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/pip-egg-info
cwd: /private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/
Complete output (27 lines):
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/setup.py", line 21, in
setup(**config)
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/site-packages/setuptools/dist.py", line 448, in init
k: v for k, v in attrs.items()
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/distutils/dist.py", line 292, in init
self.finalize_options()
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/site-packages/setuptools/dist.py", line 740, in finalize_options
ep.load()(self)
File "/Users/yunusa/anaconda3/envs/ILTIS/lib/python3.7/site-packages/setuptools/dist.py", line 747, in _finalize_setup_keywords
ep.load()(self, ep.name, value)
File "/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/.eggs/setuptools_scm-3.3.3-py3.7.egg/setuptools_scm/integration.py", line 17, in version_keyword
dist.metadata.version = get_version(**value)
File "/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/.eggs/setuptools_scm-3.3.3-py3.7.egg/setuptools_scm/init.py", line 150, in get_version
parsed_version = _do_parse(config)
File "/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd/.eggs/setuptools_scm-3.3.3-py3.7.egg/setuptools_scm/init.py", line 113, in _do_parse
"use git+https://github.com/user/proj.git#egg=proj" % config.absolute_root
LookupError: setuptools-scm was unable to detect version for '/private/var/folders/vy/smlwmtbn5mgb_8gvx_jpns500000gn/T/pip-req-build-z6ukxymd'.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

PST data when loaded is flipped vertically

We took some calibration images with Till Photonics setup and found that the PST data loaded in ILTIS are vertically flipped compared to how they appear in the microscope.

PS: I have raised a pull request 'iss025' that fixes this. Have a look!

Exporting traces based on ROIs in PyQt5 port

With data loaded and ROIs created, when using "save->export traces based on ROIs" leads to a crash

Traceback (most recent call last):
  File "/home/aj/repos/iltis-pyqt5/iltis/Objects/IO_Object.py", line 458, in export_traces
    self.Main.Processing.calc_traces(self.Main.Data.extraction_mask)
  File "/home/aj/repos/iltis-pyqt5/iltis/Objects/Processing_Object.py", line 118, in calc_traces
    if extraction_mask == None:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
[1]    9517 abort (core dumped)  iltis

error upon loading .lst

Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R01_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R02_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R03_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R04_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R05_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R06_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R07_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R08_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R09_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R10_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R11_GR1_B1_fullglobal
Experiment not found in the .lst file!
EXP8DO_R12_GR1_B1 EXP8DO_R12_GR1_B1_fullglobal
Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 502, in load_lst
concs = [str(self.Main.Data.Metadata.LSTdata.loc[ind_map[n]]['OConc']) for n in range(self.Main.Data.nTrials)]
IndexError: list index out of range

short term 2do

  • clean up instantiation order, especially options_object
  • widget size upon ini needs to be adjusted
  • traces_visualization_stimsorted doesn't fit parent widget
  • options update

log to lis

alja@Smoothie:$ cd programs/ILTIS/
alja@Smoothie:
/programs/ILTIS$ sh run.sh
using pyqtgraph at /home/alja/programs/ILTIS/ILTIS/../lib/pyqtgraph-master/pyqtgraph/init.pyc
/home/alja/programs/ILTIS/ILTIS/../lib/tifffile.py:1527: UserWarning: failed to import _tifffile.decodepackbits
warnings.warn("failed to import %s" % module_function)
/home/alja/programs/ILTIS/ILTIS/../lib/tifffile.py:1527: UserWarning: failed to import _tifffile.decodelzw
warnings.warn("failed to import %s" % module_function)
/home/alja/programs/ILTIS/ILTIS/../lib/tifffile.py:1527: UserWarning: failed to import _tifffile.unpackints
warnings.warn("failed to import %s" % module_function)
no startup msg set
Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 627, in convert_log2lst
gio.log2lst(log_path)
File "/home/alja/programs/ILTIS/ILTIS/../lib/gioIO.py", line 187, in log2lst
setting = setting
UnboundLocalError: local variable 'setting' referenced before assignment

error upon loading .lst

Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 428, in load_lst
self.Main.Data.Metadata.trial_labels = [self.Main.Data.Metadata.LSTdata.loc[ind_map[n]]['Odour']+str(self.Main.Data.Metadata.LSTdata.loc[ind_map[n]]['OConc']) for n in range(self.Main.Data.nTrials)]
IndexError: list index out of range

fail at loading some (but not all!) .pst files

strange error occurs when loading some .pst files
/home/georg/Dropbox/python/ILTIS/ILTIS/../lib/pyqtgraph-master/pyqtgraph/graphicsItems/ImageItem.py:361: RuntimeWarning: invalid value encountered in double_scalars bins = np.arange(mn, mx+1.01*step, step, dtype=np.int)

error upon loading .lst

Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 596, in load_lst
ind_map = self.map_lst_inds_to_path_inds()
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 565, in map_lst_inds_to_path_inds
info = self.Main.Data.Metadata.LSTdata['DBB1'][ind].strip().split('')
File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1658, in getitem
return self._getitem_column(key)
File "/usr/lib/python2.7/dist-packages/pandas/core/frame.py", line 1665, in _getitem_column
return self._get_item_cache(key)
File "/usr/lib/python2.7/dist-packages/pandas/core/generic.py", line 1005, in _get_item_cache
values = self._data.get(item)
File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 2874, in get
_, block = self._find_block(item)
File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3186, in _find_block
self._check_have(item)
File "/usr/lib/python2.7/dist-packages/pandas/core/internals.py", line 3193, in _check_have
raise KeyError('no item named %s' % com.pprint_thing(item))
KeyError: u'no item named DBB1'

error upon loading .lst

Beim Laden des List files erscheint folgende Zeile im Terminal fortlaufend (es hört gar nicht mehr auf...)

QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already running
QCoreApplication::exec: The event loop is already

error upon loading .lst

loading dataset from /media/KenyonCellPostOdor/OK107-dataset/fh_141205_b_OK107___Xavier/Images/exp4toR8/EXP4DO_R12_GR1_B1_fullbsplineglobal.tif
Dataset size: 10.198746 MB
default options
loading default options
Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 602, in load_lst
if sp.int32(conc) > 0: # info is in dilutions
ValueError: invalid literal for long() with base 10: '10000.0'
Traceback (most recent call last):
File "/home/alja/programs/ILTIS/ILTIS/IO_Object.py", line 602, in load_lst
if sp.int32(conc) > 0: # info is in dilutions
ValueError: invalid literal for long() with base 10: '10000.0'

Mouse interaction issues with ILTIS and PyQt5

In the ported version of PyQt5, the following issues exit as of commit 93743a7 on branch ported2PyQt5:

  1. Left clicking in Frame_Visualizer_Widget before loading data leads to a crash.
  2. Scrolling in Traces_Visualizer_Widget and Traces_Visualizer_Stimsorted_Widget lead to crashes. This happened both before and after loading the data.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.