jefferis / igorr Goto Github PK

View Code? Open in Web Editor NEW

6.0 3.0 4.0 399 KB

R package to load Igor Pro waves (ibw) and experiment files (pxp) in R

Home Page: http://jefferis.github.io/IgorR/

R 100.00%

r igor pro electrophysiology neuromatic

igorr's Introduction

IgorR

Introduction

The IgorR package for R provides routines to read binary files generated by Igor Pro.

This includes both the standalone .ibw Igor Binary Wave format (v2 and the latest v5) and the .pxp Packed Experiment Format including both waves and variables. In addition the package provides functions to read files generated by the Neuromatic analysis suite written by Jason Rothman for Igor Pro.

Installation

Standard Install

NB IgorR is off CRAN due to an unfixed bug with windows text on tecent versions of R. Whenever it is back there you can install the release version from CRAN

install.packages('IgorR')

Github Install

To install the latest development version from github

# install remotes package if required
if(!require('remotes')) install.packages('remotes')
remotes::install_github('jefferis/IgorR')

Development Install

To checkout a version that can be used for development (RStudio is strongly recommended), in your terminal application.

cd /some/suitable/dir
git clone https://github.com/jefferis/IgorR.git

In R

install.packages('devtools') # install hadley's devtools
library(devtools)
load_all('/some/suitable/dir/IgorR')
test()

#hack
load_all()

# ready for release
check()
build_win() # test for Windows
release()

Details

Details of the ibw file format were derived from the Igor Technical Note 003.

For those interested in the source code, ReadIgorBinary.R provides code for standard pxp and ibw files, while ReadNclamp.R provides routines to read packed experiment files generated by the Nclamp data acquisition package.

igorr's People

Contributors

Stargazers

Watchers

Forkers

t-b b-rich dmthvlngm haniehkhoshnevisansari

igorr's Issues

Change ReturnTimeSeries when using regex

At the moment when using a regex only wave names are checked. All other object types are returned as is. This is probably not the expected behaviour. Furthermore when ReturnTimeSeries=TRUE, those non wave items give a warning message. We might want to change this behaviour, particularly if we e.g. returned all the waves as an mts object if they are of compatible lengths.

errors in read.ibw when null-terminated strings are not null-terminated

specific example reported by Troy Margrie, Mateo Velez and Charly Rousseau
(one of) the bad string(s) looks like this:

deg/s\003

and fails like this in .readNullTermString:

Error in readBin(con, what = "integer", size = 1, n = totalLength - (nchar(s) +  : 
  invalid 'n' argument
In addition: Warning message:
In readBin(con, what = character(), n = 1, size = 1, ...) :
  incomplete string at end of file has been discarded

The string was reported to have total length 5.

Presumably I could be reading only up to totalLength in .readNullTermString and then reading anything left over only if fewer than totalLength bytes have been read.

read.pxp fails when Igor folder is not a valid R name

specific example reported by Troy Margrie, Mateo Velez and Charly Rousseau

root$Packages$WMPolarGraphs$_default_$vars

in this instance _default_ starts with an underscore, which is not a valid R name.

Options include

running make.names on all folder names
quoting anything dodgy in backticks

Think about using C API to increase read speed.

TN003IgorBinaryFormat provides a simple C interface for reading/writing Igor binary files. This could be worth considering, perhaps as an optional extension. In general this probably isn't too big issue, as the some simple experiments suggest that most of the time is spent reading the file rather than processing it. However R's readBin is not that speedy, so there could be room for improvement.

As an example, two 38 Mb files read from network storage

> system.time(w10 <- read.pxp("/Volumes/JData5/JPeople/Shahar/Data/160201/nm20160201c0/nm20160201c0_010.pxp"))
   user  system elapsed 
  0.744   0.166   1.333 

> f='/Volumes/JData5/JPeople/Shahar/Data/160201/nm20160201c0/nm20160201c0_011.pxp'
> system.time(rb <- readBin(f, what='raw', n=file.info(f)$size))
   user  system elapsed 
  0.002   0.069   0.519 
> system.time(rb <- readBin(f, what='raw', n=file.info(f)$size))
   user  system elapsed 
  0.001   0.027   0.033 

> system.time(w11 <- read.pxp("/Volumes/JData5/JPeople/Shahar/Data/160201/nm20160201c0/nm20160201c0_011.pxp"))
   user  system elapsed 
  0.788   0.150   0.956 

> system.time(w10 <- read.pxp("/Volumes/JData5/JPeople/Shahar/Data/160201/nm20160201c0/nm20160201c0_010.pxp"))
   user  system elapsed 
  0.774   0.131   0.908

So the the raw read time (which probably isn't much slower than a C implementation) is about 40% of the total file parsing time. So that (2.5x) is the kind of speedup I might imagine achieving.

For the same files coming from the built-in flash memory on a macbook

> system.time(w10 <- read.pxp("~/dev/Igor/nm20160201c0/nm20160201c0_010.pxp"))
   user  system elapsed 
  0.778   0.093   0.876 
> system.time(w10 <- read.pxp("~/dev/Igor/nm20160201c0/nm20160201c0_010.pxp"))
   user  system elapsed 
  0.754   0.083   0.840 
> f='~/dev/Igor/nm20160201c0/nm20160201c0_011.pxp'
> system.time(rb <- readBin(f, what='raw', n=file.info(f)$size))
   user  system elapsed 
  0.002   0.039   0.116

so here I could imagine as much as an 8-fold speed improvement.