I'm trying to optimize xcmsSet parameters for a large study by taking subsets of the data (10 - 20 mzML files) as representatives of different LC-MS modes and running them separately. optimizeXcmsSet() does work with some of the data subsets just fine. But with some datasets, function stops after a few DoE with an error:

Error in peaks(xset)[, c("mz", "rt", "sample", "into", "mzmin", "mzmax", :
subscript out of bounds
Calls: system.time ... xcmsSetExperimentsCluster -> sapply -> lapply -> FUN -> calcPPS

I tried running the code with the same mzMLs on two systems:

R version 3.4.0 (2017-04-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

R version 3.4.0 (2017-04-21)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: SUSE Linux Enterprise Server 12 SP1

Code below is for running on x86_64-pc-linux-gnu cluster:

ppparam <- getDefaultXcmsSetStartingParams('centWave')
ppparam$min_peakwidth <- c(1,5) 
ppparam$max_peakwidth <- c(5,10)
ppparam$ppm <-c(20,25) 
ppparam$noise <- c(300, 600) 
ppparam$mzdiff <- c(-0.001, 0.010) 
ppparam$prefilter <- 8  
ppparam$value_of_prefilter <- 6000 
ppparam$integrate <- 2 # 
ppparam$mzCenterFun <- 'wMean' 
ppparam$snthresh <- 10 

opp <- optimizeXcmsSet(files = files,
                         params = ppparam,
                         BPPARAM = MulticoreParam(workers = 10),
                         nSlaves = 1,
                         subdir = output_dir)

Have you encountered such issue before? I don't understand why this issue occurs, since function works with the same, but fewer files (e.g., 2 files, 5 files or 8 files out of 10 files). Furthermore, these mzML files are processed succesfully with xcmsSet() and are not corrupted in any way.

Thank you.

BiocCheck("IPO") results in:

$requirements
[1] "'vignettes' directory!"

Hello,

I'm quite new in R and have not much experiences in programming. I tried IPO for XCMS parameter optimization and everything worked quite good. But I have a small issue, when I try the code to plot the response surfaces from the DoE's I receive the following error message:

Error in contour.lm(resultRetcorGroup[[i]]$model, plots, image = TRUE,  : 
'at' must be a NAMED list of values used for surface slices

additional information
IPO used 4 DoE's in total before it found the best parameters settings. By the way I get the same error message when I try to plot the results from the retention time correction and grouping parameters. What I have done so far was to delte the at=max part from the code and in that way it was posible to get some plots. But I'm not sure if I get the correct plots in that way?

current results: Image

I also got several warning messages (>50):

$`implicit list embedding of S4 objects is deprecated`
`[<-`(`*tmp*`, "object", value = <S4 object of class "xcmsRaw">)

Another question, is it possible to produce those plots also via ggplot and is there already some code chunk I could use?

original code for DoE plotting

 DoEs <- (length(resultPeakpicking)-1)
 opt_params <-
  sum(unlist(lapply(peakpickingParameters, function(x) {length(x)==2})))

 plots <- c()
 for(i in 1:(opt_params-1)) {
   for(j in (i+1):opt_params) {
     plots <- c(plots, as.formula(paste('~ x', i, '* x', j, sep='')))
   }
 }

 plot_rows <- round(sqrt(length(plots)))
 plot_cols <-
  if(plot_rows==1){length(plots)}else{ceiling(sqrt(length(plots)))}

 for(i in 1:(DoEs)) {
   #plot.new()
   par(mfrow=c(2, 2))#, oma=c(3,0,2,0))
   max <- resultPeakpicking[[i]]$max_settings[-1]
   max[is.na(max)] <- 1
   contours <- contour(resultPeakpicking[[i]]$model, plots, image=TRUE, at=max)
   mtext(paste('Response surface models of DoE', i), side = 3,
    line = -2, outer = TRUE)
 }

code I used for DoE plotting

 DoEs <- (length(resultPeakpicking)-1)
 opt_params <-
  sum(unlist(lapply(peakpickingParameters, function(x) {length(x)==2})))

 plots <- c()
 for(i in 1:(opt_params-1)) {
   for(j in (i+1):opt_params) {
     plots <- c(plots, as.formula(paste('~ x', i, '* x', j, sep='')))
   }
 }

 plot_rows <- round(sqrt(length(plots)))
 plot_cols <-
  if(plot_rows==1){length(plots)}else{ceiling(sqrt(length(plots)))}

 for(i in 1:(DoEs)) {
   #plot.new()
   par(mfrow=c(2, 2))#, oma=c(3,0,2,0))
   max <- resultPeakpicking[[i]]$max_settings[-1]
   max[is.na(max)] <- 1
   contours <- contour(resultPeakpicking[[i]]$model, plots, image=TRUE)
   mtext(paste('Response surface models of DoE', i), side = 3,
    line = -2, outer = TRUE)
 }

Greetings

/Mexxxx

Good Afternoon,

I wanted to report an error I had running the IPO function "optimizeXcmsSet()", specifically when I tried running this function with multiple data files using CAMERA as the Isotope detection algorithm rather than the IPO algorithm. When I try this, I get the error "Error in xsAnnotate(xset, sample = sample) : First argument must be a xcmsSet with group information or contain only one sample." after centWave appeared to have finished running. The function does work with a single file, as with the example provided in the R documentation for the function:

mzmlfile <- file.path(find.package("msdata"), "microtofq/MM14.mzML")
 
paramsPP <- getDefaultXcmsSetStartingParams()
paramsPP$mzdiff <- -0.001
paramsPP$min_peakwidth <- c(7,14)
paramsPP$max_peakwidth <- c(20,30)

#example using CAMERA isotope identification
resultPP <- optimizeXcmsSet(mzmlfile, paramsPP, isotopeIdentification="CAMERA", 
                            subdir="mtbls2", ppm=15, maxcharge=2)

I did try to review the source code to try to diagnose why the error takes place with multiple samples, but I was a little confused by the treatment of the xset object.

I would be very grateful for your assistance in helping resolve the problem.

Craig

During execution of the function optimizeXcmsSet, I am repeatedly encountering an error that throws up the following message:

Error in makeCluster(nSlaves, type = "PSOCK") : unknown cluster type

When typing makeCluster(8, type = 'SOCK') manually, I'm able to make a cluster on the localhost. Perhaps a small but important typo?

sessionInfo() output:

R version 3.2.1 (2015-06-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C LC_TIME=English_United Kingdom.1252

attached base packages:
[1] tcltk grid parallel stats graphics grDevices utils datasets methods base

other attached packages:
[1] RUnit_0.4.28 BiocInstaller_1.18.4 BiocParallel_1.2.20 Simile_1.3.3 intervals_0.15.0 R.utils_2.1.0
[7] R.oo_1.19.0 R.methodsS3_1.7.0 plyr_1.8.3 snow_0.3-13 Hmisc_3.16-0 ggplot2_1.0.1
[13] Formula_1.2-1 survival_2.38-3 lattice_0.20-33 IPO_1.7.2 CAMERA_1.24.1 igraph_1.0.1
[19] rsm_2.7-2 xcms_1.44.0 Biobase_2.28.0 ProtGenerics_1.0.0 BiocGenerics_0.14.0 mzR_2.2.1
[25] Rcpp_0.12.0 CORE_3.0

loaded via a namespace (and not attached):
[1] futile.logger_1.4.1 RColorBrewer_1.1-2 futile.options_1.0.0 tools_3.2.1 rpart_4.1-10 digest_0.6.8
[7] gtable_0.1.2 graph_1.46.0 proto_0.3-10 gridExtra_2.0.0 stringr_1.0.0 cluster_2.0.3
[13] stats4_3.2.1 nnet_7.3-10 RBGL_1.44.0 foreign_0.8-65 latticeExtra_0.6-26 lambda.r_1.1.7
[19] reshape2_1.4.1 magrittr_1.5 scales_0.2.5 codetools_0.2-14 splines_3.2.1 MASS_7.3-43
[25] colorspace_1.2-6 stringi_0.5-5 acepack_1.3-3.3 munsell_0.4.2

Hi,

by using optimizeRetGroup() with mzR 2.3.1 (devel version) I get an Error:

starting new DoE with:
$gapInit
[1] 0.00 0.48

$gapExtend
[1] 2.04 2.76

$profStep
[1] 0.71 1.00

$bw
[1] 20.4 39.6

$minfrac
[1] 0.26 0.74

$mzwid
[1] 0.013 0.037

$distFunc
[1] "cor_opt"

$plottype
[1] "none"

$response
[1] 1

$factorDiag
[1] 2

$factorGap
[1] 1

$localAlignment
[1] 0

$retcorMethod
[1] "obiwarp"

$minsamp
[1] 1

$max
[1] 50

$center
[1] 1
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘peaks’ for signature ‘"xcmsSet"’
Calls: retcor ... retcor.obiwarp -> retcor.obiwarp -> .local -> peaks ->

Traceback:

traceback()
15: structure(x, class = unique(c("AsIs", oldClass(x))))
14: FUN(X[[5L]], ...)
13: lapply(X = X, FUN = FUN, ...)
12: sapply(list(...), I)
11: FO(x1, x2, x3, x4, x5, x6)
10: eval(expr, envir, enclos)
9: eval(predvars, data, env)
8: model.frame.default(Terms, newdata, na.action = na.action, xlev = object$xlevels)
7: model.frame(Terms, newdata, na.action = na.action, xlev = object$xlevels)
6: predict.lm(model, testdata)
5: predict(model, testdata)
4: getMaxSettings(testdata, model)
3: getMaximumLevels(retcor_result$model)
2: retGroupExperimentStatistic(retcor_result, subdir, iterator,
xset)
1: optimizeRetGroup(xset = xSetRG, params = retcor_parameter, nSlaves = nSlaves,
subdir = outdir)

kind regards,
Emanuel

Hi,
I was successfully running the R-scripts from the original paper supplement (see WIKI)
and that gave no issues. However after installing the library(Rmpi) I got the following error:

starting new DoE with:
min_peakwidth:  c(12, 28)       max_peakwidth:  c(35, 65)       ppm:    c(17, 32)       mzdiff: c(-0.001, 0.01) snthresh:       10      noise:  0       prefilter:      3       value_of_prefilter:     100     mzCenterFun:    wMean   integrate:      1       fitgauss:       FALSE   verbose.columns:        FALSE   nSlaves:        1       
Error in mpi.spawn.Rslaves(nslaves = nSlaves, needlog = FALSE) : 
  Spawning is not implemented. Please use mpiexec with Rprofile.
Timing stopped at: 0.44 0.32 1134.28 

I also wonder why it would complain after 18 minutes, instead of telling me that beforehand?
Because it correctly spawned 32 rscript processes. Just broke down later. Also I did not set nSlaves = 1. I think I had it set to 32 (no matter if that makes sense or not).

Very surprising behavior.

Cheers
Tobias

Hi, the retcor(method="loess") has quite a performance benefit
over the method="obiwarp", so it would be cool if that could
be optimised as well. The downside is that this requires
method-dependent getDefaultRetGroupStartingParams().

Also, obiwarp is independent of the grouping, while loess depends
on reasonable grouping. If oscillation of optimal parameters occurs,
that could be a reason. Yours, Steffen

Hi,

I have some issues installing IPO. Among which include not loading 'curl' 'CAMERA' 'xcms' even though they are already installed and loaded using library. In addition I have the problem below:

Downloading github repo glibiseller/IPO@master
Installing IPO
Skipping 2 packages not available: CAMERA, xcms
"C:/PROGRA1/R/R-321.1/bin/x64/R" --no-site-file --no-environ --no-save
--no-restore CMD INSTALL
"C:/Users/[...]/AppData/Local/Temp/Rtmpc1yP3q/devtoolsfa041361295/glibiseller-IPO-3738d2a"
--library="[...]/R/win-library/3.2" --install-tests

• installing source package 'IPO' ...
  ** R
  ** inst
  ** tests
  ** preparing package for lazy loading
  Warning: replacing previous import by 'igraph::groups' when loading 'CAMERA'
  ** help
  *** installing help indices
  ** building package indices
  ** testing if installed package can be loaded
  *** arch - i386
  Warning in library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) :
  there is no package called 'IPO'
  Error: loading failed
  Execution halted
  *** arch - x64
  Warning in library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) :
  there is no package called 'IPO'
  Error: loading failed
  Execution halted
  ERROR: loading failed for 'i386', 'x64'
• removing '[...]/R/win-library/3.2/IPO'
  Error: Command failed (1)

Please advise. Thanks in advance.

Hello @sneumann @rietho @glibiseller ,
I'm trying to optimize the parameters of XCMS, but I found the code of "matchedfilter" in the instruction and it's easy to use.
However, I know little about "centWave", I refer to the code from Titan in the issue"optimization of time"

I also read the instruction: "Currently the xcms peak picking methods centWave and matchedFilter are supported. The parameter peakwidth of the peak picking method centWave needs two values defining a minimum and maximum peakwidth. These two values need separate optimization and are therefore split into min_peakwidth and max_peakwidth in getDefaultXcmsSetStartingParams. Also for the centWave parameter prefilter two values have to be set. To optimize these use set prefilter to optimize the first value and prefilter_value to optimize the second value respectively."

I just wonder how to optimize the centWave parameter prefilter two values. or Would you tell me what's the usual code if I want to optimize the min_peakwidth,max_peakwidth,ppm,prefilter? Thank you very much!
I'm a green hand and I'm appreciated that if you can give some suggestion.

peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')
peakpickingParameters$min_peakwidth <- c(10,20)
peakpickingParameters$max_peakwidth <- c(26,42)
peakpickingParameters$ppm <- 3
peakpickingParameters$nSlaves <- 1
resultPeakpicking <- optimizeXcmsSet(files= datafiles[1:2],params=peakpickingParameters, nSlaves=1, subdir='rsmDirectory')

Hi, currently, RT deviations are added up linearly. Possibly, an exponent might make sense,
which punishes values for span=X which cause overcorrection. Yours, Steffen

Hi,
just a question, can you create a image folder for the IPO WIKI so I can upload pictures to the WIKI without creating pull requests from my fork all the time, I basically would like to fully edit the original WIKI with pictures. Not sure if this is commonly done in this way and OK.

Cheers
Tobias

Hi,

when running the code from the vignette on own files,
and changing to centWave/loess:

peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')

time.xcmsSet <- system.time({ # measuring time
  resultPeakpicking <- optimizeXcmsSet(files = files, 
                    params = peakpickingParameters, 
                    nSlaves = nSlaves, 
                    subdir = NULL,
                    plot = TRUE)
})

optimizedXcmsSetObject <- resultPeakpicking$best_settings$xset

retcorGroupParameters <- getDefaultRetGroupStartingParams(retcorMethod="loess")
time.RetGroup <- system.time({ # measuring time
  resultRetcorGroup <- optimizeRetGroup(xset = optimizedXcmsSetObject, 
                     params = retcorGroupParameters, 
                     nSlaves = 1, 
                     subdir = NULL,
                     plot = TRUE)
})

I get an error in the DoE plotting:

Error in plot.new() : figure margins too large
 
12.     plot.new()
11.     plot.default(xlim, ylim, xlim = xlim, ylim = ylim, type = "n", 
        xaxs = xaxs, yaxs = yaxs, xlab = xlab, ylab = ylab, ...)
10.     plot(xlim, ylim, xlim = xlim, ylim = ylim, type = "n", xaxs = xaxs, 
        yaxs = yaxs, xlab = xlab, ylab = ylab, ...) 
9.      image.default(x = c(1, 1.08, 1.16, 1.24, 1.32, 1.4, 1.48, 1.56, 
        1.64, 1.72, 1.8, 1.88, 1.96, 2.04, 2.12, 2.2, 2.28, 2.36, 2.44, 
        2.52, 2.6, 2.68, 2.76, 2.84, 2.92, 3), y = c(1, 1.08, 1.16, 1.24, 
        1.32, 1.4, 1.48, 1.56, 1.64, 1.72, 1.8, 1.88, 1.96, 2.04, 2.12,  ... 
8.      image(x = c(1, 1.08, 1.16, 1.24, 1.32, 1.4, 1.48, 1.56, 1.64, 
        1.72, 1.8, 1.88, 1.96, 2.04, 2.12, 2.2, 2.28, 2.36, 2.44, 2.52, 
        2.6, 2.68, 2.76, 2.84, 2.92, 3), y = c(1, 1.08, 1.16, 1.24, 1.32, 
        1.4, 1.48, 1.56, 1.64, 1.72, 1.8, 1.88, 1.96, 2.04, 2.12, 2.2,  ...
7.      do.call("image", args)
6.      contour.lm(model, plots, image = TRUE, at = maximum_slice)
5.      contour(model, plots, image = TRUE, at = maximum_slice)
4.      plotContours(model = retcor_result$model, maximum_slice = tmp, 
        plot_name = NULL)
3.      retGroupExperimentStatistic(retcor_result = retcor_result, subdir = subdir, 
        plot = plot, iterator = iterator, xset = xset)
2.      optimizeRetGroup(xset = optimizedXcmsSetObject, params = retcorGroupParameters, 
        nSlaves = 1, subdir = NULL, plot = TRUE) 
1.      system.time({
        resultRetcorGroup <- optimizeRetGroup(xset = optimizedXcmsSetObject, 
                params = retcorGroupParameters, nSlaves = 1, subdir = NULL, 
                plot = TRUE) ...

Yours, Steffen

Hi,
I try to run the IPO code from folder test:

• library("msdata")
• BiocGenerics:::testPackage("IPO")

BiocGenerics:::testPackage("IPO")
Error: package ‘BiocGenerics’ was built before R 3.0.0: please re-install it

setRepositories()
install.packages("BioGenerics")
Warning message:
package ‘BioGenerics’ is not available (for R version 3.2.0)

just to let you know.
Tobias

When following the current installation instruction, the line install_github("glibiseller/IPO")
threw this error:
Downloading GitHub repo glibiseller/IPO@master Fehler in curl::curl_fetch_memory(url, handle = handle) : Couldn't resolve host name

However, install_github("rietho/IPO") worked fine.
So I guess, the instruction needs a little update.

Hi,
Due to issues with "devtools" the IPO package under WIN XP requires
R 3.03 (which is not the latest version) and Rtools31.exe (which is not the latest version)

Otherwise the following error occours:
"The procedure entry point _chsize_s could not be located in the dynamic link library msvcrt.dll"

The error can not be easily solved by copying different msvcrt.dll versions to system32
https://support.microsoft.com/en-us/kb/324762

Additionally this will happen under R 3.2.1 (the newer versions)
Error in inDL(x, as.logical(local), as.logical(now), ...) :
unable to load shared object 'C:/Mathematics/R/R-3.2.1/library/git2r/libs/i386/git2r.dll':
LoadLibrary failure: The specified procedure could not be found.

Error: package or namespace load failed for ‘devtools’

Cheers
Tobias

Hello @sneumann @rietho @glibiseller ,
I am trying to optimize xcms parameters using IPO. 18.5 GB .mzXML files. It is taking forever for me to get result. I have i7-4790 CPU @ 3.60GHz, 32 GB RAM and 64 bit operating system.

Is there anyway to get it done faster? Below is the command that I used to run the program.

working_dir = ("C://Users/Lipidomics/Desktop/Tularemia_untar_pos_mzXML/")
setwd(working_dir)

library(xcms)
library(IPO)
peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')
peakpickingParameters$min_peakwidth <- c(10,20)
peakpickingParameters$ppm <- 5
resultPeakpicking <- optimizeXcmsSet(files=working_dir, params=peakpickingParameters, nSlaves=4, subdir='rsmDirectory')

starting new DoE with:
min_peakwidth: c(10, 20)
max_peakwidth: c(35, 65)
ppm: 5
mzdiff: c(-0.001, 0.01)
snthresh: 10
noise: 0
prefilter: 3
value_of_prefilter: 100
mzCenterFun: wMean
integrate: 1
fitgauss: FALSE
verbose.columns: FALSE
nSlaves: 1

Using PSOCK type cluster, this increases memory requirements.
Reduce number of slaves if your have out of memory errors.

Exporting variables to cluster...

Thank you for your help.

Titan

Hi,
the lipidomics set from the paper creates the following warnings after an IPO experiment. However looking at the data it is actually centroided, so there should be no warning?

### Source (https://health.joanneum.at/IPO/LipidomicsTrainingSet.zip)

library(xcms)

library(Rmpi)

xset <- xcmsSet(method="centWave", peakwidth=c(24.16, 123.5), ppm=10.4, noise=0, snthresh=10, mzdiff=-0.0197, prefilter=c(3, 100), mzCenterFun="wMean", integrate=1, fitgauss=FALSE, verbose.columns=FALSE, nSlaves=32)
xset <- retcor(xset, method="obiwarp", plottype="none", distFunc="cor_opt", profStep=0.66, center=2, response=1, gapInit=0.6976, gapExtend=2.9904, factorDiag=2, factorGap=1, localAlignment=0)
xset <- group(xset, method="density", bw=0.25, mzwid=0.00538, minfrac=0.963, minsamp=1, max=50)

xset <- fillPeaks(xset, nSlaves=32)

Warning messages:
1: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

2: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

3: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

4: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

5: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

6: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

7: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

8: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

9: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

10: In .local(object, ...) :
  It looks like this file is in profile mode. centWave can process only centroid mode data !

Cheers
Tobias

Hi,
The function optimizeXcmsSet can bemodified to set the number of slaves to spawn.
For example 32, it will do that.

resultPeakpicking <- optimizeXcmsSet(files=mzdatafiles[1:4],

params=peakpickingParameters, nSlaves=32, subdir='rsmDirectory')

However this and other functions have the following call with "Slaves=4" hard coded
optimizeRetGroup <-
function(xset, params=getDefaultRetGroupStartingParams(), nSlaves=4, subdir="IPO")

Is that correct? Or is that optimized to 4 threads.

Plus once it says "Starting new DOE with" it will print out "nSlaves=1"
BTW this line should be plotted with CR/LF instead of one line,
I have to set my Monitors to 3840 pixels in order to read that line.

Cheers
Tobias

Dear Gunnar,

I'm sorry to bother you again.
Recently, I have met some unexpected problems handling metanolomics data by IPO. Here are the details.

(1) Instrument: UFLC-IT-TOF-MS (SHIMADZU)
File type from vendor software (LCMSSolution, SHIMADZU): .lcd

(2) If I converted the .lcd files to .netCDF files by vendor software, (since the mass spectrometry switch between positive and negative models) the data were sorted by the scan time with one positive record followed by one negative record. IPO failed to pick peaks with the method “centWave”.

Then I tried to converte the .lcd files to .mzXML (or .mzData) files, and each .lcd file would generate two .mzXML files containing positive and negative records respectively. When handing the positive or negative data, IPO failed to detect isotope ions neither.
I checked the .mzXML files by OpenChrom and found something strange. It seems that the mass data were un-centroided or false centroided. Part of the data at the same scan time sorted by mz values were below.
m/z abundance
230.21 0
230.225 8832
230.239 15360
230.254 13184
230.268 0
256.232 0
256.247 15360
256.262 19712
256.277 13184
256.293 0

It is very different with the .netCDF file.
m/z abundance
230.24 124278
231.24 8832
256.26 37504

The vender software seemed to be wrong when converting the .lcd files to mzXML files (or mzData).

How could I get the appropriate .netCDF files for IPO? I do really need a method to transform the .netCDF file to two separated .netCDF files with one containing the positive data and another containing the negative data. I have tried the R package “ncdf” to transform the files, but failed since I am not familiar with R.

Since the network of xcms-websites doesn’t work, I am sorry to interrupt you.
Looking forward to your help and advises!
Thanks in advance.

Best Regards,
Jiaq Chen

Why not bump to 1.99 now, and have it as 2.0 in BioC ?

Dear developers team,
Running the IPO scripts ('optimize XcmsSet()') I run into the error:

Fehler in UseMethod("sendData") :
nicht anwendbare Methode für 'sendData' auf Objekt der Klasse "c('forknode', 'SOCK0node')" angewendet
(in German, meaning "Error in UseMethod("sendData") : not applicable method for 'sendData' on an object of class "c('forknode', 'SOCK0node')"").

Description:

• System: R 3.2.4 on Debian 8.4, all packages actualized.
• Packages load properly.
• A new DoE is started when the function 'optimize XcmsSet()' is called.
• Package 'snow' is attached.
• A snow cluster is started with [nSlaves] local sockets.
• Image 'rsm_1.jpg' is drawn.
• A new DoE is started automatedly with a different parametrization.
• After the expected runtime, the run stops with the above-mentioned error message.

Suspicion:
IPO calls 'parallel', and after the execution of the DoE code, 'snow' is attached. I suppose, 'snow' should be detached before a new DoE run starts. A similar issue has been described for package {secr}.
Any help would be greatly appreciated!
Best regards,
Alex

If I load IPO together with another package that Depends on snow, I can't perform the following script:

library(doSNOW)
library(IPO)
mzdatapath <- system.file("mzData", package = "mtbls2")
mzdatafiles <- list.files(mzdatapath, recursive = TRUE, full.names=TRUE)

peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')
#setting levels for min_peakwidth to 10 and 20 (hence 15 is the center point)
peakpickingParameters$min_peakwidth <- c(10,20) 
peakpickingParameters$max_peakwidth <- c(26,42)
#setting only one value for ppm therefore this parameter is not optimized
peakpickingParameters$ppm <- 20 
resultPeakpicking <- 
  optimizeXcmsSet(files = mzdatafiles[1:4], 
                  params = peakpickingParameters, 
                  nSlaves = 2, 
                  subdir = 'rsmDirectory')

It fails with the following:

Error: package ‘snow’ is required by ‘doSNOW’ so will not be detached

The obvious work around is to not load the offending packages when doing IPO, but I was wondering if this is an IPO problem or a e.g. doSNOW problem?

> devtools::session_info()
Session info --------------------------------------------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.3.0 (2016-05-03)
 system   x86_64, linux-gnu           
 ui       RStudio (0.99.896)          
 language (EN)                        
 collate  en_US.UTF-8                 
 tz       <NA>                        
 date     2016-06-06                  

Packages ------------------------------------------------------------------------------------------------------------------------------------------------------
 package      * version date       source                          
 acepack        1.3-3.3 2014-11-24 CRAN (R 3.3.0)                  
 Biobase      * 2.32.0  2016-05-17 Bioconductor                    
 BiocGenerics * 0.18.0  2016-05-17 Bioconductor                    
 CAMERA       * 1.28.0  2016-05-17 Bioconductor                    
 chron          2.3-47  2015-06-24 CRAN (R 3.0.2)                  
 cluster        2.0.4   2016-04-18 CRAN (R 3.2.5)                  
 codetools      0.2-14  2015-07-15 CRAN (R 3.2.2)                  
 colorspace     1.2-6   2015-03-11 CRAN (R 3.3.0)                  
 data.table     1.9.6   2015-09-19 CRAN (R 3.3.0)                  
 devtools       1.11.1  2016-04-21 CRAN (R 3.0.2)                  
 digest         0.6.9   2016-01-08 CRAN (R 3.0.2)                  
 doSNOW       * 1.0.14  2015-10-14 CRAN (R 3.3.0)                  
 foreach      * 1.4.3   2015-10-13 CRAN (R 3.3.0)                  
 foreign        0.8-66  2015-08-19 CRAN (R 3.2.2)                  
 Formula        1.2-1   2015-04-07 CRAN (R 3.3.0)                  
 ggplot2        2.1.0   2016-03-01 CRAN (R 3.3.0)                  
 graph          1.50.0  2016-05-17 Bioconductor                    
 gridExtra      2.2.1   2016-02-29 CRAN (R 3.3.0)                  
 gtable         0.2.0   2016-02-26 CRAN (R 3.3.0)                  
 Hmisc          3.17-4  2016-05-02 CRAN (R 3.3.0)                  
 igraph         1.0.1   2015-06-26 CRAN (R 3.3.0)                  
 IPO          * 1.7.5   2016-05-17 Github (glibiseller/IPO@a0798ea)
 iterators    * 1.0.8   2015-10-13 CRAN (R 3.3.0)                  
 lattice        0.20-33 2015-07-14 CRAN (R 3.2.1)                  
 latticeExtra   0.6-28  2016-02-09 CRAN (R 3.3.0)                  
 magrittr       1.5     2014-11-22 CRAN (R 3.0.2)                  
 Matrix         1.2-6   2016-05-02 CRAN (R 3.3.0)                  
 memoise        1.0.0   2016-01-29 CRAN (R 3.0.2)                  
 munsell        0.4.3   2016-02-13 CRAN (R 3.3.0)                  
 mzR          * 2.5.8   2016-05-04 Bioconductor                    
 nnet           7.3-12  2016-02-02 CRAN (R 3.2.3)                  
 plyr           1.8.3   2015-06-12 CRAN (R 3.3.0)                  
 ProtGenerics * 1.4.0   2016-05-17 Bioconductor                    
 RBGL           1.48.0  2016-05-17 Bioconductor                    
 RColorBrewer   1.1-2   2014-12-07 CRAN (R 3.3.0)                  
 Rcpp         * 0.12.5  2016-05-14 CRAN (R 3.3.0)                  
 rpart          4.1-10  2015-06-29 CRAN (R 3.2.1)                  
 rsm          * 2.7-4   2015-10-07 CRAN (R 3.3.0)                  
 scales         0.4.0   2016-02-26 CRAN (R 3.3.0)                  
 snow         * 0.4-1   2015-10-31 CRAN (R 3.3.0)                  
 survival       2.39-2  2016-04-16 CRAN (R 3.2.5)                  
 withr          1.0.1   2016-02-04 CRAN (R 3.0.2)                  
 xcms         * 1.48.0  2016-05-17 Bioconductor 

Hi,

I am getting an "object 'x1' not found" in a traceback(), see below.
For debugging, I have restricted the parameters to be optimised
to only ppm. This happens regardless whether or not I set nSlaves.
What do I have to check to debug this further ?

Yours,
Steffen

peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')

if (DEBUG) {
   ## reduce what gets optimised
   peakpickingParameters$min_peakwidth <- 20
   peakpickingParameters$max_peakwidth <- 20
   peakpickingParameters$mzdiff <- 0
   peakpickingParameters$ppm <- c(30,60)
}

iporesult <- optimizeXcmsSet(files=msfile,
                             params=peakpickingParameters,
                             subdir="/tmp")

Error in lapply(X = X, FUN = FUN, ...) : object 'x1' not found
> 
> traceback()
16: lapply(X = X, FUN = FUN, ...)
15: sapply(list(...), I)
14: FO(x1)
13: eval(expr, envir, enclos)
12: eval(predvars, data, env)
11: model.frame.default(formula = resp ~ FO(x1) + +PQ(x1), data = design, 
        drop.unused.levels = TRUE)
10: stats::model.frame(formula = resp ~ FO(x1) + +PQ(x1), data = design, 
        drop.unused.levels = TRUE)
9: eval(expr, envir, enclos)
8: eval(mf, parent.frame())
7: lm(formula = resp ~ FO(x1) + +PQ(x1), data = design)
6: eval(expr, envir, enclos)
5: eval(CALL, parent.frame())
4: rsm(formula, data = design)
3: createModel(xcms_result$design, params$to_optimize, resp)
2: xcmsSetStatistic(xcms_result, subdir, iterator)
1: optimizeXcmsSet(files = msfile, params = peakpickingParameters, 
       subdir = paste(outdir, basename(msfile), sep = "/"))

Currently, calcPPS() returns:

> calcPPS(iporesult$best_settings$xset)
     [,1] [,2] [,3] [,4]    [,5]
[1,]    0  692  462  194 81.4632

There should be reasonable column names mentioning

 1. Space for experimentid of the Design of Experiments (0 since not known in calcPPS)
 2. Number of peaks
 3. Number of peaks without LIP and RP
 4. Reliable peaks (RP)
 5. Peak picking score (PPS)

Another question is why that is a one-row array, are there cases imaginable
where calcPPS() returns multiple values ? If not, I suggest simplifying
to a vector.

Hi,

I get Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 0 (non-NA) cases
when doing:

> retcorGroupParameters <- getDefaultRetGroupStartingParams()
> retcorGroupParameters$profStep <- 1
> resultRetcorGroup <- optimizeRetGroup(xset=mtblsSets[["negative"]], 
+                                       params=retcorGroupParameters, 
+                                       nSlaves=12, subdir="rsmDirectory")

starting new DoE with:
$distFunc
[1] "cor_opt"

$gapInit
[1] 0.0 0.4

$gapExtend
[1] 2.1 2.7

$profStep
[1] 1

$plottype
[1] "none"

$response
[1] 1

$factorDiag
[1] 2

$factorGap
[1] 1

$localAlignment
[1] 0

$bw
[1] 22 38

$minfrac
[1] 0.3 0.7

$mzwid
[1] 0.015 0.035

$minsamp
[1] 1

$max
[1] 50

$center
[1] 86

Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 
  0 (non-NA) cases
In addition: Warning messages:
1: closing unused connection 6 (<-localhost:11147) 
2: closing unused connection 5 (<-localhost:11147) 
3: closing unused connection 4 (<-localhost:11147) 
4: closing unused connection 3 (<-localhost:11147) 

Traceback shows:

> traceback()
9: stop("0 (non-NA) cases")
8: lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...)
7: lm(formula = resp ~ FO(x1, x2, x3, x4, x5) + TWI(x1, x2, x3, 
       x4, x5) + PQ(x1, x2, x3, x4, x5), data = design)
6: eval(expr, envir, enclos)
5: eval(CALL, parent.frame())
4: rsm(formula, data = design)
3: createModel(retcor_result$design, params$to_optimize, resp)
2: retGroupExperimentStatistic(retcor_result, subdir, iterator, 
       xset)
1: optimizeRetGroup(xset = mtblsSets[["negative"]], params = retcorGroupParameters, 
       nSlaves = 12, subdir = "rsmDirectory")

For a while now I've come across the following error message when running optimizeRetGroup in single core mode:

Error in .local(object, ...) : 
  Dimensions of profile matrices do not match !

I ignored for a while, up until my data processing pipeline crashed during the retcor.obiwarp step with the above message using optimized IPO parameters.
Once I reran IPO in single core mode, I could see the message was being printed for every sample, which seems to be a rare case?

With the following script the error can be reproduced:

library(msdata)
library(IPO)
library(xcms)

mzdatapath <- system.file("mzData", package = "mtbls2")
mzdatafiles <- list.files(mzdatapath, recursive = TRUE, full.names=TRUE)

xset <- xcmsSet(
  mzdatafiles[1:12],  method = "centWave",  peakwidth = c(6.65, 25), ppm = 25, 
  mzdiff = -0.001, snthresh = 10, noise = 10000, prefilter = c(3, 100), nSlaves = 3
)

paramsRG <- getDefaultRetGroupStartingParams()

resultRG <- optimizeRetGroup(xset, paramsRG, nSlaves=1)

And this is the output: The retcor step seems to stop after sample MSpos-Ex1-cyp79-48h-Ag-1_1-B,1_01_9819 in every iteration.

starting new DoE with:

distFunc: cor_opt
gapInit: c(0, 0.4)
gapExtend: c(2.1, 2.7)
profStep: c(0.7, 1)
plottype: none
response: 1
factorDiag: 2
factorGap: 1
localAlignment: 0
retcorMethod: obiwarp
bw: c(22, 38)
minfrac: c(0.3, 0.7)
mzwid: c(0.015, 0.035)
minsamp: 1
max: 50
center: 10

center sample:  MSpos-Ex2-Col0-48h-Ag-2_1-A,3_01_9829 
Processing: MSpos-Ex1-Col0-48h-Ag-1_1-A,1_01_9818  MSpos-Ex1-Col0-48h-Ag-2_1-A,1_01_9820  MSpos-Ex1-Col0-48h-Ag-3_1-A,1_01_9822  MSpos-Ex1-Col0-48h-Ag-4_1-A,1_01_9824  MSpos-Ex1-cyp79-48h-Ag-1_1-B,1_01_9819  
Error in .local(object, ...) : 
  Dimensions of profile matrices do not match !

108 112 115 119 123 127 130 134 138 142 145 149 153 157 160 164 168 172 175 179 183 187 190 194 198 202 205 209 213 217 220 224 228 232 235 239 243 247 250 254 258 262 265 269 273 277 280 284 288 292 295 299 303 307 310 314 318 322 325 329 333 337 340 344 348 352 355 359 363 367 370 374 378 382 385 389 393 397 400 404 408 412 415 419 423 427 430 434 438 442 445 449 453 457 460 464 468 472 475 479 483 487 490 494 498 502 505 509 513 517 520 524 528 532 535 539 543 547 550 554 558 562 565 569 573 577 580 584 588 592 595 599 603 607 610 614 618 622 625 629 633 637 640 644 648 652 655 659 663 667 670 674 678 682 685 689 693 697 700 704 708 712 715 719 723 727 730 734 738 742 745 749 753 757 760 764 768 772 775 779 783 787 790 794 798 802 805 809 813 817 820 824 828 832 835 839 843 847 850 854 858 862 865 869 
center sample:  MSpos-Ex2-Col0-48h-Ag-2_1-A,3_01_9829 
Processing: MSpos-Ex1-Col0-48h-Ag-1_1-A,1_01_9818  MSpos-Ex1-Col0-48h-Ag-2_1-A,1_01_9820  MSpos-Ex1-Col0-48h-Ag-3_1-A,1_01_9822  MSpos-Ex1-Col0-48h-Ag-4_1-A,1_01_9824  MSpos-Ex1-cyp79-48h-Ag-1_1-B,1_01_9819  
Error in .local(object, ...) : 
  Dimensions of profile matrices do not match !

108 112 115 119 123 127 130 134 138 142 145 149 153 157 160 164 168 172 175 179 183 187 190 194 198 202 205 209 213 217 220 224 228 232 235 239 243 247 250 254 258 262 265 269 273 277 280 284 288 292 295 299 303 307 310 314 318 322 325 329 333 337 340 344 348 352 355 359 363 367 370 374 378 382 385 389 393 397 400 404 408 412 415 419 423 427 430 434 438 442 445 449 453 457 460 464 468 472 475 479 483 487 490 494 498 502 505 509 513 517 520 524 528 532 535 539 543 547 550 554 558 562 565 569 573 577 580 584 588 592 595 599 603 607 610 614 618 622 625 629 633 637 640 644 648 652 655 659 663 667 670 674 678 682 685 689 693 697 700 704 708 712 715 719 723 727 730 734 738 742 745 749 753 757 760 764 768 772 775 779 783 787 790 794 798 802 805 809 813 817 820 824 828 832 835 839 843 847 850 854 858 862 865 869 
center sample:  MSpos-Ex2-Col0-48h-Ag-2_1-A,3_01_9829 
Processing: MSpos-Ex1-Col0-48h-Ag-1_1-A,1_01_9818  MSpos-Ex1-Col0-48h-Ag-2_1-A,1_01_9820  MSpos-Ex1-Col0-48h-Ag-3_1-A,1_01_9822  MSpos-Ex1-Col0-48h-Ag-4_1-A,1_01_9824  MSpos-Ex1-cyp79-48h-Ag-1_1-B,1_01_9819  
Error in .local(object, ...) : 
  Dimensions of profile matrices do not match !

It looks to me like a xcms bug, but I was wondering if IPO can perform any retcor optimization with this error?

Hi,
under WINDOWS7 I uninstalled Rmpi and only use snow now. IPO fires up,
but after the first optimization round it breaks. BTW this was a tested and working version.
Central Carbon Metabolism was the script I used. SO first round is OK and then in the second DoE round it breaks, I guess everything is fine in the first round, then in the second round something happens.

Why would IPO built use makeCluster(nSlaves, type = "PSOCK") in the second round, but not in the first round?

starting new DoE with:
min_peakwidth:  c(12, 28)       max_peakwidth:  c(35, 65)       ppm:    c(17, 32)       mzdiff: c(-0.001, 0.01) snthresh:       10      noise:  0       prefilter:      3       value_of_prefilter:     100     mzCenterFun:    wMean   integrate:      1       fitgauss:       FALSE   verbose.columns:        FALSE   nSlaves:        1       

 Starting snow cluster with 32 local sockets.
Detecting features in file # 1 : 1_QC.mzXML 
Detecting features in file # 2 : 2_QC.mzXML 
Detecting features in file # 3 : 3_QC.mzXML 
Detecting features in file # 4 : 4_QC.mzXML 
Detecting features in file # 5 : 5_QC.mzXML 
Detecting features in file # 6 : 6_QC.mzXML 



starting new DoE with:
min_peakwidth:  c(4, 20)        max_peakwidth:  c(35, 59)       ppm:    c(9.5, 24.5)    mzdiff: c(0.00175, 0.01275)     snthresh:       10      noise:  0       prefilter:      3       value_of_prefilter:     100     mzCenterFun:    wMean   integrate:      1       fitgauss:       FALSE   verbose.columns:        FALSE   nSlaves:        1       
Error in makeCluster(nSlaves, type = "PSOCK") : unknown cluster type
In addition: Warning messages:
1: package ‘Rcpp’ was built under R version 3.1.3 
2: In fun(libname, pkgname) :
  mzR has been built against a different Rcpp version (0.11.3)
than is installed on your system (0.12.0). This might lead to errors
when loading mzR. If you encounter such issues, please send a report,
including the output of sessionInfo() to the Bioc mailing list at 
http://www.bioconductor.org/help/mailing-list. For details see also
https://github.com/sneumann/mzR/wiki/mzR-Rcpp-compiler-linker-issue.
3: package ‘rsm’ was built under R version 3.1.3 
4: package ‘igraph’ was built under R version 3.1.3 
5: replacing previous import by ‘igraph::groups’ when loading ‘CAMERA’ 
Timing stopped at: 6.26 1.25 546.46 
> 

T

Will be fixed in 1.5.7

Hi,
optimizeXcmsSet$best_settings does include an $xset, while optimizeRetGroup$best_settings does not. I think the two should be consistent, or was there a reason ?
Yours, Steffen

Hi,
under WINDOWS7 I uninstalled Rmpi and only use snow now. IPO fires up,
but after the first optimization round it breaks. BTW this was a tested and working version.
Central Carbon Metabolism was the script I used. SO first round is OK and this is the second DoE round, I guess everything is fine in the first round, then in the second round something happens.

Why would IPO built use makeCluster(nSlaves, type = "PSOCK") in the second round, but not in the first round?

starting new DoE with:
min_peakwidth:  c(12, 28)       max_peakwidth:  c(35, 65)       ppm:    c(17, 32)       mzdiff: c(-0.001, 0.01) snthresh:       10      noise:  0       prefilter:      3       value_of_prefilter:     100     mzCenterFun:    wMean   integrate:      1       fitgauss:       FALSE   verbose.columns:        FALSE   nSlaves:        1       

 Starting snow cluster with 32 local sockets.
Detecting features in file # 1 : 1_QC.mzXML 
Detecting features in file # 2 : 2_QC.mzXML 
Detecting features in file # 3 : 3_QC.mzXML 
Detecting features in file # 4 : 4_QC.mzXML 
Detecting features in file # 5 : 5_QC.mzXML 
Detecting features in file # 6 : 6_QC.mzXML 



starting new DoE with:
min_peakwidth:  c(4, 20)        max_peakwidth:  c(35, 59)       ppm:    c(9.5, 24.5)    mzdiff: c(0.00175, 0.01275)     snthresh:       10      noise:  0       prefilter:      3       value_of_prefilter:     100     mzCenterFun:    wMean   integrate:      1       fitgauss:       FALSE   verbose.columns:        FALSE   nSlaves:        1       
Error in makeCluster(nSlaves, type = "PSOCK") : unknown cluster type
In addition: Warning messages:
1: package ‘Rcpp’ was built under R version 3.1.3 
2: In fun(libname, pkgname) :
  mzR has been built against a different Rcpp version (0.11.3)
than is installed on your system (0.12.0). This might lead to errors
when loading mzR. If you encounter such issues, please send a report,
including the output of sessionInfo() to the Bioc mailing list at 
http://www.bioconductor.org/help/mailing-list. For details see also
https://github.com/sneumann/mzR/wiki/mzR-Rcpp-compiler-linker-issue.
3: package ‘rsm’ was built under R version 3.1.3 
4: package ‘igraph’ was built under R version 3.1.3 
5: replacing previous import by ‘igraph::groups’ when loading ‘CAMERA’ 
Timing stopped at: 6.26 1.25 546.46 
> 

T

Hi, just wanted to inform you that with the recent changes in xcms the nSlaves argument is deprecated (actually defunct) as we changed from the homebrew parallel processing to parallel processing using BiocParallel. Eventually you might consider to update IPO too.

Also, there might be some changes in xcms that might speed up the processing, i.e. by using MSnExp objects from the MSnbase package to hold the raw data from a full experiment it will no longer be required to read the data from the raw files in each iteration.

Hello there,
I am attempting to use the IPO package to optimize some settings for an algal lipid dataset I'm working with. The .mzXML data files I use in the code below (the most "well-behaved" 4 of 18 total in the dataset) are online at https://github.com/vanmooylipidomics/LOBSTAHS/Pt_H2O2_mzXML_ms1_pos/0_uM_H2O2/
When I run the following code:

# optim_xcmsParams.R
#
# Created 11/28/2015 by J.R.C.
#
# Purpose: Optimize parameters for several commands in the xcms package using the R package "IPO." Currently, the script is written to optimize parameters for peak-picking, alignment, etc., of lipid data from the experiment described in Graff van Creveld et al., 2015, "Early perturbation in mitochondria redox homeostasis in response to environmental stress predicts cell fate in diatoms," ISME Journal 9:385-395. This dataset is used to demonstrate the LOBSTAHS lipidomics pipeline in Collins, J.R., B.R. Edwards, H.F. Fredricks, and B.A.S. Van Mooy, 2015, "Untargeted discovery and identification of oxidative stress biomarkers using a lipidomics pipeline for complex datasets."
#
# IPO is described in Libiseller et al., 2015, "IPO: a tool for automated optimization of XCMS parameters," BMC Bioinformatics 16:118; see https://github.com/glibiseller/IPO/blob/master/vignettes/IPO.Rmd for installation instructions
#
# See https://github.com/vanmooylipidomics/LOBSTAHS for current versions of all pipeline scripts

################ Initial setup and variable definition #############

# load required packages

library(tools) 

library(xcms)

library(CAMERA)

library(rsm)

# run two lines below only if IPO hasn't been installed already

# library(devtools)
# install_github("glibiseller/IPO") 

library(IPO)

library(snowfall) # if multicore tasking is desired

################# User: define locations of data files and database(s) #############

working_dir = "/Users/jrcollins/Dropbox/code/LOBSTAHS/" # specify working directory
setwd(working_dir) # set working directory to working_dir

# specify directories subordinate to the working directory in which the .mzXML files for xcms can be found; per xcms documentation, use subdirectories within these to divide files according to treatment/primary environmental variable (e.g., station number along a cruise transect) and file names to indicate timepoint/secondary environmental variable (e.g., depth)
mzXMLfiles_folder_pos = "Pt_H2O2_mzXML_ms1_pos/" 
mzXMLfiles_folder_neg = "Pt_H2O2_mzXML_ms1_neg/" 

################# Load in mzXML files #############

mzXMLfiles = list.files(mzXMLfiles_folder_pos, recursive = TRUE, full.names = TRUE)

# # exclude any files you don't want to push through xcms (e.g., blanks); note that the blanks for the Pt H2O2 dataset (Orbi_0481.mzXML and Orbi_0482.mzXML) have already been removed
# mzXMLfiles = mzXMLfiles[-c(1,2)]

################# Use IPO to optimize some of xcms parameters #############

# will use IPO to optimize settings for method = centWave

# define ranges of parameters to be tested
# if single value is specified for a parameter, or centWave default is used, that parameter will not be optimized

peakpickingParameters <- getDefaultXcmsSetStartingParams('centWave')
peakpickingParameters$min_peakwidth <- c(10,20) # centerpoint is 15
peakpickingParameters$max_peakwidth <- c(40,80) # centerpoint is 60
peakpickingParameters$ppm <- c(1.5,3.5)
peakpickingParameters$prefilter <- c(3,5)
peakpickingParameters$value_of_prefilter <- c(5000,10000)
peakpickingParameters$snthresh <- c(3,10)
peakpickingParameters$noise <- c(5000)

# only going to use the first 6 files from the dataset (0 uM H2O2 treatment) for optimization routine

resultPeakpicking <- optimizeXcmsSet(files= mzXMLfiles[1:4], 
                                     params=peakpickingParameters, nSlaves=4, subdir='rsmDirectory')
optimizedXcmsSetObject <- resultPeakpicking$best_settings$xset'

Things start working just fine:

starting new DoE with:
min_peakwidth: c(10, 20)
max_peakwidth: c(40, 80)
ppm: c(1.5, 3.5)
mzdiff: c(-0.001, 0.01)
snthresh: c(3, 10)
noise: 5000
prefilter: c(3, 5)
value_of_prefilter: c(5000, 10000)
mzCenterFun: wMean
integrate: 1
fitgauss: FALSE
verbose.columns: FALSE
nSlaves: 1

It runs for a few hours. But then, I receive this error:

Error in checkForRemoteErrors(val) : 
  2 nodes produced errors; first error: m/z sort assumption violated ! (scan 121, p 1756, current 4.0416 (I=0.00), last 282.9776) 
> optimizedXcmsSetObject <- resultPeakpicking$best_settings$xset
Error: object 'resultPeakpicking' not found
> resultPeakpicking
Error: object 'resultPeakpicking' not found

Any ideas what's happening? I am very excited about the idea of optimizing some of these parameters via objective functions, but I can't figure out what the problem is!

Thanks very much, in advance.

Jamie Collins
Woods Hole Oceanographic Institution

I'm a bit of an R novice, but when installing IPO with R 3.2.2, I get the following error:

install_github("glibiseller/IPO")
Downloading GitHub repo glibiseller/IPO@master
Installing IPO
"C:/Program Files/R/R-32~1.2/bin/x64/R" --no-site-file --no-environ --no-save --no-restore CMD INSTALL
"C:/Users/jjkellog/AppData/Local/Temp/Rtmpqkznw3/devtools5b863d91ce0/glibiseller-IPO-47aff39"
--library="C:/Users/jjkellog/Documents/R/win-library/3.2" --install-tests

'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
Error: Command failed (1)

Everything else, the devtools and Rtools, installed just fine, it is getting IPO installed that is serving up this error.

Hi,
the writeRScript function "prints" the optimized parameters for xcms, but at the same time
actually executes them when called within a function, not sure if this is intended behavour after an IPO experiment.

Basically the function called from the lipidomics IPO set
writeRScript(ppResult$best_settings$parameters, rgResult$best_settings, nSlaves=1)

would write and execute to the console:

library(xcms)

library(Rmpi)

xset <- xcmsSet(method="centWave", peakwidth=c(24.16, 123.5), ppm=10.4, noise=0, snthresh=10, mzdiff=-0.0197, prefilter=c(3, 100), mzCenterFun="wMean", integrate=1, fitgauss=FALSE, verbose.columns=FALSE, nSlaves=1)
xset <- retcor(xset, method="obiwarp", plottype="none", distFunc="cor_opt", profStep=0.66, center=2, response=1, gapInit=0.6976, gapExtend=2.9904, factorDiag=2, factorGap=1, localAlignment=0)
xset <- group(xset, method="density", bw=0.25, mzwid=0.00538, minfrac=0.963, minsamp=1, max=50)

xset <- fillPeaks(xset, nSlaves=1)

Cheers
Tobias

I'm trying to do an optimization of all kinds of parameters for data from LC-Orbitrap-MS, and IPO gets stuck in a loop on the ppm step for centWave. I want to optimize between 1 and 3 ppm, and it's actually trying 2.7 and 4.7 ppm and just switching between those two non-stop.

So I'm guessing there is an issue with the step size for incremental optimization when small steps are necessary.

Hi,
package ‘rsm’ is not available (for R version 3.2.2)

this is under WIN7 with R3.2.2 (64bit). Basically after installing a number of
R packages from 3.2.15 to 3.2.0 and other is basically either this or devtools
throwing errors. One solution is

setRepositories()
install.packages("rsm")

But one could also install and compile from source
https://cran.r-project.org/web/packages/rsm/index.html

Quite interesting updating R and packages always breaks
multiple things, maybe some Docker type R solutions would be good,
basically to freeze packages and installs.
https://www.google.com/search?num=20&q=docker+for+R

Cheers
T.

BiocCheck("IPO") results in:

$recommendations
"The following files call library or require on IPO.\n            This is not necessary.\nman/IPO-package.Rd"

Hi,
currently, optimizeRetGroup() optimises all parameters
for all group() methods. It would be great if the method(s)
could be restricted, e.g. retcorGroupParameters$groupMethod=c("density", "obiwarp")
or just retcorGroupParameters$groupMethod="density"
Yours, Steffen

as requested in #50

Hi,
Because I'm working on the IPO Galaxy wrappers. I set some functional tests so I often launch IPO with the same set of inputs and parameters.

First of all, note that I get this behaviour on a Mac.

And maybe it's completely normal but I get different results for this value xcmsSet.centWave.mzdiff sometimes I get 0.01 and sometimes I get -0.001.

I thought that was because I was loading or not snow at the beginning of my script?
But I also got this switch when I just add or not a print(sessionInfo()) before IPO functions.

Meanwhile, it is stable at 0.01 under Linux and through my TravisCI. So it's not a big deal because Linux is my target. But I would like to share this experience.

Hi,
doing a clean R install (consider a VM) with R.3.2.0 under windows
according to https://github.com/glibiseller/IPO/blob/master/vignette/IPO.Rmd
will lead to

library("devtools")
install_github("glibiseller/IPO")
Downloading github repo glibiseller/IPO@master
Error in str_detect(url, pattern) :
cannot open file '/R-3.2.0/library/stringi/R/stringi.rdb': No such file or directory
In addition: Warning message:
In str_detect(url, pattern) : restarting interrupted promise evaluation

For those working with R obvious solution, use install.packages("stringi") or
install.packages("foo", dependencies=TRUE)

I think there are issues with R under WINDOWS, especially working with different versions,
sometimes older versions that are required and R32 and R64 bit dependencies. I think its
quite challenging.
T.

Hi,

I am running optimizeXcmsSet() on LC-ToF-MS dataset - only on 5 pooled sample mzML files - and receive the following error after around ~10min, during which 5 independent R processes were successfully initiated:

starting new DoE with:
min_peakwidth: c(1, 5)
max_peakwidth: c(5, 10)
ppm: 20
mzdiff: -0.001
snthresh: 10
noise: 600
prefilter: 3
value_of_prefilter: 100
mzCenterFun: wMean
integrate: 2
fitgauss: FALSE
verbose.columns: FALSE

Error in checkForRemoteErrors(val) :
3 nodes produced errors; first error: cannot open the connection

xcmsSet() on these files works perfectly fine. I am able to run xcmsSet() through both parallel (using PSOCK cluster) and BiocParallel packages (using BPPARAM).

The script for IPO optimisation:

ppparam <- getDefaultXcmsSetStartingParams('centWave')
ppparam$min_peakwidth <- c(1,5)
ppparam$max_peakwidth <-c(5,10)
ppparam$ppm <-20 
ppparam$noise <- 600 
ppparam$integrate <- 2
ppparam$mzdiff <- (-0.001)

opp <- optimizeXcmsSet(files = files, 
                      params = ppparam, 
                      BPPARAM = MulticoreParam(workers = 5),
                      subdir = output_dir)

> sessionInfo()
R version 3.4.0 (2017-04-21)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.5

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] IPO_1.2.0           CAMERA_1.32.0       rsm_2.8             xcms_1.52.0         MSnbase_2.2.0       ProtGenerics_1.8.0 
 [7] mzR_2.10.0          Rcpp_0.12.11        BiocParallel_1.10.1 Biobase_2.36.2      BiocGenerics_0.22.0

loaded via a namespace (and not attached):
 [1] vsn_3.44.0             splines_3.4.0          foreach_1.4.3          Formula_1.2-1          affy_1.54.0           
 [6] stats4_3.4.0           latticeExtra_0.6-28    RBGL_1.52.0            impute_1.50.1          backports_1.1.0       
[11] lattice_0.20-35        limma_3.32.2           digest_0.6.12          RColorBrewer_1.1-2     checkmate_1.8.2       
[16] colorspace_1.3-2       sandwich_2.3-4         htmltools_0.3.6        preprocessCore_1.38.1  Matrix_1.2-10         
[21] plyr_1.8.4             MALDIquant_1.16.2      XML_3.98-1.9           zlibbioc_1.22.0        xtable_1.8-2          
[26] mvtnorm_1.0-6          scales_0.4.1           RANN_2.5.1             affyio_1.46.0          lsmeans_2.26-3        
[31] htmlTable_1.9          tibble_1.3.3           IRanges_2.10.2         ggplot2_2.2.1          TH.data_1.0-8         
[36] nnet_7.3-12            lazyeval_0.2.0         MassSpecWavelet_1.42.0 survival_2.41-3        magrittr_1.5          
[41] estimability_1.2       doParallel_1.0.10      nlme_3.1-131           MASS_7.3-47            foreign_0.8-68        
[46] graph_1.54.0           BiocInstaller_1.26.0   tools_3.4.0            data.table_1.10.4      multcomp_1.4-6        
[51] stringr_1.2.0          S4Vectors_0.14.3       munsell_0.4.3          cluster_2.0.6          pcaMethods_1.68.0     
[56] compiler_3.4.0         mzID_1.14.0            rlang_0.1.1            grid_3.4.0             iterators_1.0.8       
[61] htmlwidgets_0.8        igraph_1.0.1           base64enc_0.1-3        gtable_0.2.0           codetools_0.2-15      
[66] multtest_2.32.0        gridExtra_2.2.1        zoo_1.8-0              knitr_1.16             Hmisc_4.0-3           
[71] stringi_1.1.5          rpart_4.1-11           acepack_1.4.1          coda_0.19-1     

What could went wrong with optimizeXcmsSet()?

Thank you in advance for you help!

Hi,
very nice concept, however the manual/wiki pages should have a better place,
It was hard to find how to use that tool. The R man pages are usually not easy readable
but this excellent page is somehow hidden in a path where nobody looks.

https://github.com/glibiseller/IPO/blob/master/vignette/IPO.Rmd

If that would have a central link or would be on https://github.com/glibiseller/IPO/
that would be great.

Best
Tobias

Hi,
I have met a same installing problem which had been described in #10

library(CAMERA)
library("devtools")
install_github("glibiseller/IPO")
Downloading GitHub repo glibiseller/IPO@master
Installing IPO
Skipping 2 packages not available: CAMERA, xcms
"F:/R/R-3.2.2/bin/i386/R" --no-site-file --no-environ --no-save --no-restore CMD INSTALL "F:/TEMP/RtmpEZ6xui/devtools8e86ff33276/glibiseller-IPO-47aff39"
--library="F:/R/R-3.2.2/library" --install-tests

Error in file(tmp_user_profile, "w") :

How can I solve the problem and install IPO ?

In that issue, you have sent the R-package as zip-file to him, so can you send me the zip-file too?
My email is： [email protected]
Thanks in advance.

Best regards
Jiaq Chen

Hi,
using the examples from the readme and retention correlation
https://github.com/glibiseller/IPO/blob/master/vignette/IPO.Rmd

fails in R (32 bit) with
"Error: cannot allocate vector of size 654.3 Mb"

That is of course not satisfying after waiting 2.2 h to finish.
Simple solution is to run IPO always in R (64 bit).

gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 1135705 30.4 4755841 127.0 5543382 148.1
Vcells 5040773 38.5 231231507 1764.2 305222496 2328.7

see
https://stat.ethz.ch/R-manual/R-devel/library/base/html/Memory-limits.html

Cheers
Tobias

After updating to R-3.2.2, I am unable to install any package. It saying the error:

install.packages("ggplot2")
Installing package into ‘/home/drishti/R/x86_64-redhat-linux-gnu-library/3.2’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
Error in download.file(url, destfile = f, quiet = TRUE) :
unsupported URL scheme
Warning: unable to access index for repository https://mirrors.nics.utk.edu/cran/src/contrib
Warning message:
package ‘ggplot2’ is not available (for R version 3.2.2)

In fact, i tried to install other package, problem is same.
the repository link is accessible from my browser, but i dont know why this error is giving.
Please help.

Hi, one value in the unit tests seems to have slightly changed,
most likely requires just updating the expected value or tolerance.

Running the tests in ‘tests/runTests.R’ failed.
Last 13 lines of output:
  IPO RUnit Tests - 1 test function, 0 errors, 1 failure
  FAILURE in test_ipo: Error in checkEqualsNumeric(unlist(TV)[4], 0.0006312929, tolerance = 1e-06) : 
    Mean relative difference: 0.006195529

Hi, it seems that optimizeRetGroup() is somehow still
using method="obiwarp" even if the retcorGroupParameters
were created with getDefaultRetGroupStartingParams(retcorMethod="loess")

retcorGroupParameters <- getDefaultRetGroupStartingParams(retcorMethod="loess")
retcorGroupParameters$profStep <- 1

resultRetcorGroup <- optimizeRetGroup(xset=optimizedXcmsSetObject, params=retcorGroupParameters, 
                                      nSlaves=1, subdir="rsmDirectory")

results in:

> resultRetcorGroup$best_settings$retcorMethod
[1] "obiwarp"

Yours, Steffen