emoryuniversitytheoreticalbiophysics / sirisaac Goto Github PK

Due to changes in the module structure and names, older pickled fittingProblems (the current preferred format) won't load using simplePickle.

I think this can be fixed with some fancy footwork using sys.modules.

Loading older versions of saved SloppyCellFittingModels (as in simpleExample.ipynb) on different machines can produce errors when attempting to evaluate the models. This appears to be due to C code saved within the SloppyCell network that needs to be updated.

A quick fix appears to be to force the SloppyCell network to recompile:

n = model.net
n._last_structure = ()
n.compile()

It's time to make the transition...

When functions using parallel capabilities fail, their error messages are not visible. For instance, running p.fitAll() in simpleExample.py uses fittingProblem.generateEnsemble_pypar, which calls mpirun to run generateEnsembleParallel.py in parallel using pypar. When something goes wrong, the error message only refers generically to generateEnsembleParallel. We should be able to redirect error messages from mpirun to display to the user.

fittingProblem.generateEnsemble_pypar tries to do this already by redirecting stdout and stderr to a file, but for some reason the file always remains empty.

Warnings from SloppyCell C compilation, which are typically not useful, should be hidden by default. At the moment they appear to be going to stdout?

running the jupyter notebook, at the "Analyze the selected model" stage, after p.plotBestModelResults(plotInitialConditions=True,indices=range(10));
I get the following error
WARNING:daskr: WARNING:daskr:The matrix of partial derivatives is singular, with the use of the direct methods. Some of your equations may be redundant. DDASKR cannot solve the problem as stated. It is possible that the redundant equations could be removed, and then DDASKR could solve the problem. It is also possible that a solution to your problem either does not exist or is not unique. (IDID=-8) WARNING:ReactionNetworks.Dynamics:Integration ended prematurely in network PowerLaw_Network on node 0.

I am running simpleExample.py in iPython after installing SirIsaac and SloppyCell in python2.7 (anaconda). An error is thrown stating that the Optimization module has not attribute fmin_lm. (line ~1925 of FittingProblem.py). It appears that this method is part of the lmopt module of the SloppyCell installation that I downloaded from CVS. How to fix?

Hi,Professor Daniels,

It seems that i bump into some unknown mistake after i run the model and tried to display the result with p.plotBestModelResults(plotInitialConditions=True,indices=range(10));

I didn't get any plot back. Instead, it returned with

maxLogLikelihoodName: Warning: Only 9 of 9 fits have been performed.
maxLogLikelihoodName: bestIndex > maxIndex.  Returning None.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-14-204fc9c0e868> in <module>
      1 plt.figure(figsize=(20,2))
----> 2 a = p.plotBestModelResults(plotInitialConditions=True,indices=range(10));
      3 
      4 for i in range(len(a[0])):
      5     a[0][i].plot(RNAtheoreticalData[0], RNAtheoreticalData[1+i],linewidth=1)

~/SirIsaac/SirIsaac/fittingProblem.py in plotBestModelResults(self, modelName, maxIndex, verbose, **kwargs)
    750         m = self.getBestModel(modelName=modelName,maxIndex=maxIndex,
    751             verbose=verbose)
--> 752         return self.plotModelResults(m,**kwargs)
    753 
    754 

~/SirIsaac/SirIsaac/fittingProblem.py in plotModelResults(self, model, filename, indices, plotFittingData, outOfSampleData, **kwargs)
    711 
    712         # plot model results
--> 713         plots = m.plotResults(fittingData,indepParamsList,
    714             plotFittingData=plotFittingData,
    715             outOfSampleData=outData,**kwargs)

AttributeError: 'NoneType' object has no attribute 'plotResults'

<Figure size 1440x144 with 0 Axes>

It seems that i run

p.fitAll()
fittingProblem.save(p,'mRNAconcentrationS2.pkl')

successfully.

Could you give me any hint on what may go wrong and how to fix it?

Thank you for your time!

Sincerely,
Yu

A detailed code and error file is attached here:
mRNA-sigmoidal.pdf

Due to what appears to be a limitation of SloppyCell's Network.calculate, including data for given independent parameters that consists of a single timepoint at t=0 produces a "list index out of range" error from Dynamics.integrate.

For now we will disallow this, as it seems to be somewhat unlikely. A hacky way to get around it is to change lonely t=0 data to, e.g., t=10^-40. In the future we may want to revise this in SloppyCell.

Hi, I tested it with my own dataset and I got some errors. I used the same parameter as in the simpleExample.ipynb:

complexityStepsize = 2 # increase complexity with steps of size 2
complexityMax = 25 # don't try models with complexity > 25
complexityList = range(0,complexityMax,complexityStepsize) 
totalSteps = 1e3
keepSteps = 10
seeds = (1,1) # use a fixed random seed
ensTemperature = 100.
ensGen = fittingProblem.EnsembleGenerator( totalSteps, keepSteps,
    temperature=ensTemperature, seeds=seeds )
avegtol = 1e-2
maxiter = 100
priorSigma = 3.
numprocs = 3
N = len(indepParamsList)

Here is how my data looks like:

indepParamsList[:10 ]
[[1.0], [1.0], [1.0], [1.0], [1.0], [1.0], [1.0], [1.0], [1.0], [2.0]]

sirIsaacData[:3]
[{'x1': {3.0: (1.2346039866581013, 0.19381314496763835)},
  'x2': {3.0: (5.223171191926012, 0.5260055984594344)},
  'x3': {3.0: (7.072974151579377, 0.704560417875636)}},
 {'x1': {3.5: (1.9176442151604471, 0.7198853888751312)},
  'x2': {3.5: (3.871928103727898, 0.41154764318844306)},
  'x3': {3.5: (6.751740435788017, 0.842283495315975)}},
 {'x1': {4.5: (1.8585518592890489, 0.9657200975255507)},
  'x2': {4.5: (4.711265598482662, 0.5967139128320695)},
  'x3': {4.5: (9.877799185687941, 1.1240765611335537)}}]

And here is my error message:

error.txt

Fitting of the example data in simpleExample.py is now slower, taking ~10 hours on my laptop instead of "a few" hours claimed in the original. Comparing minimization times stored in minimizationTimeSecondsList, they are indeed roughly 5x longer. We may need to check that integration is actually being done using compiled C code or whether there is a fallback to python code.

Diagnostic timing information is saved when running fitting (in minimizationTimeSecondsList and ensTimeSecondsList) but we may want to revisit this and make it more interpretable—e.g. the typical time it takes to integrate the equations once. Also, the ensemble time appears to be incorrect when running in parallel.

It will be useful for future development to include a test suite. Let's start by covering some of the most frequently used functionality.

Trying to run simpleExample

`---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
in
1 import scipy, pylab
----> 2 import FittingProblem
3 get_ipython().run_line_magic('matplotlib', 'inline')

/Library/Python/2.7/site-packages/SirIsaac/FittingProblem.py in
14 SIRISAACDIR = '/Library/Python/2.7/site-packages/SirIsaac/'
15
---> 16 from SloppyCell.ReactionNetworks import *
17 import PowerLawNetwork
18 reload(PowerLawNetwork)

ModuleNotFoundError: No module named 'SloppyCell'`

Not sure if this is an installation issue, since import SloppyCell works in ipython.

I'm also trying to run it in python 3 and have change all prints and Exceptions, and not sure if the above is python 3 related?

EDIT: python SloppyCellTest.py ran fine in python 2, but obviously with python3 it fails for the above reason

Thanks for fixing the fmin_lm bug. There is another bug in your simple example - the following line (894) of FittingProblem:
plots = m.plotResults(fittingData,indepParamsList,
plotFittingData=plotFittingData,**kwargs)

model does not have a method plotResults - seems this should be an instance of FittingProblem.PowerLawFittingProblem.

thanks

The original Sir Isaac code performs model selection using a modified Bayesian information criterion. This makes sense in cases with limited data, but in many applications, we have enough data that it makes more sense to use cross validation (fit to a subset of the data and select a model based on its out-of-sample performance).

In particular, this is what we did in applying Sir Isaac to C. elegans behavior data. (Note to self: This was implemented in gatherResults.goodnessOfFit_model.) We should make this easier to do within Sir Isaac.

This could also be an excuse to automate what has become the usual model fitting workflow, in which we first fit to a very small subset of data and then gradually add data, refitting at each step.

I'm trying to run the simpleExample. When I set numprocs equal to 1, everything works fine. However, when I set numprocs to be greater than 1, an exception occurred.

I run it in a Docker container.
The system is ubuntu 16.04.
I use anaconda to manage python, and Created a Python 2.7 environment called SirIssac.

I don't know how to fix it.

Here is the message:

Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.

mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

Process name: [[57214,1],0]
Exit code: 2

Exception Traceback (most recent call last)
in ()
2
3 time_start=time.time()
----> 4 p.fitAll()
5 time_end=time.time()
6

/data/SirIsaac-master/FittingProblem.pyc in fitAll(self, **kwargs)
852
853 def fitAll(self,**kwargs):
--> 854 FittingProblem.fitAll(self,**kwargs)
855 # we also want to save the convergence information in a
856 # convenient location:

/data/SirIsaac-master/FittingProblem.pyc in fitAll(self, usePreviousParams, fitPerfectModel, resume, maxNumFit, **kwargs)
224 fittingModel.fitToData(self.fittingData,self.indepParamsList,
225 otherStartingPoint=smallerBestParams,
--> 226 fittingDataDerivs=fittingDataDerivs,**kwargs)
227
228 if not hasattr(self,'fittingDataDerivs'):

/data/SirIsaac-master/FittingProblem.pyc in fitToData(self, fittingData, indepParamsList, _unclampedSpeciesID, otherStartingPoint, fittingDataDerivs, createEnsemble)
1623 if self.numprocs > 1:
1624 ens,ratio = self.ensGen.generateEnsemble_pypar(self.numprocs,
-> 1625 dataModel,initialParameters,verbose=self.verbose)
1626 else:
1627 ens,ratio = self.ensGen.generateEnsemble(dataModel,

/data/SirIsaac-master/FittingProblem.pyc in generateEnsemble_pypar(self, numprocs, dataModel, initialParameters, returnCosts, scaleByDOF, verbose)
2629 os.remove(prefix+"stdout.txt")
2630 raise Exception, "generateEnsemble_pypar:"
-> 2631 + " error in generateEnsembleParallel.py"
2632
2633 return output

Exception: generateEnsemble_pypar: error in generateEnsembleParallel.py

Hi, Professor Daniels,

I am trying to perform modelling mRMA network data with sigmoidal class. I created a fitting problem with 100 data point, after I run my program at terminal for approximately 100 minutes, the program ended with an overflow error:

***STDERR***
b"<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n<string>:1: SyntaxWarning: 'int' object is not subscriptable; perhaps you missed a comma?\n"
localFitParallel: worker 1 working on work_array 0...
localFitParallel: worker 6 working on work_array 5...
localFitParallel: worker 7 working on work_array 6...
localFitParallel: worker 3 working on work_array 2...
localFitParallel: worker 8 working on work_array 7...
localFitParallel: worker 4 working on work_array 3...
localFitParallel: worker 9 working on work_array 8...
localFitParallel: worker 2 working on work_array 1...
WARNING:ReactionNetworks.Components:Failed to import dynamically compiled C module CTSN_0_wirelessprv_10_194_237_143_near_illinois_edu_78849_8762937861_48443 for network CTSN.
WARNING:ReactionNetworks.Components:No module named 'CTSN_0_wirelessprv_10_194_237_143_near_illinois_edu_78849_8762937861_48443'
localFitParallel: worker 5 working on work_array 4...
capi_return is NULL
Call-back cb_jac_in_daskr__user__routines failed.
Fatal Python error: F2PySwapThreadLocalCallbackPtr: PyLong_AsVoidPtr failed
Python runtime state: initialized
Traceback (most recent call last):
  File "<string>", line 6, in ddaskr_jac
  File "<string>", line 38, in dres_dc_function
OverflowError: (34, 'Result too large')

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 48442 RUNNING AT wirelessprv-10-194-237-143.near.illinois.edu
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Abort trap: 6 (signal 6)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

Traceback (most recent call last):
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/fittingProblem.py", line 1823, in localFitToData_parallel
    output = load(outputFilename)
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/simplePickle.py", line 24, in load
    fin = io.open(filename,'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'temporary_94132_localFitToData_parallel_output.data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/penguinaugustus/Desktop/mrna_sig_terminal/mRNA-sigm.py", line 195, in <module>
    p.fitAll()
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/fittingProblem.py", line 979, in fitAll
    FittingProblem.fitAll(self,**kwargs)
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/fittingProblem.py", line 216, in fitAll
    fittingModel.fitToData(self.fittingData,self.indepParamsList,         \
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/fittingProblem.py", line 1720, in fitToData
    outputDict = self.localFitToData_parallel(self.numprocs,fittingData,
  File "/Users/penguinaugustus/SirIsaac/SirIsaac/fittingProblem.py", line 1832, in localFitToData_parallel
    raise Exception("localFitToData_parallel:"                            \
Exception: localFitToData_parallel: error in localFitParallel.py

########### EXT COMPILER OPTIMIZATION ###########
Platform      : 
  Architecture: x64
  Compiler    : gcc

CPU baseline  : 
  Requested   : 'min'
  Enabled     : SSE SSE2 SSE3
  Flags       : -msse -msse2 -msse3
  Extra checks: none

CPU dispatch  : 
  Requested   : 'max -xop -fma4'
  Enabled     : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
  Generated   : none
CCompilerOpt._cache_write[796] : write cache to path -> /Users/penguinaugustus/Desktop/mrna_sig_terminal/build/temp.macosx-10.9-x86_64-3.8/ccompiler_opt_cache_ext.py

Could you give me any hint on what may go wrong and how to fix it?

Thank you for your time!

Sincerely,
Yu

As pointed out by @IntegerLuoHua (issue #17), default parallelization using MPI currently requires manually setting SIRISAACDIR in FittingProblem.py. This should be fixed in a future release to be automatically set during installation.