Comments (13)
a big chunk comes from pypesto.visualize
from pypesto.
import pypesto
currently imports all of those packages:
amici
asttokens
attr
backcall
cffi
cloudpickle
cma
colorama
cyipopt
dateutil
defusedxml
_distutils_hack
dlib
emcee
executing
fides
h5py
idna
IPython
jedi
joblib
jsonschema
kiwisolver
libsbml
llvmlite
matplotlib
mpl_toolkits
mpmath
networkx
nlopt
numba
numpy
packaging
pandas
parso
petab
pexpect
PIL
pkg_resources
prompt_toolkit
psutil
ptyprocess
pure_eval
pycparser
pygments
pynndescent
pyparsing
pyrsistent
pysb
pyswarm
pyswarms
pytz
scipy
seaborn
setuptools
sklearn
stack_data
sympy
tqdm
traitlets
umap
wcwidth
yaml
from pypesto.
that's ofc bad. thought we had fixed that already, but how do all the optimizers and samplers come in again?
from pypesto.
ok .visualize internally refers to at least .sample and .profile (which in turn refers to .optimize) for utility methods. Thus, we need to either
- remove .visualize from standard import (then need to manually import it each time it's used; but this keeps the long import time then);
- or disentangle those utility functions from the heavy submodule imports;
- or lazily import each heavy import only when it's really used (not sure how to tho)
from pypesto.
.visualize internally refers to at least .sample and .profile (which in turn refers to .optimize) for utility methods.
Thanks for checking.
or disentangle those utility functions from the heavy submodule imports;
👍
I didn't check in detail, but I guess a big part can be resolved by moving some imports under if TYPE_CHECKING
and importing the expensive things only inside individual functions.
from pypesto.
I guess actually importing pymc, julia, emcee, nlopt etc only at the beginning of the respective classes' __init__
methods should be both fine and canonical (i.e. only entry point), so that sounds like a feasible solution which would also allow us to keep default imports without requiring to manually import everything.
from pypesto.
Reduced in #946 from 19s to 3.5s on my system. Further improvements could be gained from not importing .visualize by default (2s), and not auto-importing amici+petab, which are main culprits according to
pynndescent 3,923
umap 3,831
seaborn 1,372
petab 0,851
amici 0,835
cma 0,610
pysb 0,505
sklearn 0,451
emcee 0,443
pandas 0,399
numba 0,354
fides 0,302
cyipopt 0,287
pyswarms 0,279
sympy 0,266
IPython 0,258
matplotlib 0,230
joblib 0,180
nlopt 0,155
h5py 0,153
setuptools 0,148
pkg_resources 0,146
scipy 0,140
libsbml 0,138
numpy 0,136
pyswarm 0,130
prompt_toolkit 0,110
networkx 0,105
jedi 0,094
jsonschema 0,089
pyparsing 0,059
parso 0,048
psutil 0,042
mpmath 0,042
stack_data 0,040
attr 0,040
pycparser 0,039
cloudpickle 0,035
pyrsistent 0,034
yaml 0,033
tqdm 0,032
pure_eval 0,032
dlib 0,032
cffi 0,032
_distutils_hack 0,031
pexpect 0,030
packaging 0,030
ptyprocess 0,027
llvmlite 0,027
kiwisolver 0,027
executing 0,027
backcall 0,027
wcwidth 0,026
traitlets 0,026
PIL 0,026
pytz 0,025
defusedxml 0,025
asttokens 0,025
idna 0,023
colorama 0,023
pygments 0,022
mpl_toolkits 0,022
dateutil 0,020
from pypesto.
Reduced in #946 from 19s to 3.5s on my system.
Thanks. Sounds much better already.
Further improvements could be gained from not importing .visualize by default (2s), and not auto-importing amici+petab, which are main culprits according to
In favor of not importing those by default.
from pypesto.
So you would also not import amici+petab by default? Then there'd be two options: Either not auto-import e.g. pypesto.objective.amici
when importing from pypesto.objective
(and pypesto.petab
, and some tidy-ups in pypesto.ensemble
may be needed), or moving each import to lazy level in-class/function. (1) would require some user code adaptations, (2) would look a bit nasty. Preferences?
from pypesto.
So you would also not import amici+petab by default?
Yes, I think it's fair to require from pypesto.petab import PetabImporter
, as pypesto's focus is not necessarily PEtab.
from pypesto.
For the petab submodule that's already the case, it would rather affect from pypesto.objective.amici import AmiciObjective
from pypesto.
For the petab submodule that's already the case, it would rather affect
from pypesto.objective.amici import AmiciObjective
amici should not be imported by default, only when importing the PetabImporter.
With the current state if #946, import takes about 3s for me. Much better, but removing amici (1.5s import time) would be preferable.
from pypesto.
from pypesto.
Related Issues (20)
- Multistart optimization with parameter constraints HOT 12
- Visualize: add description of scale in parameter plots
- Update jax objective to pure callback
- Petab import in try/except clause that fails when amici is not available HOT 3
- SacessOptimizer: Allow specifying multiprocessing start method
- `Ensemble.from_optimization_{history,endpoints}` assumes that `OptimizerResult.id` is always convertible to `int`
- Relative: implement log-normal distribution
- ImportError: cannot import name 'gaussian' from 'scipy.signal' HOT 5
- What's the purpose of `Ensemble.predictions`? HOT 1
- General Discussion regarding Ensemble HOT 7
- Handling of simulation failures in AmiciPredictor
- Frequent timeouts of RTD builds HOT 3
- Waterfall plot error: `order_by_id` and `start_indices` do not work together
- Optimization with conditions specific for observables fails HOT 4
- Frequent timeouts for `mac (3.11)` workflow runs HOT 3
- Check pypesto/sample/geweke_test.py calculate_zscore/spectrum0/spectrum
- Frequent readthedocs timeout HOT 1
- `pyjulia` is no longer maintained
- Roadrunner petab test case 0018 fails on Mac HOT 3
- Lazy loading of optimization results from HDF5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pypesto.