caglorithm / mopet Goto Github PK
View Code? Open in Web Editor NEWThe mildly ominous parameter exploration toolkit 🛵 - Run huge simulations on distributed clusters, because why not? 🤯
License: MIT License
The mildly ominous parameter exploration toolkit 🛵 - Run huge simulations on distributed clusters, because why not? 🤯
License: MIT License
If I run an exploration with a specific name and then rerun it, it fails as there exists already a node in the HD5 file.
I would have not expected this behavior, instead I expected that it would overwrite the previous run. E.g., I was changing the parameter combinations and just wanted to rerun it (iterative improvement of the experiment design).
However, there are also reasons why this might be a desired behavior, such as protecting against accidental overwrite. If so, I would propose two things: (1) handle exception and provide user friendly warning and (2) add an optional overwrite
param to the run method that will delete existing node and save the new results.
Run the script two times. Second time will fail as HD5 node already exists.
from mopet import mopet
import numpy as np
def run(params):
return {}
params = {"a": np.arange(0, 1, 0.5)}
ex = mopet.Exploration(run, params, exploration_name="testA")
ex.run()
ex.load_results()
Traceback (most recent call last):
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/synchronization/mopet_test.py", line 11, in <module>
ex.run()
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 129, in run
self._pre_storage_routine()
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 328, in _pre_storage_routine
self._init_hdf()
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 300, in _init_hdf
self.run_group = self.h5file.create_group("/", self.exploration_name)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/file.py", line 940, in create_group
title=title, new=True, filters=filters)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/group.py", line 235, in __init__
super(Group, self).__init__(parentnode, name, _log)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/node.py", line 255, in __init__
parentnode._g_refnode(self, name, validate)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/group.py", line 526, in _g_refnode
% (self._v_pathname, childname))
tables.exceptions.NodeError: group ``/`` already has a child node named ``testA``
Closing remaining open files:exploration.h5...done
In der _read_group_as_dict
Methode wird in der if value.ndim == 0:
Condition ein 0-dim array ggfs in numpy.float64 umgewandelt. Jedoch kann value
auch von Typ int
sein, welcher kein ndim
attribut besitzt. Das führt momentan zu einem nicht abgefangenen AttributeError
.
from mopet import mopet
def run(params):
print(params)
return {
"test": 1
}
params = { "a": [1, 2]}
ex = mopet.Exploration(run, params)
ex.run()
ex.load_results()
Traceback (most recent call last):
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/synchronization/mopet_test.py", line 14, in <module>
ex.load_results()
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 282, in load_results
self._hdf_get_data_from_exploration(exploration_name)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 318, in _hdf_get_data_from_exploration
self._get_run_results(child, run_id)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 328, in _get_run_results
self.results[run_id] = self._read_group_as_dict(hdf_run)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 346, in _read_group_as_dict
if value.ndim == 0:
AttributeError: 'int' object has no attribute 'ndim'
Den Zugriff auf ndim
absichern mit:
if hasattr(value, 'ndim') and value.ndim == 0:`
If we're using mopet on a shared server, it is of high interest to optionally limit the number of CPU cores or GPUs for a given exploration.
Otherwise mopet acts greedy and uses all available cores to full extent and other users are not able to run their own simulations anymore.
Example: we have a server with 64 CPU cores. For my exploration it is sufficient to use 12 cores and accept longer waiting times.
Ray offers this kind of functionality and can be configured in ray.init()
.
We just have to offer these parameters in the Exploration as well.
It's also nice to have if you run simulations on your local machine as you can reserve some CPU cores for other processes so that you can continue to work on other tasks.
It would be neat to have a feature that allows the user to extend the search space, as the search goes on.
Installing mopet
in Python 3.8.x environment is currently not possible as the project's python requirement is set to >= 3.6, < 3.8.
pip install mopet
ERROR: Could not find a version that satisfies the requirement mopet (from versions: none)
ERROR: No matching distribution found for mopet
The error is thrown when ray's ObjectStore is full. This problem is described in FAQ of ray.
Proposed solution is to enable LRU fallback:
ray.init(lru_evict=True)
LRU strategy was used in previous versions of ray, FAQ suggests to enable it again for applications that keep references to all objects ever created (such as iPython/JupyterNotebook).
As this library is focused mainly on scientific computing where many use Jupyter Nootebok, I would suggest to set enable lru_evict=True
per default and allow in the Exploration.__init__()
to disable it.
When calling .run(a = 0)
, they should be passed on to the exploration function a la
def function(params, kwargs):
a = kwargs['a']
b = params['b']
if a * b > 0:
return {"good" : True}
else:
return {"good" : False}
Travis.org is shutting down and Github Actions has been a lot more reliable and faster in my experience
Problem is that 0 dim arrays get always converted to float without checking the type of the given value.
import numpy as np
from mopet import mopet
def run(params):
return {
"test": True,
"test2": False
}
ex = mopet.Exploration(run, {"a": np.arange(0, 1, 0.5)})
ex.run()
ex.load_results()
print(ex.df)
Output:
a test test2
0 0 1.0 0.0
1 0.5 1.0 0.0
Values for test and test2 changed from bool type to numpy float type.
Currently, if you try to load results of an exploration that has not been stored in the HDF, a tables.exceptions.NoSuchNodeError
gets thrown that is not handled in mopet
and terminates the python script.
I would suggest to catch the exception and log an error message that points out that the Exploration with the specified exploration_name
has not been stored in the HDF file.
We could return either an empty DataFrame or throw a custom exception such as ExplorationNotFound
holding the error message. If we throw an exception we can add it to the method docs so that users can react to it.
from mopet import mopet
import numpy as np
def run(params):
return {}
params = {"a": np.arange(0, 1, 0.5)}
ex = mopet.Exploration(run, params, exploration_name="testA")
ex.load_results()
Traceback (most recent call last):
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/synchronization/mopet_test.py", line 12, in <module>
ex.load_results()
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 184, in load_results
self._load_all_results(exploration_name, all=all)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/mopet/mopet.py", line 446, in _load_all_results
hdf_runs_group = self.h5file.get_node("/" + self.exploration_name, "runs")
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/file.py", line 1642, in get_node
node = self._get_node(nodepath)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/file.py", line 1590, in _get_node
node = self._node_manager.get_node(nodepath)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/file.py", line 432, in get_node
node = self.node_factory(key)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/group.py", line 1178, in _g_load_child
node_type = self._g_check_has_child(childname)
File "/Users/lrebscher/git/Synchronization-by-Uncorrelated-Noise/venv/lib/python3.7/site-packages/tables/group.py", line 395, in _g_check_has_child
% (self._v_pathname, name))
tables.exceptions.NoSuchNodeError: group ``/`` does not have a child named ``/testA/runs``
Closing remaining open files:exploration.h5...done
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.