greschd / aiida-optimize Goto Github PK

The max_iter argument of NelderMead does not seem to have any effect.
Also, a quick look at the code seems to suggest that the attribute self.max_iter is set but never used.

Some optimization engines, e.g. NelderMead, implement a check after each iteration to decide whether the optimization converged. The check looks at the difference between the best (two or more) results so far and compare it with a threshold st by the user.
This is perfectly fine if one knows roughly the magnitude of the quantity to optimize in advance, so it can select the threshold accordingly, however in high-throughput calculation this is not always the case. For this it would be nice to implement the possibility to set a FRACTIONAL tolerance instead of the absolute value of it. Of course this would be an optional possibility and retaining the current behavior as standard,

Hi @zooks97 , I got the following exception when I running the convergence test. Do you have any idea how this happened?
I just paste part of the traceback, ask me for more information if it is needed.

....
  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 593, in construct_python_object
    instance = self.make_python_instance(suffix, node, newobj=True)
  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 566, in make_python_instance
    cls = self.find_python_name(suffix, node.start_mark)  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 593, in construct_python_object
    instance = self.make_python_instance(suffix, node, newobj=True)
  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 566, in make_python_instance
    cls = self.find_python_name(suffix, node.start_mark)
  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 538, in find_python_name
    "module %r is not imported" % module_name, mark)
yaml.constructor.ConstructorError: while constructing a Python object
module 'aiida_optimize.engines._result_mapping' is not imported
  in "<unicode string>", line 49, column 10:
          0: !!python/object:aiida_optimize.e ...
  File "/data/CONDA_ENV/aiida-production/lib/python3.7/site-packages/yaml/constructor.py", line 538, in find_python_name
    "module %r is not imported" % module_name, mark)
yaml.constructor.ConstructorError: while constructing a Python object
module 'aiida_optimize.engines._result_mapping' is not imported
  in "<unicode string>", line 49, column 10:
          0: !!python/object:aiida_optimize.e ...

Here is the example file I changed a little on top of your demo https://gist.github.com/unkcpz/87d8564560ecb8066b390dea09ef80d5

When merging the inputs in the OptimizationWorkChain, allow the keys coming from the engine to have the foo.bar:bla.baz syntax as implemented in the AddInputsWorkChain.

Relevant part of the code:

Currently, if one of the evaluate process failed in OptimizationWorkChain, the workflow will stop with exit_code 201. There are lots of scenarios that some failed processes are allowed. So a way to elegantly handle the failed process is needed.

Tagging @greschd @zooks97 for inputs.

The naming of the workchain / process evaluating the function to be optimized "calculation_workchain" is a bit unfortunate, because it's loaded with meaning in the context of AiiDA.

This naming scheme can definitively be improved.

Related: #4.

Off the top of my head: "evaluation_process"

Implement automatically distributing the inputs to different keys if input_key is a list for the NelderMead engine.

see https://travis-ci.org/ltalirz/aiida-registry/builds/590118189#L504-L514

consider version selector here:

or an update to aiida 1.0

I got an error when I using Convergence WorkChain with CreateEvaluationWorkChain to combine a WorkChain and a evaluate calcfunction process. The exception is long so I only paste part of it.

....
  File "/home/unkcpz/pyProject/aiida_core/aiida/orm/utils/serialize.py", line 118, in represent_mapping
    return dumper.represent_mapping(tag, mapping)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/home/unkcpz/pyProject/aiida_core/aiida/orm/utils/serialize.py", line 176, in represent_data
    return super().represent_data(data)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/home/unkcpz/pyProject/aiida_core/aiida/orm/utils/serialize.py", line 176, in represent_data
    return super().represent_data(data)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 52, in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
  File "/data/CONDA_ENV/aiida-testing38/lib/python3.8/site-packages/yaml/representer.py", line 319, in represent_object
    reduce = data.__reduce_ex__(2)
TypeError: __reduce_ex__() takes exactly one argument (0 given)

I also print the data here when the exception happened. data is my WorkChain <class 'aiida_sssp_workflow.workflows.cohesive_energy.CohesiveEnergyWorkChain'> and data.__reduce__ex__(2) is <method '__reduce_ex__' of 'object' objects>.

Tagging @greschd @zooks97 for inputs. I have no idea whats going on 😿

If a child calcjob crashes, it would be nice to have its pk reported in the log. Right now it looks like only the CreateEvaluateWorkChain is reported.

In order to remove duplication, we're removing the development status from the aiida plugin registry and parsing it from the setup.json instead.

Just mentioning that on the code registry, the plugin was listed as "stable", while the setup.json currently indicates "beta" in master (not in develop).

Feel free to close this issue if you're fine with the "beta" status - otherwise please update the setup.json of the master branch.

For example, the calculation_workchain input to OptimizationWorkchain can probably be a generic process now that CalcJob is also a process.

Hi @greschd, I have a genetic algorithm workflow in my pseudopotential optimization plugin https://github.com/unkcpz/aiida-opsp which share and borrow lots of concept from this repository. I am thinking to separete and implementing genetic algorithm here and call aiida-optimize from my plugin. Leave only pseudo related code in aiida-opsp.
But I working on aiida-opsp from develop branch aiida-core and compatible with just released aiida-core 2.0.0.

What is your opinion on processing this? I can either change my code to be compatible with aiida-1.x (then not compatible with 2.x). Or try to migrate aiida-optimize to compatible with aiida-core 2.x.
I go through the code here and find the difficulty of the second option comes from there are also dependencies of aiida-pytest and aiida-tools. I think some fixture in aiida-pytest is now in aiida-core, but still some are special in aiida-pytest. Could you explain a bit about concept of aiida-pytest and your opinion in how it should be updated? I am happy to help and work on that.

Add a helper function to get a (nested) output value from an outputs dictionary using the foo.baz:baz.bla syntax.

The engine should be able to declare an optimization as "failed", and a corresponding exit code should be returned by the OptimizationWorkChain.

The optimization engine should be able to decide if a failed evaluate process means failure of the entire workchain, or if it can continue in some way.

This can probably be done by extending the interface of the engine's update method, but the details need fleshing out.

The _get_optimal_result is fix to give a three elements return value. This limit the chance to give more outputs from the engine inherit from OptimizeWorkChain. How can I add more output into custom engine?

hi @greschd, I use some features added by @zooks97. But it is now in the develop branch. Do you have any plan to release next version? So I do not need to use git install in my ci setting.